Student: “Giorgos Petsangourakis”
Program: “Data Science and Information Technologies”
Title: “Semantically-Guided Image Synthesis: Augmenting VAE Decoders with Foundation Model Representations”
Abstract: Variational Autoencoders (VAEs) serve as the critical first stage in modern latent generative modeling, yet they often face an inherent trade-off between latent compression and reconstruction fidelity.
Moreover, standard VAE architectures frequently struggle to recover fine-grained textures and complex semantic structures from the low-dimensional latent bottleneck z.
In this work, we propose an architectural enhancement to the VAE decoder that leverages high-level semantic information from Vision Foundation Models (VFMs). Central to our approach is the adaptation of the lightweight
convolutional semantic compressor introduced in the REGLUE framework. While REGLUE utilizes this module to entangle semantic features within a diffusion process, we extend its application to the VAE decoder to non-linearly aggregate multi-layer DINO VFM features into a spatially structured, low-dimensional representation that directly conditions the VAE decoder.
By injecting these “semantic maps” into the decoder’s upsampling blocks during finetuning, we provide the model with a structural signal that supplements the information in the primary latent space. Our experimental results demonstrate that this semantically-guided decoding strategy outperforms baseline VAEs across key metrics.
Most notably, we observe a substantial improvement in rFID (Reconstruction FID), indicating a superior ability to synthesize images that are both distributionally and structurally faithful to the ground truth. Furthermore, improvements in standard generative FID suggest that the augmented decoder provides a more robust foundation for downstream synthesis tasks.
Our findings highlight that the non-linear compression of VFM features is not only beneficial for diffusion backbones but is a transformative tool for overcoming the fundamental reconstruction bottlenecks of autoencoder architectures.

Date/Time: July 7, 2026 – 13:00 PM.
Examination Committee:
Dr. Bill Psomas
Dr. Stavros Perantonis
Dr. Ioannis Kakogeorgiou

Presentation link: https://meet.google.com/vsy-kkih-qah

—
Bill Psomas
MSCA Postdoctoral Fellow
VRG, FEE, Czech Technical University in Prague
Karlovo nám. 13, 120 00 Nové Město, Czech Republic