Wavelet-based Unsupervised Label-to-Image Translation

Semantic Image Synthesis (SIS) is a subclass of image-to-image translationwhere a semantic layout is used to generate a photorealistic image.State-of-the-art conditional Generative Adversarial Networks (GANs) need a hugeamount of paired data to accomplish this task while generic unpairedimage-to-image translation frameworks underperform in comparison, because theycolor-code semantic layouts and learn correspondences in appearance instead ofsemantic content. Starting from the assumption that a high quality generatedimage should be segmented back to its semantic layout, we propose a newUnsupervised paradigm for SIS (USIS) that makes use of a self-supervisedsegmentation loss and whole image wavelet based discrimination. Furthermore, inorder to match the high-frequency distribution of real images, a novelgenerator architecture in the wavelet domain is proposed. We test ourmethodology on 3 challenging datasets and demonstrate its ability to bridge theperformance gap between paired and unpaired models.