Image Processing Using Multi-Code GAN Prior

Despite the success of Generative Adversarial Networks (GANs) in imagesynthesis, applying trained GAN models to real image processing remainschallenging. Previous methods typically invert a target image back to thelatent space either by back-propagation or by learning an additional encoder.However, the reconstructions from both of the methods are far from ideal. Inthis work, we propose a novel approach, called mGANprior, to incorporate thewell-trained GANs as effective prior to a variety of image processing tasks. Inparticular, we employ multiple latent codes to generate multiple feature mapsat some intermediate layer of the generator, then compose them with adaptivechannel importance to recover the input image. Such an over-parameterization ofthe latent space significantly improves the image reconstruction quality,outperforming existing competitors. The resulting high-fidelity imagereconstruction enables the trained GAN models as prior to many real-worldapplications, such as image colorization, super-resolution, image inpainting,and semantic manipulation. We further analyze the properties of the layer-wiserepresentation learned by GAN models and shed light on what knowledge eachlayer is capable of representing.