Progressive Semantic-Aware Style Transformation for Blind Face Restoration

Face restoration is important in face image processing, and has been widelystudied in recent years. However, previous works often fail to generateplausible high quality (HQ) results for real-world low quality (LQ) faceimages. In this paper, we propose a new progressive semantic-aware styletransformation framework, named PSFR-GAN, for face restoration. Specifically,instead of using an encoder-decoder framework as previous methods, we formulatethe restoration of LQ face images as a multi-scale progressive restorationprocedure through semantic-aware style transformation. Given a pair of LQ faceimage and its corresponding parsing map, we first generate a multi-scalepyramid of the inputs, and then progressively modulate different scale featuresfrom coarse-to-fine in a semantic-aware style transfer way. Compared withprevious networks, the proposed PSFR-GAN makes full use of the semantic(parsing maps) and pixel (LQ images) space information from different scales ofinput pairs. In addition, we further introduce a semantic aware style losswhich calculates the feature style loss for each semantic region individuallyto improve the details of face textures. Finally, we pretrain a face parsingnetwork which can generate decent parsing maps from real-world LQ face images.Experiment results show that our model trained with synthetic data can not onlyproduce more realistic high-resolution results for synthetic LQ inputs and butalso generalize better to natural LQ face images compared with state-of-the-artmethods. Codes are available at https://github.com/chaofengc/PSFRGAN.