Learning Warped Guidance for Blind Face Restoration

This paper studies the problem of blind face restoration from anunconstrained blurry, noisy, low-resolution, or compressed image (i.e.,degraded observation). For better recovery of fine facial details, we modifythe problem setting by taking both the degraded observation and a high-qualityguided image of the same identity as input to our guided face restorationnetwork (GFRNet). However, the degraded observation and guided image generallyare different in pose, illumination and expression, thereby making plain CNNs(e.g., U-Net) fail to recover fine and identity-aware facial details. To tacklethis issue, our GFRNet model includes both a warping subnetwork (WarpNet) and areconstruction subnetwork (RecNet). The WarpNet is introduced to predict flowfield for warping the guided image to correct pose and expression (i.e., warpedguidance), while the RecNet takes the degraded observation and warped guidanceas input to produce the restoration result. Due to that the ground-truth flowfield is unavailable, landmark loss together with total variationregularization are incorporated to guide the learning of WarpNet. Furthermore,to make the model applicable to blind restoration, our GFRNet is trained on thesynthetic data with versatile settings on blur kernel, noise level,downsampling scale factor, and JPEG quality factor. Experiments show that ourGFRNet not only performs favorably against the state-of-the-art image and facerestoration methods, but also generates visually photo-realistic results onreal degraded facial images.