RFSR: Improving ISR Diffusion Models via Reward Feedback Learning

Generative diffusion models (DM) have been extensively utilized in imagesuper-resolution (ISR). Most of the existing methods adopt the denoising lossfrom DDPMs for model optimization. We posit that introducing reward feedbacklearning to finetune the existing models can further improve the quality of thegenerated images. In this paper, we propose a timestep-aware training strategywith reward feedback learning. Specifically, in the initial denoising stages ofISR diffusion, we apply low-frequency constraints to super-resolution (SR)images to maintain structural stability. In the later denoising stages, we usereward feedback learning to improve the perceptual and aesthetic quality of theSR images. In addition, we incorporate Gram-KL regularization to alleviatestylization caused by reward hacking. Our method can be integrated into anydiffusion-based ISR model in a plug-and-play manner. Experiments show that ISRdiffusion models, when fine-tuned with our method, significantly improve theperceptual and aesthetic quality of SR images, achieving excellent subjectiveresults. Code: https://github.com/sxpro/RFSR