DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Images

Optical satellite images are a critical data source; however, cloud coveroften compromises their quality, hindering image applications and analysis.Consequently, effectively removing clouds from optical satellite images hasemerged as a prominent research direction. While recent advancements in cloudremoval primarily rely on generative adversarial networks, which may yieldsuboptimal image quality, diffusion models have demonstrated remarkable successin diverse image-generation tasks, showcasing their potential in addressingthis challenge. This paper presents a novel framework called DiffCR, whichleverages conditional guided diffusion with deep convolutional networks forhigh-performance cloud removal for optical satellite imagery. Specifically, weintroduce a decoupled encoder for conditional image feature extraction,providing a robust color representation to ensure the close similarity ofappearance information between the conditional input and the synthesizedoutput. Moreover, we propose a novel and efficient time and condition fusionblock within the cloud removal model to accurately simulate the correspondencebetween the appearance in the conditional image and the target image at a lowcomputational cost. Extensive experimental evaluations on two commonly usedbenchmark datasets demonstrate that DiffCR consistently achievesstate-of-the-art performance on all metrics, with parameter and computationalcomplexities amounting to only 5.1% and 5.4%, respectively, of those previousbest methods. The source code, pre-trained models, and all the experimentalresults will be publicly available at https://github.com/XavierJiezou/DiffCRupon the paper's acceptance of this work.