PHNet: Patch-based Normalization for Portrait Harmonization

A common problem for composite images is the incompatibility of theirforeground and background components. Image harmonization aims to solve thisproblem, making the whole image look more authentic and coherent. Most existingsolutions predict lookup tables (LUTs) or reconstruct images, utilizing variousattributes of composite images. Recent approaches have primarily focused onemploying global transformations like normalization and color curve renderingto achieve visual consistency, and they often overlook the importance of localvisual coherence. We present a patch-based harmonization network consisting ofnovel Patch-based normalization (PN) blocks and a feature extractor based onstatistical color transfer. Extensive experiments demonstrate the network'shigh generalization capability for different domains. Our network achievesstate-of-the-art results on the iHarmony4 dataset. Also, we created a new humanportrait harmonization dataset based on FFHQ and checked the proposed method toshow the generalization ability by achieving the best metrics on it. Thebenchmark experiments confirm that the suggested patch-based normalizationblock and feature extractor effectively improve the network's capability toharmonize portraits. Our code and model baselines are publicly available.