MRFP: Learning Generalizable Semantic Segmentation from Sim-2-Real with Multi-Resolution Feature Perturbation

Deep neural networks have shown exemplary performance on semantic sceneunderstanding tasks on source domains, but due to the absence of stylediversity during training, enhancing performance on unseen target domains usingonly single source domain data remains a challenging task. Generation ofsimulated data is a feasible alternative to retrieving large style-diversereal-world datasets as it is a cumbersome and budget-intensive process.However, the large domain-specfic inconsistencies between simulated andreal-world data pose a significant generalization challenge in semanticsegmentation. In this work, to alleviate this problem, we propose a novelMultiResolution Feature Perturbation (MRFP) technique to randomizedomain-specific fine-grained features and perturb style of coarse features. Ourexperimental results on various urban-scene segmentation datasets clearlyindicate that, along with the perturbation of style-information, perturbationof fine-feature components is paramount to learn domain invariant robustfeature maps for semantic segmentation models. MRFP is a simple andcomputationally efficient, transferable module with no additional learnableparameters or objective functions, that helps state-of-the-art deep neuralnetworks to learn robust domain invariant features for simulation-to-realsemantic segmentation.