Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net

Convolutional neural networks (CNNs) have achieved great successes in manycomputer vision problems. Unlike existing works that designed CNN architecturesto improve performance on a single task of a single domain and notgeneralizable, we present IBN-Net, a novel convolutional architecture, whichremarkably enhances a CNN's modeling ability on one domain (e.g. Cityscapes) aswell as its generalization capacity on another domain (e.g. GTA5) withoutfinetuning. IBN-Net carefully integrates Instance Normalization (IN) and BatchNormalization (BN) as building blocks, and can be wrapped into many advanceddeep networks to improve their performances. This work has three keycontributions. (1) By delving into IN and BN, we disclose that IN learnsfeatures that are invariant to appearance changes, such as colors, styles, andvirtuality/reality, while BN is essential for preserving content relatedinformation. (2) IBN-Net can be applied to many advanced deep architectures,such as DenseNet, ResNet, ResNeXt, and SENet, and consistently improve theirperformance without increasing computational cost. (3) When applying thetrained networks to new domains, e.g. from GTA5 to Cityscapes, IBN-Net achievescomparable improvements as domain adaptation methods, even without using datafrom the target domain. With IBN-Net, we won the 1st place on the WAD 2018Challenge Drivable Area track, with an mIoU of 86.18%.