Template-Based Automatic Search of Compact Semantic Segmentation Architectures

Automatic search of neural architectures for various vision and naturallanguage tasks is becoming a prominent tool as it allows to discoverhigh-performing structures on any dataset of interest. Nevertheless, on moredifficult domains, such as dense per-pixel classification, current automaticapproaches are limited in their scope - due to their strong reliance onexisting image classifiers they tend to search only for a handful of additionallayers with discovered architectures still containing a large number ofparameters. In contrast, in this work we propose a novel solution able to findlight-weight and accurate segmentation architectures starting from only fewblocks of a pre-trained classification network. To this end, we progressivelybuild up a methodology that relies on templates of sets of operations, predictswhich template and how many times should be applied at each step, while alsogenerating the connectivity structure and downsampling factors. All thesedecisions are being made by a recurrent neural network that is rewarded basedon the score of the emitted architecture on the holdout set and trained usingreinforcement learning. One discovered architecture achieves 63.2% mean IoU onCamVid and 67.8% on CityScapes having only 270K parameters. Pre-trained modelsand the search code are available athttps://github.com/DrSleep/nas-segm-pytorch.