Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer

Human body part parsing, or human semantic part segmentation, is fundamentalto many computer vision tasks. In conventional semantic segmentation methods,the ground truth segmentations are provided, and fully convolutional networks(FCN) are trained in an end-to-end scheme. Although these methods havedemonstrated impressive results, their performance highly depends on thequantity and quality of training data. In this paper, we present a novel methodto generate synthetic human part segmentation data using easily-obtained humankeypoint annotations. Our key idea is to exploit the anatomical similarityamong human to transfer the parsing results of a person to another person withsimilar pose. Using these estimated results as additional training data, oursemi-supervised model outperforms its strong-supervised counterpart by 6 mIOUon the PASCAL-Person-Part dataset, and we achieve state-of-the-art humanparsing results. Our approach is general and can be readily extended to otherobject/animal parsing task assuming that their anatomical similarity can beannotated by keypoints. The proposed model and accompanying source code areavailable at https://github.com/MVIG-SJTU/WSHP