Improving Facial Attribute Prediction using Semantic Segmentation

Attributes are semantically meaningful characteristics whose applicabilitywidely crosses category boundaries. They are particularly important indescribing and recognizing concepts where no explicit training example isgiven, \textit{e.g., zero-shot learning}. Additionally, since attributes arehuman describable, they can be used for efficient human-computer interaction.In this paper, we propose to employ semantic segmentation to improve facialattribute prediction. The core idea lies in the fact that many facialattributes describe local properties. In other words, the probability of anattribute to appear in a face image is far from being uniform in the spatialdomain. We build our facial attribute prediction model jointly with a deepsemantic segmentation network. This harnesses the localization cues learned bythe semantic segmentation to guide the attention of the attribute prediction tothe regions where different attributes naturally show up. As a result of thisapproach, in addition to recognition, we are able to localize the attributes,despite merely having access to image level labels (weak supervision) duringtraining. We evaluate our proposed method on CelebA and LFWA datasets andachieve superior results to the prior arts. Furthermore, we show that in thereverse problem, semantic face parsing improves when facial attributes areavailable. That reaffirms the need to jointly model these two interconnectedtasks.