Object-Centric Multi-Task Learning for Human Instances

Human is one of the most essential classes in visual recognition tasks suchas detection, segmentation, and pose estimation. Although much effort has beenput into individual tasks, multi-task learning for these three tasks has beenrarely studied. In this paper, we explore a compact multi-task networkarchitecture that maximally shares the parameters of the multiple tasks viaobject-centric learning. To this end, we propose a novel query design to encodethe human instance information effectively, called human-centric query (HCQ).HCQ enables for the query to learn explicit and structural information of humanas well such as keypoints. Besides, we utilize HCQ in prediction heads of thetarget tasks directly and also interweave HCQ with the deformable attention inTransformer decoders to exploit a well-learned object-centric representation.Experimental results show that the proposed multi-task network achievescomparable accuracy to state-of-the-art task-specific models in humandetection, segmentation, and pose estimation task, while it consumes lesscomputational costs.