Detection, Pose Estimation and Segmentation for Multiple Bodies: Closing the Virtuous Circle

Human pose estimation methods work well on isolated people but struggle withmultiple-bodies-in-proximity scenarios. Previous work has addressed thisproblem by conditioning pose estimation by detected bounding boxes orkeypoints, but overlooked instance masks. We propose to iteratively enforcemutual consistency of bounding boxes, instance masks, and poses. The introducedBBox-Mask-Pose (BMP) method uses three specialized models that improve eachother's output in a closed loop. All models are adapted for mutualconditioning, which improves robustness in multi-body scenes. MaskPose, a newmask-conditioned pose estimation model, is the best among top-down approacheson OCHuman. BBox-Mask-Pose pushes SOTA on OCHuman dataset in all three tasks -detection, instance segmentation, and pose estimation. It also achieves SOTAperformance on COCO pose estimation. The method is especially good in sceneswith large instances overlap, where it improves detection by 39% over thebaseline detector. With small specialized models and faster runtime, BMP is aneffective alternative to large human-centered foundational models. Code andmodels are available on https://MiraPurkrabek.github.io/BBox-Mask-Pose.