Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks

In monocular video 3D multi-person pose estimation, inter-person occlusionand close interactions can cause human detection to be erroneous andhuman-joints grouping to be unreliable. Existing top-down methods rely on humandetection and thus suffer from these problems. Existing bottom-up methods donot use human detection, but they process all persons at once at the samescale, causing them to be sensitive to multiple-persons scale variations. Toaddress these challenges, we propose the integration of top-down and bottom-upapproaches to exploit their strengths. Our top-down network estimates humanjoints from all persons instead of one in an image patch, making it robust topossible erroneous bounding boxes. Our bottom-up network incorporateshuman-detection based normalized heatmaps, allowing the network to be morerobust in handling scale variations. Finally, the estimated 3D poses from thetop-down and bottom-up networks are fed into our integration network for final3D poses. Besides the integration of top-down and bottom-up networks, unlikeexisting pose discriminators that are designed solely for single person, andconsequently cannot assess natural inter-person interactions, we propose atwo-person pose discriminator that enforces natural two-person interactions.Lastly, we also apply a semi-supervised method to overcome the 3D ground-truthdata scarcity. Our quantitative and qualitative evaluations show theeffectiveness of our method compared to the state-of-the-art baselines.