Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation

In this paper, a novel Diffusion-based 3D Pose estimation (D3DP) method withJoint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposedfor probabilistic 3D human pose estimation. On the one hand, D3DP generatesmultiple possible 3D pose hypotheses for a single 2D observation. It graduallydiffuses the ground truth 3D poses to a random distribution, and learns adenoiser conditioned on 2D keypoints to recover the uncontaminated 3D poses.The proposed D3DP is compatible with existing 3D pose estimators and supportsusers to balance efficiency and accuracy during inference through twocustomizable parameters. On the other hand, JPMA is proposed to assemblemultiple hypotheses generated by D3DP into a single 3D pose for practical use.It reprojects 3D pose hypotheses to the 2D camera plane, selects the besthypothesis joint-by-joint based on the reprojection errors, and combines theselected joints into the final pose. The proposed JPMA conducts aggregation atthe joint level and makes use of the 2D prior information, both of which havebeen overlooked by previous approaches. Extensive experiments on Human3.6M andMPI-INF-3DHP datasets show that our method outperforms the state-of-the-artdeterministic and probabilistic approaches by 1.5% and 8.9%, respectively. Codeis available at https://github.com/paTRICK-swk/D3DP.