The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation

Nonparametric based methods have recently shown promising results inreconstructing human bodies from monocular images while model-based methods canhelp correct these estimates and improve prediction. However, estimating modelparameters from global image features may lead to noticeable misalignmentbetween the estimated meshes and image evidence. To address this issue andleverage the best of both worlds, we propose a framework of three consecutivemodules. A dense map prediction module explicitly establishes the dense UVcorrespondence between the image evidence and each part of the body model. Theinverse kinematics module refines the key point prediction and generates aposed template mesh. Finally, a UV inpainting module relies on thecorresponding feature, prediction and the posed template, and completes thepredictions of occluded body shape. Our framework leverages the best ofnon-parametric and model-based methods and is also robust to partial occlusion.Experiments demonstrate that our framework outperforms existing 3D humanestimation methods on multiple public benchmarks.