3D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by Prior Knowledge for Hand-Object Interaction Scenario

Recently, 3D hand reconstruction has gained more attention in human-computercooperation, especially for hand-object interaction scenario. However, it stillremains huge challenge due to severe hand-occlusion caused by interaction,which contain the balance of accuracy and physical plausibility, highlynonlinear mapping of model parameters and occlusion feature enhancement. Toovercome these issues, we propose a 3D hand reconstruction network combiningthe benefits of model-based and model-free approaches to balance accuracy andphysical plausibility for hand-object interaction scenario. Firstly, we presenta novel MANO pose parameters regression module from 2D joints directly, whichavoids the process of highly nonlinear mapping from abstract image feature andno longer depends on accurate 3D joints. Moreover, we further propose avertex-joint mutual graph-attention model guided by MANO to jointly refine handmeshes and joints, which model the dependencies of vertex-vertex andjoint-joint and capture the correlation of vertex-joint for aggregatingintra-graph and inter-graph node features respectively. The experimentalresults demonstrate that our method achieves a competitive performance onrecently benchmark datasets HO3DV2 and Dex-YCB, and outperforms all onlymodel-base approaches and model-free approaches.