ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

Estimating the articulated 3D hand-object pose from a single RGB image is ahighly ambiguous and challenging problem, requiring large-scale datasets thatcontain diverse hand poses, object types, and camera viewpoints. Mostreal-world datasets lack these diversities. In contrast, data synthesis caneasily ensure those diversities separately. However, constructing both validand diverse hand-object interactions and efficiently learning from the vastsynthetic data is still challenging. To address the above issues, we proposeArtiBoost, a lightweight online data enhancement method. ArtiBoost can coverdiverse hand-object poses and camera viewpoints through sampling in aComposited hand-object Configuration and Viewpoint space (CCV-space) and canadaptively enrich the current hard-discernable items by loss-feedback andsample re-weighting. ArtiBoost alternatively performs data exploration andsynthesis within a learning pipeline, and those synthetic data are blended intoreal-world source data for training. We apply ArtiBoost on a simple learningbaseline network and witness the performance boost on several hand-objectbenchmarks. Our models and code are available athttps://github.com/lixiny/ArtiBoost.