PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

Existing 3D human pose estimators suffer poor generalization performance tonew datasets, largely due to the limited diversity of 2D-3D pose pairs in thetraining data. To address this problem, we present PoseAug, a newauto-augmentation framework that learns to augment the available training posestowards a greater diversity and thus improve generalization of the trained2D-to-3D pose estimator. Specifically, PoseAug introduces a novel poseaugmentor that learns to adjust various geometry factors (e.g., posture, bodysize, view point and position) of a pose through differentiable operations.With such differentiable capacity, the augmentor can be jointly optimized withthe 3D pose estimator and take the estimation error as feedback to generatemore diverse and harder poses in an online manner. Moreover, PoseAug introducesa novel part-aware Kinematic Chain Space for evaluating local joint-angleplausibility and develops a discriminative module accordingly to ensure theplausibility of the augmented poses. These elaborate designs enable PoseAug togenerate more diverse yet plausible poses than existing offline augmentationmethods, and thus yield better generalization of the pose estimator. PoseAug isgeneric and easy to be applied to various 3D pose estimators. Extensiveexperiments demonstrate that PoseAug brings clear improvements on bothintra-scenario and cross-scenario datasets. Notably, it achieves 88.6% 3D PCKon MPI-INF-3DHP under cross-dataset evaluation setup, improving upon theprevious best data augmentation based method by 9.1%. Code can be found at:https://github.com/jfzhang95/PoseAug.