CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition

Skeleton-based multi-entity action recognition is a challenging task aimingto identify interactive actions or group activities involving multiple diverseentities. Existing models for individuals often fall short in this task due tothe inherent distribution discrepancies among entity skeletons, leading tosuboptimal backbone optimization. To this end, we introduce a Convex HullAdaptive Shift based multi-Entity action recognition method (CHASE), whichmitigates inter-entity distribution gaps and unbiases subsequent backbones.Specifically, CHASE comprises a learnable parameterized network and anauxiliary objective. The parameterized network achieves plausible,sample-adaptive repositioning of skeleton sequences through two key components.First, the Implicit Convex Hull Constrained Adaptive Shift ensures that the neworigin of the coordinate system is within the skeleton convex hull. Second, theCoefficient Learning Block provides a lightweight parameterization of themapping from skeleton sequences to their specific coefficients in convexcombinations. Moreover, to guide the optimization of this network fordiscrepancy minimization, we propose the Mini-batch Pair-wise Maximum MeanDiscrepancy as the additional objective. CHASE operates as a sample-adaptivenormalization method to mitigate inter-entity distribution discrepancies,thereby reducing data bias and improving the subsequent classifier'smulti-entity action recognition performance. Extensive experiments on sixdatasets, including NTU Mutual 11/26, H2O, Assembly101, Collective Activity andVolleyball, consistently verify our approach by seamlessly adapting tosingle-entity backbones and boosting their performance in multi-entityscenarios. Our code is publicly available at https://github.com/Necolizer/CHASE .