Multimodal Multipart Learning for Action Recognition in Depth Videos

The articulated and complex nature of human actions makes the task of actionrecognition difficult. One approach to handle this complexity is dividing it tothe kinetics of body parts and analyzing the actions based on these partialdescriptors. We propose a joint sparse regression based learning method whichutilizes the structured sparsity to model each action as a combination ofmultimodal features from a sparse set of body parts. To represent dynamics andappearance of parts, we employ a heterogeneous set of depth and skeleton basedfeatures. The proper structure of multimodal multipart features are formulatedinto the learning framework via the proposed hierarchical mixed norm, toregularize the structured features of each part and to apply sparsity betweenthem, in favor of a group feature selection. Our experimental results exposethe effectiveness of the proposed learning method in which it outperforms othermethods in all three tested datasets while saturating one of them by achievingperfect accuracy.