A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognition

Current researches of action recognition mainly focus on single-view andmulti-view recognition, which can hardly satisfies the requirements ofhuman-robot interaction (HRI) applications to recognize actions from arbitraryviews. The lack of datasets also sets up barriers. To provide data forarbitrary-view action recognition, we newly collect a large-scale RGB-D actiondataset for arbitrary-view action analysis, including RGB videos, depth andskeleton sequences. The dataset includes action samples captured in 8 fixedviewpoints and varying-view sequences which covers the entire 360 degree viewangles. In total, 118 persons are invited to act 40 action categories, and25,600 video samples are collected. Our dataset involves more participants,more viewpoints and a large number of samples. More importantly, it is thefirst dataset containing the entire 360 degree varying-view sequences. Thedataset provides sufficient data for multi-view, cross-view and arbitrary-viewaction analysis. Besides, we propose a View-guided Skeleton CNN (VS-CNN) totackle the problem of arbitrary-view action recognition. Experiment resultsshow that the VS-CNN achieves superior performance.