FineGym is an action recognition dataset based on gymnastics videos. Compared with existing datasets of the same type, the FineGym dataset is richer, more diverse, and of higher quality. The dataset has a three-level semantic hierarchy, providing time annotations for both the action level and the sub-action level. For example, a "balance beam" event can be annotated as a series of basic sub-actions from five sets, and the sub-actions in these sets will be further annotated with class labels.