VidSitu Video Understanding Dataset

VidSitu is a dataset for semantic role labeling in videos (VidSRL). VidSitu is a large-scale video understanding data source, including 29K 10-second movie clips, annotated with verbs and semantic roles in 2-second units. Entities are co-referenced in each event of the clip, and events are connected by event-event relations.

The clips in VidSitu come from a large collection of movies (3K), and are selected to be complex (4.2 unique verbs in a single video) and diverse (200 verbs with more than 100 tokens each).