VidProM Large-scale Text-to-video Prompt Dataset

The VidProM dataset is the first large-scale real-user text-to-video prompt dataset jointly developed by the University of Technology Sydney and Zhejiang University. It contains 1.67 million unique text-to-video prompts and 6.69 million videos generated by four state-of-the-art diffusion models.

The dataset not only provides a large amount of video content, but also includes the NSFW (Not Safe For Work) probability associated with each prompt, 3072-dimensional prompt embedding, and additional related metadata. Its construction process includes collecting source HTML files, extracting and embedding text prompts, assigning NSFW probabilities to them, crawling and generating videos, and finally filtering out semantically unique prompts.

The release of VidProM aims to promote research in the field of text-to-video generation, covering aspects such as model evaluation, efficient video generation, fake video detection, and video copyright detection, providing researchers with valuable resources to explore and develop new text-to-video generation techniques.