Verse-Bench is a benchmark dataset for evaluating the joint generation of audio and video, released in 2025 by StepFun in collaboration with the Hong Kong University of Science and Technology, the Hong Kong University of Science and Technology (Guangzhou) and other institutions. The relevant paper results are "UniVerse-1: Unified Audio-Video Generation via Stitching of Experts", which aims to push generative models to not only generate videos, but also maintain strict temporal alignment with audio content (including ambient sound and speech).

The dataset contains 600 image-text prompt pairs, sourced from YouTube, Bilibili, TikTok video frames, movie/anime screenshots, AI model generated images, and public web images.

Data distribution

The dataset is divided into three subsets (Set1-I, Set2-V, and Set3-Ted), covering a variety of audio categories, such as human voices, animal sounds, instrumental music, natural sounds, human-object interaction sounds, object impacts, and mechanical noises, respectively, suitable for different scenarios and content types. The specific distribution is as follows:

• Set 1-I contains 205 image-text pairs, including AI-generated images, web scraping, and media screenshots. Each image is used as the visual input, and the corresponding video/audio captions and speech content are generated by a large language model (LLM) and human annotation.
• Set2-V contains 295 samples of short video clips from YouTube and BiliBili, which are accompanied by subtitles generated by LLM and transcribed text using Whisper for automatic speech recognition (ASR) and manually verified.
• Set3-Ted contains TED talk videos from September 2025, with a total of 100 samples, using the same annotation process as Set2.