MM-AU is a large-scale dataset focusing on multimodal accident video understanding, which aims to provide high-quality data support for the field of safe driving perception. It was released in 2024 by researchers from Xi'an Jiaotong University, Chang'an University, National University of Singapore, Cleveland State University and Nanyang Technological University. The related paper results are "Abductive Ego-View Accident Video Understanding for Safe Driving Perception" and was accepted by CVPR2024 Highlight.

The dataset consists of multiple public self-perspective accident datasets (such as CCD, A3D, DoTA, and DADA-2000) and video clips from video platforms such as YouTube, Bilibili, and Tencent. MM-AU is currently the largest and most fine-grained self-perspective multimodal accident dataset, containing 11,727 wild self-perspective accident videos, totaling 2,195,613 frames. These videos cover a variety of accident scenarios, providing researchers with rich data resources.

The dataset is rich in annotation information, including object detection, accident cause description, and text description. In terms of object detection, more than 2.23 million bounding boxes are annotated for 7 types of road participants (cars, traffic lights, pedestrians, trucks, buses, cyclists, and motorcycles). In addition, the dataset also annotates more than 58,650 pairs of video accident causes, covering 58 accident categories. Each video is accompanied by a time-aligned text description, including the cause of the accident, prevention suggestions, and accident category description. These annotation information not only helps in the analysis of the cause of the accident, but also provides important reference for the autonomous driving system.