DataComp-12M is an enhanced image-text dataset released by Apple in 2024, with the associated research paper titled "MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training", aiming to improve the training efficiency and performance of image-text models through multimodal reinforcement training strategies.

The dataset contains the UIDs of 12 million samples from the DataComp-1B-BestPool subset and is part of the DataCompDR series of data. DataCompDR enhances the original DataComp data using multimodal dataset reinforcement strategies, optimizing the training process by generating synthetic captions and image augmentations. Experiments show that image-text models trained on DataCompDR-12M significantly outperform benchmark datasets such as CC-12M, YFCC-15M, and DataComp-Small/Medium on tasks like zero-shot classification, achieving a 10x to 1,000x improvement in learning efficiency. The original images and text samples in this dataset are released by DataComp under the CC-BY-4.0 license, while the metadata is released by Apple under the Apple AMLR license.

Dataset Composition

This dataset primarily consists of UID list files used to identify samples in the DataComp-12M dataset. The specific structure is as follows:

• uids.txt: A text list containing 12,779,520 UIDs, with one UID per line.
• uids.npy: A NumPy array file containing 12,779,520 UIDs, with the data type numpy.dtype("u8,u8").

These UIDs correspond one-to-one with the shard UIDs in the original DataComp-12M dataset and the DataCompDR-12M dataset, ensuring data consistency and traceability.

Citation

@InProceedings{mobileclip2024,
  author = {Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, Oncel Tuzel},
  title = {MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2024},
}