Abstract

We present MM1.5, a new family of multimodal large language models (MLLMs)designed to enhance capabilities in text-rich image understanding, visualreferring and grounding, and multi-image reasoning. Building upon the MM1architecture, MM1.5 adopts a data-centric approach to model training,systematically exploring the impact of diverse data mixtures across the entiremodel training lifecycle. This includes high-quality OCR data and syntheticcaptions for continual pre-training, as well as an optimized visualinstruction-tuning data mixture for supervised fine-tuning. Our models rangefrom 1B to 30B parameters, encompassing both dense and mixture-of-experts (MoE)variants, and demonstrate that careful data curation and training strategiescan yield strong performance even at small scales (1B and 3B). Additionally, weintroduce two specialized variants: MM1.5-Video, designed for videounderstanding, and MM1.5-UI, tailored for mobile UI understanding. Throughextensive empirical studies and ablations, we provide detailed insights intothe training processes and decisions that inform our final designs, offeringvaluable guidance for future research in MLLM development.