We propose to build omni-modal intelligence, which is capable ofunderstanding any modality and learning universal representations. In specific,we propose a scalable pretraining paradigm, named Multimodal Context (MiCo),which can scale up the numbers of modalities and amount of data, together withthe model parameters, in the pretraining process. With MiCo, the pretrainedmodels show significant emergent abilities in multimodal learning, which areevaluated on the following tasks: i) single-modality perception benchmarks of10 different modalities, ii) 25 cross-modality understanding tasks ofretrieval, question-answering, captioning, and iii) 18 multimodal largelanguage model benchmarks. Our models establish 37 new records forstate-of-the-art performance. We hope that our research could contribute to thedevelopment of omni-modal intelligence. Code and Models are athttps://github.com/invictus717/MiCo