NeoMME Unveils Efficient Multimodal and Multilingual Encoder

H Company researchers Aurélien Lac and Tony Wu have unveiled NeoMME, a new family of multilingual, multimodal foundation encoders designed to streamline visual document retrieval. Released under the Apache 2.0 license, the NeoMME architecture challenges prevailing multimodal paradigms by replacing separate pretrained vision towers and causal language decoders with a single bidirectional Transformer. This unified backbone natively processes raw 32 by 32 image patches and multilingual text tokens within one computational path, enabling efficient pretraining, fine-tuning, and inference across both modalities. Available in 260-million and 800-million parameter variants, NeoMME was trained from scratch using a masked discrete-diffusion objective across 524 billion tokens of multilingual text, code, mathematics, and natural and document images. The resulting model, NeoMME-Retriever, is optimized for visual document retrieval through ColPali page-image methodology. By treating document pages as raw images rather than extracted text, the system preserves critical layout, tabular, and typographic cues often lost during optical character recognition. A single forward pass simultaneously generates both dense and late-interaction vector embeddings, providing deployment flexibility without requiring multiple model passes. Benchmarks on the ViDoRe v3 dataset demonstrate strong competitive performance. The 260-million parameter variant achieves a 0.523 nDCG@10 score, surpassing all evaluated models under 800 million parameters while utilizing roughly 14 times fewer parameters than comparable systems. At a matched 2048 by 2048 resolution on an NVIDIA L40S GPU, the smaller model processes approximately 51 pages per second, delivering nearly double the encoding throughput of leading alternatives like ColModernVBERT. To address the substantial storage overhead inherent to high-resolution late-interaction embeddings, the developers introduced a dual-compression pipeline combining hierarchical token pooling and asymmetric quantization. This approach reduces per-page storage from approximately 1.5 megabytes to just 6 kilobytes, a 255-fold compression ratio, while retaining over 95 percent of baseline retrieval accuracy. The framework supports direct integration with modern vector databases and Sentence Transformers version 6, facilitating independent or joint fine-tuning of retrieval heads. NeoMME is positioned as a foundational component for visual retrieval-augmented generation systems, enabling applications that natively ingest and reason over document page imagery. Full model checkpoints, processor implementations, and inference code are publicly available on Hugging Face Transformers, providing researchers and engineers with an efficient, open-source alternative for multilingual multimodal indexing and document intelligence workflows.