NVIDIA has introduced NeMo Retriever Parse, a vision-language model (VLM) designed to transform complex documents into structured, actionable data. Enterprises often struggle with unstructured content in research reports, contracts, financial statements, and technical manuals, as traditional optical character recognition (OCR) systems fail to handle intricate layouts, variable formatting, and maintain continuity across multi-page documents. This new tool addresses these challenges by leveraging advanced AI to preserve document structure, semantics, and reading order, enabling more efficient data extraction and analysis. NeMo Retriever Parse is part of NVIDIA’s NeMo Retriever suite, which focuses on building multimodal ingestion and retrieval pipelines. The model uses a transformer-based architecture, combining a vision encoder (ViT-H) with an mBART decoder, optimized for both speed and accuracy. Unlike conventional OCR methods that process text in isolation, this VLM integrates visual and textual understanding, allowing it to analyze elements like headers, footers, tables, and mathematical formulas while maintaining their spatial relationships. A key innovation lies in its unified tokenization system. The model uses specialized tokens to encode not only text but also bounding box coordinates and semantic classifications, such as titles, paragraphs, or captions. These spatial and semantic tokens are interleaved in the output sequence according to the document’s natural reading flow, enabling precise, structured outputs. This approach differs from traditional multi-stage pipelines, which often separate layout analysis from text extraction, leading to fragmented results. Training involves a two-step process: pre-training on high-quality datasets like arXiv-5M, which includes annotated text and layout information, followed by fine-tuning on diverse sources, including partially annotated public datasets. A technique called multi-token training (MTT) further enhances accuracy by allowing the decoder to predict multiple tokens at once, improving its ability to track dependencies and maintain coherent document structure. Evaluations on benchmarks like the General OCR Theory (GOT) Dense OCR Benchmark and NVIDIA’s internal document OCR tests show strong performance. On the GOT benchmark, which focuses on dense, high-resolution text, NeMo Retriever Parse achieves near-perfect scores for text extraction, demonstrating its ability to handle complex formatting. For table recognition, it outperforms existing models on PubTabNet and RD-TableBench, scoring 80.20 on TEDS and 92.20 on S-TEDS for PubTabNet, and showing significant accuracy improvements on RD-TableBench, which includes scanned tables, handwritten content, and multilingual data. The model supports outputs in plain text and markdown formats, making it adaptable for enterprise workflows. Its ability to segment documents into semantic classes—such as headers, footers, and bibliographies—ensures structured, context-aware data that aligns with retrieval systems and large language models (LLMs). This is critical for tasks like research indexing, legal document management, and financial reporting, where accuracy and coherence are paramount. NVIDIA highlights that NeMo Retriever Parse balances high text extraction fidelity with precise table reconstruction, addressing the dual demands of content and structure. While currently focused on English, the tool is expanding to support Chinese and handwritten documents, with plans to increase context length for deeper analysis. By bridging raw document data with intelligent processing, the model aims to streamline how organizations extract, organize, and utilize information, positioning itself as a competitive solution for mission-critical workflows. Developers can access the model via the NVIDIA API catalog, with additional resources available through the NGC Catalog for further implementation. The tool underscores NVIDIA’s push to advance document AI, offering a robust alternative to traditional OCR systems in an era where structured data is essential for decision-making and innovation.