One-sentence Summary

Alibaba Group's ATH-MaaS team introduces OvisOCR2, a 0.8B end-to-end document parser that generates Markdown from page images by combining real and synthetic HTML-derived data and training via supervised fine-tuning, reinforcement learning with multi-component rewards, on-policy distillation, and model fusion, achieving state-of-the-art overall scores of 96.58 on OmniDocBench v1.6 and 75.06 on PureDocBench, thereby placing an end-to-end model atop a leaderboard previously dominated by pipeline methods.

Key Contributions

• A data engine combines filtered real-document annotations with synthetic pages, where rendered images and Markdown targets are derived from the same HTML source.
• The training recipe includes supervised fine-tuning, reinforcement learning on a 4B branch with a multi-component reward design, on-policy distillation into the 0.8B model, and model fusion.
• OvisOCR2 achieves state-of-the-art overall scores on OmniDocBench v1.6 (96.58) and PureDocBench (Avg3 75.06), surpassing prior pipeline methods and obtaining the highest overall performance on an in-house benchmark covering long-tail scenarios.

Introduction

Document parsing transforms visually rich document images into structured Markdown, preserving layout, reading order, tables, formulas, and other elements for indexing and downstream use. Pipeline methods dominate public leaderboards but introduce deployment complexity and error propagation across separate layout and recognition stages. End-to-end models simplify deployment yet have lagged behind in accuracy. The authors introduce OvisOCR2, a compact 0.8B end-to-end parser that surpasses leading pipeline methods. They achieve this by post-training a small vision-language model with a data engine that blends filtered real annotations and source-aligned synthetic pages, combined with supervised fine-tuning, reinforcement learning, on-policy distillation, and model fusion. OvisOCR2 sets new state-of-the-art scores on OmniDocBench v1.6 and PureDocBench while maintaining a minimal deployment footprint.

Dataset

The authors construct a training dataset for an end-to-end document parser by combining two complementary pipelines: a real-world data pipeline and a synthetic data pipeline. Both pipelines produce image–Markdown pairs, but they differ in origin, annotation quality, and the way they expand coverage.

Real-world data subset

• Source: Large-scale real-world document images (scanned pages, digital documents, etc.) processed by two OCR parsers: PaddleOCR-VL-1.5 and MinerU2.5-Pro.
• Processing: Parser outputs are first normalized into a unified Markdown schema through source-specific rules. This includes strict category validation, text-block merging, formula normalization (LaTeX-style delimiters), table sanitization, and visual-region representation via normalized bounding boxes (<img src="images/bbox_left_top_right_bottom.jpg" /> with coordinates in [0,1000)). A pre-filtering stage removes samples with no valid text, duplicates, or severe parser failures.
• Filtering: After conversion, manual spot-checking is applied per subset. Inspectors verify text correspondence, formula accuracy, table alignment, visual-region alignment, and reading-order consistency. Subsets with frequent errors are discarded; those with only occasional minor errors are retained together with the sample-level filters.
• Schema: The final Markdown preserves reading order as given by the parser’s block order, with double newlines between blocks.

Synthetic data subset

• Source: Generated from HTML templates derived from hard failure cases identified during multi-faceted assessments. A multimodal model converts each hard sample (or cluster) into an initial HTML template that captures the challenging layout structure.
• Diversification: An agent-based procedure expands each template into many HTML pages by varying content (text, numbers, formulas) and structure (tables, sections, visual placements) while enforcing validity rules. This creates diverse, controllable samples with deterministic labels.
• Ground truth: Markdown is generated directly from the HTML source, not from the rendered image. Serialization rules map text, tables, formulas, and visual regions to the same unified schema used for real-world data. Reading order is assigned with document-type-aware rules (e.g., column partitioning for multi-column layouts).
• Rendering: Document images are rendered with Playwright, preserving realistic typography and layout. Element-level bounding boxes are recorded from the DOM and normalized to [0,1000). If geometric augmentations are applied, coordinates are transformed accordingly.
• Quality control: A preview-and-iteration stage inspects a small batch for style and layout defects before scaling. Large-scale generation is followed by automatic removal of rendering failures, empty targets, structural errors, and duplicates.

How the dataset is used

• Both subsets are combined into a single training corpus for the document parser. The paper does not disclose exact sizes or mixture ratios, but describes the real-world pipeline as large-scale and the synthetic pipeline as producing samples at scale to expand long-tail coverage.
• The unified Markdown schema (with HTML table tags, LaTeX math, and normalized image placeholders) serves as the target format for model training.
• No explicit cropping strategy is mentioned; visual regions are localized via bounding boxes, and reading order is preserved during serialization. The synthetic pipeline adjusts viewport and page boundaries during rendering to avoid invalid crops.

Method

The authors leverage a comprehensive data engine and a multi-stage training pipeline to develop an end-to-end document parser. The data engine is composed of two complementary pipelines: a real-world data pipeline and a synthetic data pipeline.

The real-world data pipeline transforms large-scale document images into reliable training data. It utilizes specialized OCR parsers to obtain structured outputs, which are then normalized into a unified Markdown format through rule-based conversion. This process includes strict category validation, text and formula normalization, and visual-region handling. To ensure quality, the pipeline incorporates manual spot-checking and subset filtering to verify text correspondence, formula accuracy, table alignment, and reading-order consistency.

The synthetic data pipeline complements the real-world data by generating controllable samples with precise ground-truth annotations. It begins with hard sample mining to identify underrepresented document patterns. A multimodal model converts these hard samples into initial HTML templates, which are then expanded into diverse HTML pages through an agent-based generation procedure. The pipeline derives Markdown ground truth directly from the HTML source and renders document images using Playwright. Iterative quality control ensures the synthetic data maintains clean annotations and faithful image-text alignment.

The training pipeline is organized into several stages to build a robust parsing policy. Supervised fine-tuning establishes the base policy using a mixture of real and synthetic data. Both 0.8B and 4B models are trained, with the 0.8B model serving as the deployable student and the 4B model as the base for reinforcement learning.

The reinforcement learning stage employs Group Relative Policy Optimization to improve the policy using verifiable rewards. The reward design is multi-component, evaluating text, tables, and formulas separately. The page reward is computed by averaging the scores of the components present in the reference: $R(y, y^*) = \frac{\sum_{c \in \mathcal{C}} a_c(y^*) s_c(y, y^*)}{\sum_{c \in \mathcal{C}} a_c(y^*)}$R(y,y∗)=∑c∈C​ac​(y∗)∑c∈C​ac​(y∗)sc​(y,y∗)​ where $a_c(y^*)$ac​(y∗) indicates the availability of component $c$c in the reference.

To transfer the reward-aligned behavior from the 4B teacher to the 0.8B student, the authors use on-policy distillation. Direct reinforcement learning on the compact model leads to higher policy divergence and unstable table quality.

In the on-policy distillation process, the student generates complete page outputs, and the teacher evaluates these trajectories with token-level distribution supervision. The objective is student top-k reverse KL divergence. Given a document image and instruction $x$x, the student policy $\pi_\theta$πθ​ samples a response $y$y. The teacher policy $\pi_\phi$πϕ​ evaluates conditional probabilities for tokens in the student top-k support $S_t = \text{TopK}_k(\pi_\theta(\cdot \mid c_t))$St​=TopKk​(πθ​(⋅∣ct​)). The distributions are normalized within this support: $\bar{p}_{t,v} = \frac{\pi_\theta(v \mid c_t)}{\sum_{u \in S_t} \pi_\theta(u \mid c_t)}, \quad \bar{q}_{t,v} = \frac{\pi_\phi(v \mid c_t)}{\sum_{u \in S_t} \pi_\phi(u \mid c_t)}, \quad v \in S_t$pˉ​t,v​=∑u∈St​​πθ​(u∣ct​)πθ​(v∣ct​)​,qˉ​t,v​=∑u∈St​​πϕ​(u∣ct​)πϕ​(v∣ct​)​,v∈St​ The distillation loss is then computed as: $L_{\text{OPD}}(\theta) = \frac{1}{|I|} \sum_{t \in I} D_{\text{KL}}(\bar{p}_t \| \bar{q}_t)$LOPD​(θ)=∣I∣1​∑t∈I​DKL​(pˉ​t​∥qˉ​t​) where $I$I denotes valid response token positions. This reverse-KL direction encourages mode-seeking behavior, discouraging the student from assigning probability mass to tokens the teacher deems unlikely.

Finally, the authors apply weighted parameter averaging across several candidate model variants to produce the final fused model.

Experiment

OvisOCR2 is evaluated against both general and specialized vision-language models, including pipeline and end-to-end approaches, on OmniDocBench v1.6, PureDocBench, and an in-house benchmark covering diverse document types. The compact 0.8B model achieves state-of-the-art overall scores on OmniDocBench and PureDocBench (clean and digital tracks), surpassing larger pipeline methods, though it trails large general VLMs on real-world degraded images, highlighting a need for improved robustness. On the in-house benchmark, OvisOCR2 consistently leads across text, formula, table, and reading-order metrics, with particular strength in handwriting and complex tables, where it avoids the table-missing errors that affect pipeline systems.

The RL reward decomposes page-level parsing quality into three component-specific scores: text fidelity via normalized edit distance, visual formula matching via CDM, and table structure via TEDS. Only components present in the ground truth are averaged to form the final reward, avoiding a single text-similarity metric and preventing missing content types from unfairly penalizing the model. Text fidelity is measured by 1 minus normalized edit distance. Formula parsing quality uses CDM (character detection matching), an image-level metric for visual formula matching. Table similarity is evaluated with TEDS, a tree-edit-distance-based metric that captures both table content and topology.

OvisOCR2 achieves state-of-the-art performance on the OmniDocBench v1.6 document parsing benchmark, surpassing both pipeline and end-to-end methods with a compact 0.8B model. It records the lowest text edit distance, highest formula recognition score, tied highest table structure score, and lowest reading-order error, demonstrating comprehensive improvements across all evaluation dimensions. OvisOCR2 attains an overall score of 96.58, outperforming the previous best end-to-end method by 1.84 points. The model achieves the lowest text edit distance and the highest formula CDM score, indicating superior text and formula recognition accuracy. OvisOCR2 ties for the highest table TEDS and leads in TEDS-S, reflecting robust table reconstruction even on structurally complex pages. It records the lowest reading-order edit distance, showing precise recovery of block sequence order across diverse layouts.

OvisOCR2 achieves the highest overall average on PureDocBench, leading on the clean and digital tracks. On the real track, it falls behind strong general vision-language models like Gemini-3.1-Pro and Qwen3.5-122B-A10B, indicating room for improvement in handling degraded real-world images. OvisOCR2 attains an Avg3 score of 75.06, surpassing all listed models including the best general VLM Qwen3.5-122B-A10B (74.11). While OvisOCR2 ranks first on clean and digital tracks, it scores lower than Gemini-3.1-Pro and Qwen3.5-122B-A10B on the real track, highlighting a challenge with physical recaptures and screen-mediated images.

On the in-house benchmark, OvisOCR2 achieves the highest overall score of 85.54, outperforming all other models by a clear margin. It simultaneously delivers the lowest text edit distance, the lowest reading-order edit distance, the highest formula CDM score, and the highest table TEDS and TEDS-S scores, demonstrating consistent superiority across all evaluation dimensions. OvisOCR2 attains an overall score of 85.54, leading the second-best model by 2.66 points. Text edit distance drops to 0.0850, substantially lower than the next best value of 0.1292. Reading-order edit distance reaches 0.1885, the lowest among all compared systems. Formula CDM score of 86.32 is the highest recorded, ahead of the next best 85.13. Table TEDS (78.80) and TEDS-S (82.87) both rank first, indicating best table structure recognition.

OvisOCR2 leads across all three difficulty tiers of the in-house benchmark, with its advantage growing as complexity increases. While the closest competitors are roughly 2–3 points behind on easy and medium documents, the margin widens to nearly 4 points on hard documents, highlighting stronger robustness to challenging layouts. OvisOCR2 achieves the highest score in every difficulty tier, outperforming PaddleOCR-VL-1.6, GLM-OCR, and earlier models. The performance drop from easy to hard is noticeably smaller for OvisOCR2 than for other methods, resulting in a wider absolute lead on hard documents.

OvisOCR2 uses a structured RL reward that separately evaluates text fidelity, formula matching, and table structure to avoid penalizing missing content types. It achieves state-of-the-art document parsing on OmniDocBench with a compact 0.8B model, outperforming prior end-to-end methods across text, formula, table, and reading order metrics. On PureDocBench it leads on clean and digital documents but trails behind large general VLMs on degraded real-world images, indicating a weakness in handling physical recaptures. The model's advantage widens with layout complexity, as shown by its growing lead on harder documents in the in-house benchmark, highlighting robust performance on challenging page structures.