Open Pipeline Generates 2.8 Trillion Indic Pretraining Tokens

NVIDIA and researchers from IIT Madras AI4Bharat have announced Nemotron-pretraining-Indic-v1, an open pretraining corpus of roughly 2.8 trillion tokens spanning the 22 languages recognized by India's Constitution, alongside the reusable GPU-accelerated generation pipeline used to create it. The corpus is intended to expand the open data available for Indic language models and address a persistent shortage of training text for languages spoken by nearly 1.2 billion people as recorded in the 2011 census. The dataset includes about 2.601 trillion native-script tokens and 224.6 billion romanized tokens under a Gemma-4 tokenizer, covering 34 language and script configurations. Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu have both native-script and romanized records, while Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali and Sindhi have native-script data. Source material comes from curated English Nemotron-CC-v2 content, including web text, question-answer material, mathematics and code. The workflow uses NeMo Data Designer, Gemma-4-31B-it and Bodhan AI Indic Translate to perform document-level translation, automated quality scoring and targeted repair. Outputs are retained when they reach a 75 out of 100 threshold on an MQM-inspired rubric, with repair loops recovering hundreds of billions of additional tokens and provenance links preserved for auditing. The initiative, discussed by AI4Bharat lead Mitesh Khapra, is designed to be more than a static dataset. The open recipe lets developers adapt prompts, quality checks and source documents, while the corpus can be combined with naturally authored Indic text. The team also points to evidence of its value in continued pretraining for Nemotron 3.5 Super VL, where 10 billion Indic tokens from a 12-language subset improved scores on five Indic knowledge and reasoning benchmarks by between 4.98 and 9.82 percentage points. Compared with the earlier Nemotron 3 Super, the updated model showed gains on five of six evaluated task areas and stronger ratings on all four IndiVibe language-quality dimensions, though instruction following dipped 2.04 points. NVIDIA and AI4Bharat say the release, paired with open evaluation tools such as Indic-LLM-Eval-Suite, aims to make large-scale Indic data generation more auditable and accessible for builders working on inclusive language technology in India and beyond.