Nemotron-Pretraining-Specialized-v1.1 is a synthetic pretraining dataset released by NVIDIA in 2026, designed for the Nemotron 3 series of large language models, aiming to enhance model capabilities in areas such as programming concepts and algorithms, formal logic, economics, and multiple-choice questions.

The dataset contains approximately 19.77 million samples, with a total size of about 34.9 GB, in Parquet format, generated by multiple large language models, and can be used for model pretraining and capability enhancement.

Dataset Composition

The dataset includes the following five subsets:

• Nemotron-Pretraining-Code-Concepts: Contains Python problems and solutions generated based on combinations of programming concepts, covering topics such as algorithms, data analysis, and data structures.
• Nemotron-Pretraining-Unconditional-Algorithmic: Contains Python algorithmic samples in various forms generated from short prompts.
• Nemotron-Pretraining-Formal-Logic: Contains multiple-choice questions and answers related to formal logic, with some samples including chain-of-thought.
• Nemotron-Pretraining-Economics: Contains multiple-choice questions and answers related to economics, with some samples including chain-of-thought.
• Nemotron-Pretraining-Multiple-Choice: Contains multi-topic multiple-choice questions from the MMLU auxiliary_train dataset.

Includes the following data fields:

• text: The text content used for pretraining.
• license: License information for the sample.
• metadata: Sample metadata, including data category and generating model; the Code-Concepts subset also includes function names, tags, question prompts, and generated solutions.
• uuid: Unique identifier for the sample.

Citation

When using this dataset for research, please cite the NVIDIA Nemotron 3 Super Technical Report.