Nemotron-Pretraining-Code-v3 is one of the code pre-training datasets created by NVIDIA for the Nemotron 3 series of large language models. It aims to enhance the code understanding, generation, completion, and reasoning capabilities of LLM. This dataset belongs to the code corpus part of the NVIDIA Nemotron Pretraining Data series. The related paper is Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning. This dataset contains 146.3 million files. Data collection ended on September 30, 2025. This is an incremental version, only including newly added source code files compared to previous versions. It needs to be compared with previous versions. v1, v2 When used together, they can form a complete code pre-training corpus.

Data Fields:

• commit_id: The first seven characters of the Git commit hash.
• rel_path: The path of the file in the root directory of the repository.
• language: The programming language for automated detection and recognition.

Citation

If you use this dataset in your research, please cite it. Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning .