AceMath RM Training Data is a dataset released by NVIDIA in 2024 for training mathematical Outcome Reward Modeling, with the related research paper titled "AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling", aimed at improving the performance of large models in mathematical reasoning tasks.

The dataset contains 356,058 unique mathematical problems, totaling 2,136,348 samples, with each problem corresponding to 6 different responses. The data is primarily used to train the AceMath-7B and AceMath-72B reward models, optimizing mathematical reasoning capabilities by evaluating the correctness of model answers. Across multiple benchmark tests, models trained on this data have demonstrated exceptional mathematical reasoning performance, surpassing various mainstream open-source and proprietary models.

Dataset Composition

The dataset mainly includes the following fields and structure:

• qid: Unique question identifier.
• message: A list of conversations containing system prompts, user questions, and assistant step-by-step solutions. Each message includes role (e.g., system, user, assistant) and content.
• label: A binary label indicating whether the response is correct/excellent (1 for correct, 0 for incorrect).

The data format is a list of JSON objects, which can be directly loaded and used for training. Each sample represents a mathematical problem along with comparative data of multiple candidate solutions.

Citation

@article{acemath2024,
  title={AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling},
  author={Liu, Zihan and Chen, Yang and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei},
  journal={arXiv preprint},
  year={2024}
}