AceMath-RM Training Data

Dataset Introduction

The AceMath-RM training data is a dataset used to train the AceMath-7B/72B-RM math outcome reward model. This dataset contains 356,058 unique math problems, each with 6 distinct answers, for a total of 2,136,348 samples. Each sample includes the question, the step-by-step solution, and a binary label (1 for correct, 0 for incorrect). This dataset aims to support the training of reward models in the field of mathematical reasoning. Reward models trained on this data have performed exceptionally well on multiple mathematical reasoning benchmarks. For example, AceMath-72B-RM achieved an average rm@8 accuracy of 69.53% on seven benchmarks, including GSM8K, MATH500, and Minerva Math, surpassing the previous best model.

Dataset composition

• Data scale356,058 unique mathematical questions, 2,136,348 samples (6 candidate answers for each question).
• Data FormatEach sample is in JSON format and contains the following fields:qid(Unique identifier for the issue) message(Conversation history, including system prompts, user questions, and assistant answers) label(Binary fraction).
• Dataset segmentationNo splitting; all data is used for training.
• Features and characteristicsThe questions cover multiple branches of mathematics, the answers include step-by-step reasoning processes, and the labels are marked by human or automated methods.
• Use cases: Train a mathematical outcome reward model to select the best answer from multiple candidate answers.
• Data format example:qid For strings,message For a list, each element contains role and content,label It can be 0 or 1.
• Important NoteThis dataset is for non-commercial use only and is licensed under CC-BY-NC-4.0. Released by NVIDIA, the original questions are derived from data generated by OpenAI models and must comply with their terms of use.
• license：Creative Commons Attribution: Non-Commercial 4.0 International.

Citation

If you find our work helpful, please cite us:

@article{acemath2024,
title={AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling},
author={Liu, Zihan and Chen, Yang and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei},
journal={arXiv preprint},
year={2024}
}