One-sentence Summary

GenSelect, introduced by the authors, proposes a generative reward model where the LLM uses long reasoning to select the best solution among $N$N candidates, leveraging comparative strengths and efficient parallel scaling, and demonstrates that reasoning models such as QwQ and DeepSeek-R1-0528 outperform existing pointwise and pairwise scoring approaches for math reasoning with simple prompting.

Key Contributions

• GenSelect is a tournament-based Best-of-N selection method that uses large language models with long reasoning to choose the best solution among N candidates, leveraging comparative evaluation strengths while scaling efficiently across parallel sampling budgets.
• The method applies N-ary knockout tournaments that respect context window constraints and substantially reduce computational overhead relative to exhaustive pairwise comparisons, addressing practical scalability for larger sampling budgets.
• In mathematical reasoning experiments, GenSelect consistently outperforms baselines including majority voting, discriminative reward models, and generative reward models; reasoning models such as QwQ and DeepSeek-R1-0528 excel with simple prompting, and the gains remain robust across different tournament configurations.

Introduction

Test-time scaling improves large language models on reasoning tasks through sequential reasoning and parallel sampling, where the core task is selecting the best response from many candidates. Prior work uses rule-based or discriminative reward models, while recent generative reward models (GenRMs) reason over candidates and show promise. However, GenRMs typically operate in pointwise or pairwise modes. Pointwise evaluation is limited by LLM verification abilities, and pairwise comparison becomes computationally prohibitive for large candidate pools, requiring O(N^2) exhaustive comparisons or O(N) knockout tournaments that add latency. To overcome these challenges, the authors propose GenSelect, which directly performs N-ary comparison among all candidate solutions, generalizing GenRM from binary to N-ary judgment. For even larger sampling budgets, they extend this with an N-ary knockout tournament that respects context window constraints. Their experiments with QwQ and DeepSeek-R1 demonstrate that GenSelect outperforms existing baselines on competition-level math benchmarks and remains stable across different inference setups, enabling efficient scaling to higher parallel sampling budgets without performance degradation.

Dataset

The authors construct an evaluation benchmark called Comp-Math-24-25 to address the high variance of the AIME competition, which contains only 30 questions. The dataset is built by combining problems from two sources:

• AIME (American Invitational Mathematics Examinations) and HMMT (Harvard-MIT Mathematics Tournaments) for the years 2024 and 2025.
• Filtering rules: Proof-based questions and items that award partial credit based on estimated accuracy are excluded.
• Scale: The final benchmark consists of 256 problems in total.

The benchmark is used as the evaluation set for math reasoning tasks in the paper's experimental setup, providing a more statistically stable measurement than AIME alone.

Method

The authors establish a Generative Reward Model (GenRM) baseline to evaluate solution correctness through zero-shot prompting of a reasoning model. This approach requires the model to systematically analyze a mathematical problem and its proposed solution to determine if it is mathematically sound. The verification process is structured into three primary evaluation criteria: a mathematical accuracy check to verify computations and formulas, a logical reasoning analysis to ensure the progression from the problem statement to the solution is coherent, and a completeness evaluation to assess the handling of edge cases.

The model is instructed to output a step-by-step verification of the mathematical work, an analysis of the logical structure, and the identification of any errors. The evaluation concludes with a binary judgment of "Yes" or "No," where "Yes" indicates a correct solution. This setup allows for straightforward test-time scaling by sampling multiple verifications to aggregate the final answer.

In contrast to the individual scoring of GenRM, the proposed GenSelect method addresses the Best-of-N (BoN) selection task by evaluating all candidate solutions simultaneously. Rather than assigning individual scores or performing sequential pairwise comparisons, GenSelect leverages the Large Language Model to perform an N-ary comparison, directly identifying the best response from the entire set of $N$N candidates.

To overcome context window limitations inherent in reasoning models, the authors implement an N-ary knockout tournament strategy. This method scales the evaluation to larger sampling budgets by grouping candidates into rounds. For example, when evaluating 64 candidate solutions, a 16-way comparison tournament requires only two rounds and five total comparisons, whereas a binary tournament would necessitate six sequential rounds and 63 comparisons. This N-ary approach significantly reduces computational overhead and latency while maintaining equivalent theoretical complexity, allowing the system to achieve efficiency gains without degrading performance.

Experiment

Experiments on the Comp-Math-24-25 benchmark, which combines AIME and HMMT problems to reduce variance, show that the proposed GenSelect method, which leverages LLMs to compare candidate solutions, substantially outperforms both majority voting and discriminative reward models, with further gains when repeated and combined with majority voting. GenSelect remains stable across different candidate set sizes, indicating that larger N can be used without performance loss. In terms of compute allocation, the stronger model prefers generation-only at low budgets but benefits from GenSelect at larger budgets, while the weaker model gains from GenSelect across all budgets, and both models show significant overall improvements over the majority baseline, particularly on the HMMT split.

The Comp-Math-24-25 dataset is constructed by combining problems from AIME and HMMT competitions for 2024 and 2025, totaling 256 problems. It deliberately excludes proof-based questions and those with partial credit scoring, focusing on answerable math problems. The dataset draws from five specific sources, with HMMT contributing the majority of problems. Dataset merges AIME and HMMT problems from 2024 and 2025, with HMMT providing a larger share of the total. Excludes proof-based and partial-credit questions, resulting in a final set of 256 problems. Problem counts vary by source, with HMMT Feb 2024 contributing the most problems and AIME years contributing 30 each.

The table compares two inference compute allocation strategies across two models: spending the entire budget on solution generation followed by majority voting versus splitting it equally between solution and GenSelect generation. For the stronger DeepSeek model, generation-only is preferred at lower sampling budgets, but GenSelect becomes advantageous at higher budgets. For the weaker QwQ model, GenSelect consistently outperforms generation-only across all tested budgets. DeepSeek's performance improves with GenSelect when the sampling budget is larger, with the advantage growing as the budget increases. QwQ achieves higher accuracy with GenSelect relative to generation-only at every tested budget level. The benefit of allocating compute to GenSelect over additional generation is more pronounced for the weaker model, QwQ.

On the Comp-Math-24-25 benchmark, selecting the best solution via comparative evaluation (GenSelect) outperforms majority voting and other reward model based baselines for QwQ-32B. The gains are larger than those from scoring each solution individually, and repeating the selection pipeline further improves accuracy. A discriminative reward model trained on a different distribution scores below majority voting, likely due to distribution mismatch. Comparative selection with GenSelect beats majority voting and individual scoring, with additional gains when the selection process is repeated and majority voting is applied to the results.

The table evaluates how changing the number of comparison rounds (N) affects GenSelect performance. Accuracy remains nearly constant across N values from 2 to 16, for both single-pass and repeated GenSelect pipelines. This indicates that larger N can speed up the selection process without sacrificing quality. GenSelect@1 accuracy stays within 72.1 to 72.6 across N values, showing minimal variation. GenSelect@8 accuracy ranges from 73.0 to 73.4, remaining stable as N increases. Higher N reduces the number of required comparison rounds, enabling faster selection without performance degradation.

Applying GenSelect to both DeepSeek-R1-0528 and QwQ-32B markedly improves accuracy over majority voting across all benchmark splits, with the largest relative gains on the HMMT-24-25 set. The repeated GenSelect pipeline with majority voting yields consistent improvements, demonstrating that comparative selection is an effective verification strategy for reasoning models. GenSelect improves performance over majority voting for both base models on every benchmark split. The largest gains from GenSelect appear on the HMMT-24-25 subset for both DeepSeek-R1-0528 and QwQ-32B. DeepSeek-R1-0528 with GenSelect achieves higher overall accuracy than QwQ-32B with GenSelect.

The evaluation is conducted on the Comp-Math-24-25 dataset, which merges AIME and HMMT problems from 2024 and 2025, totaling 256 answerable problems. Comparative selection (GenSelect) generally outperforms majority voting and individual scoring baselines for both DeepSeek-R1-0528 and QwQ-32B across all benchmark splits, with the largest relative gains on the HMMT subset; the advantage is more pronounced for the weaker QwQ model. Increasing the number of comparison rounds does not harm accuracy, allowing faster selection, and GenSelect becomes more beneficial at higher sampling budgets for DeepSeek, while being consistently superior for QwQ.