PRMBench: A Fine-grained and Challenging Benchmark for
Process-Level Reward Models
Abstract
Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since large language models (LLMs) suffer from various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios. However, current benchmarks primarily focus on step correctness, failing to evaluate PRMs’ performance systematically. To address this gap, we introduce PRMBench, a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216 carefully designed problems and 83,456 step-level labels, evaluating models across multiple dimensions, including simplicity, soundness, and sensitivity. In our experiments on 25 models, spanning across both open-source PRMs and LLMs prompted as critic models, we uncover significant weaknesses in current PRMs. These findings reveal the challenges inherent in process-level evaluation and highlight key directions for future research, establishing PRMBench as a robust testbed for advancing research on PRM evaluation and development.
|
|
|
|
Annotator |
|
| |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MR-GSM8K Zeng et al. (2023) | ✗ | ✗ | 1 | ✓ | Human | 2,999 | 8.3 | ||||||||||||
| RMBench Liu et al. (2024) | ✗ | ✗ | 1 | ✗ | Synthetic + Human | 1,327 | - | ||||||||||||
| CriticBench Lin et al. (2024) | ✗ | ✗ | 1 | ✗ | - | - | - | ||||||||||||
| MathCheck-GSM Zhou et al. (2024) | ✗ | ✗ | 1 | ✓ | Synthetic | 516 | - | ||||||||||||
| MR-Ben Zeng et al. (2024b) | ✗ | ✗ | 1 | ✓ | Human | 5,975 | 9.5 | ||||||||||||
| ProcessBench Zheng et al. (2024) | ✓ | ✗ | 1 | ✓ | Human | 3,400 | 7.1 | ||||||||||||
| PRMBench | ✓ | ✓ | 9 | ✓ | Synthetic + Human | 6,216 | 13.4 |
1 Introduction
Recent large language models (LLMs) OpenAI (2024a); OpenAI (2024b); Team (2024a), trained on large-scale reinforcement learning, have achieved significant performance in complex reasoning tasks such as mathematics and code generation Yu et al. (2023); Guo et al. (2024); DeepMind (2024); Luo et al. (2023); Qu et al. (2025). A key factor behind their successes is the use of process reward models (PRMs) Wang et al. (2023); Lightman et al. (2023); Uesato et al. (2022), which can help evaluate the correctness of reasoning steps and train LLMs with appropriate rewards. Qin et al. (2024); Zhang et al. (2024b).
However, during the reasoning process of the LLM, it suffers from various types of errors, while recent PRMs are not able to identify all these error types precisely. For instance, as illustrated in Figure 1, given a question , OpenAI o1 model OpenAI (2024b) generates a reasoning procedure containing errors, where step 2 is redundant, step 5 is inconsistent with step 3, and the theory used in step 5 is incorrect due to deception falling. Under these circumstances, ReasonEval-34B Xia et al. (2024b) and Math-Shepherd-7B Wang et al. (2023) fail to identify these errors accurately. Math-Shepherd-7B fails to recognize step 5 as an error, while ReasonEval-34B correctly identifies step 5 but incorrectly classifies step 4 as an error, indicating the unreliability of current PRMs.
To evaluate the diverse error-detection capabilities of PRMs, we present PRMBench, a comprehensive and fine-grained benchmark specifically designed for assessing PRMs. In contrast to existing process-level benchmarks, which can only evaluate the detection of a single error type Zheng et al. (2024); Zeng et al. (2024b), PRMBench offers a more nuanced evaluation. Specifically, PRMBench systematically assesses the performance of PRMs across diverse error categories, including simplicity, soundness, and sensitivity. Our benchmark includes 6,216 fine-grained data instances spreading across three major evaluation categories and nine sub-categories, whose quality is ensured by professional annotators. Additionally, we utilize style-controlled data curation methods to ensure evaluation samples under consistent difficulty levels, mitigating confounding variables.
In our study, we conduct extensive experiments using PRMBench to evaluate 25 models, including dedicated PRMs and SOTA general-purpose or mathematical LLMs, prompted as critic models. We observe that all PRMs partially grasp multi-step process evaluation. Specifically, Gemini-2-Thinking achieves the best performance of 68.8, but still significantly falls behind the human performance of 83.8. Through extensive analysis, we discover a significant inconsistency between step-level and outcome-level evaluations. By evaluating models with PRMBench, we can assess PRMs’ ability to detect step-level errors and false positives, reducing the risk of outcome hacking Gao et al. (2025). To sum up, our contributions are as follows:
- •
We present PRMBench, the first comprehensive process-level reward model benchmark, comprising 6,216 carefully curated samples and 83,456 step-level labels for a series of evaluations on process-level reward models.
- •
PRMBench covers three carefully-crafted evaluation categories and nine sub-categories including simplicity, soundness, and sensitivity. With these fine-grained evaluation axes, we can conduct tailored assessments of models on their specified capabilities and reveal their potential weaknesses during the rewarding procedure.
- •
Based on our proposed PRMBench, we conduct in-depth pilot experiments on twenty-five models including PRMs along with SOTA LLMs. Our findings uncover critical weaknesses and provide valuable insights to guide future research to improve the capabilities of PRMs.
- •
To facilitate future research, we release the PRM-EVAL toolkit, offering an automated evaluation framework and customizable data generation system. We hope PRMBench will drive progress in step-level reasoning for RLHF and foster further development of more reliable PRMs.
2 Related Work
2.1 Process-level Reward Models
Process-level reward models (PRMs) have shown improvements over traditional outcome-level reward models (ORMs) in enhancing process-level reasoning accuracy and long-process reasoning abilities (Lightman et al., 2023; Uesato et al., 2022). Recently, several PRMs have been proposed for process-level RLHF Wang et al. (2023); Xia et al. (2024b); o1 Team (2024), with Lightman et al. (2023) releasing a large dataset for multi-step reasoning, and Wang et al. (2023) introducing an automatic self-supervised pipeline for process-level labeling. Xia et al. (2024b) uses PRMs as auto-evaluators for multi-step reasoning accuracy. As PRM training and data curation have grown, numerous PRMs o1 Team (2024); Xiong et al. (2024); Team (2024b); Gao et al. (2024) have emerged, along with critic models using LLM-generated feedback McAleese et al. (2024); Zhang et al. (2024a); Gao et al. (2024). However, both PRMs and critic models remain fallible, highlighting the need for comprehensive benchmarks. In this paper, we propose PRMBench, a comprehensive benchmark for evaluating PRMs on fine-grained subjects, establishing a strong foundation for PRM evaluation.
2.2 Reasoning Benchmarks
Evaluating the reasoning capabilities of LLMs is crucial for understanding their potential and limitations. ROSCOE Golovneva et al. (2022) introduces a semantic comparison-based multi-step reasoning accuracy evaluation benchmark. However, recent research suggests that labeled data cannot be assumed to cover all possible solution paths exhaustively Wang et al. (2023); Xia et al. (2024b). To address this, Xia et al. (2024b) uses PRMs or Critic models to evaluate step-level reasoning accuracy. However, PRMs are not always accurate in assessing process-level data, underscoring the need for a comprehensive evaluation benchmark. While other benchmarks Liu et al. (2024); Li et al. (2024a); Lin et al. (2024); Su et al. (2024b) exist, they are not tailored for PRMs and can’t assess step-level reasoning. Some works Zeng et al. (2023); Zeng et al. (2024b); Yan et al. (2024) use LLMs to evaluate reasoning steps, but they often overlook implicit error types. Existing error classification works are not specific to PRMs and lack fine-grained step-level labels Li et al. (2024b). To address these gaps, we propose PRMBench, a solution that offers fine-grained evaluation and detects various error types.
| Overall | NR. | NCL. | ES. | SC. | DC. | CI. | PS. | DR. | MS. | |
| Avg. Steps | 13.4 | 15.3 | 10.3 | 13.8 | 14.2 | 13.3 | 14.2 | 12.7 | 13.4 | 14.1 |
| Avg. Error Steps | 2.1 | 2.0 | 2.8 | 2.8 | 1.6 | 1.8 | 1.7 | 2.5 | 2.3 | 0.0 |
| Avg. First Error Step | 7.8 | 7.8 | 4.9 | 8.0 | 9.1 | 6.8 | 11.4 | 6.2 | 8.3 | N/A |
| Avg. Question Length | 152.7 | 153.6 | 152.5 | 153.5 | 149.7 | 152.5 | 152.7 | 158.0 | 153.5 | 132.2 |
| # of Instances | 6216 | 758 | 758 | 757 | 758 | 757 | 757 | 756 | 750 | 165 |
3 PRMBench
3.1 Evaluation Categories
In this section, we provide a detailed introduction to the evaluation categories of PRMBench, which is organized into three main domains:
- •
Simplicity evaluates the ability of PRMs to detect redundancy in reasoning steps. Although redundant steps do not affect correctness, they increase computational costs and reduce efficiency. Additionally, simplifying the reasoning process enhances the clarity of the problem’s core and improves overall understandability.
- •
Soundness assesses the accuracy of the rewards produced by PRMs. As discussed in Section 1, errors in reasoning can vary in both causes and manifestations Li et al. (2024b). Therefore, we evaluate not only the correctness of rewards but also the fine-grained performance across different error types and their nuances.
- •
Sensitivity measures PRMs’ robustness to details, such as critical conditions or implicit requirements. Sensitivity is vital for ensuring logical completeness and resilience to misleading information Wen et al. (2025), contributing to the overall robustness of PRMs.
Each domain is further divided into detailed sub-categories for a more granular evaluation, which is discussed in detail below. The overall structure of PRMBench along with representative examples of each sub-category are illustrated in Figure 2, and the details of every evaluation category and sub-category are shown in Appendix A.1.
3.1.1 Simplicity
Specifically, the simplicity evaluation category is divided into two sub-categories: Non-Redundancy and Non-Circular Logic, with detailed descriptions provided below:
Non-Redundancy
evaluates the PRMs’ ability to identify redundancy within the reasoning process. Redundancy occurs when the reasoning includes unnecessary steps that do not contribute to the solution, making the process less concise and efficient. These steps can be removed without affecting the correctness of the final solution path.
Non-Circular Logic
assesses the PRMs’ ability to detect circular reasoning within the process. Circular logic is a form of redundancy where the reasoning eventually loops back to a previous step, creating an infinite cycle. This sub-category is treated separately due to the frequent occurrence of circular logic in reasoning processes.
3.1.2 Soundness
We divide the Soundness category into four sub-categories due to its complexity: Empirical Soundness, Step Consistency, Domain Consistency, and Confidence Invariance. The definition of each sub-category is discussed below.
Empirical Soundness
demands PRMs to detect the counterfactual mistakes within the reasoning process. A counterfactual step refers to a statement within a reasoning chain that contradicts established ground truth .
Step Consistency
expects PRMs to detect the step-wise contradiction, which means a conflict between a specific step and other steps within a reasoning path. Given a reasoning path , a step contradiction exists if , where and .
Domain Consistency
requires PRMs to detect domain inconsistency mistakes, which is a special type of counterfactual. It refers to a step within the reasoning chain that uses a statement or theory valid in other domains or cases but is not valid within the current reasoning chain.
Confidence Invariance
demands PRMs to detect over-confident errors, a type of counterfactual where an incorrect statement is made with high confidence, contradicting established ground truth.
3.1.3 Sensitivity
This category includes three sub-categories: Prerequisite Sensitivity, Deception Resistance, and Multi-Solution Consistency, with detailed descriptions provided below.
Prerequisite Sensitivity
requires PRMs to maintain sensitivity to missing conditions or prerequisite mistakes, which means a flaw in the reasoning chain where critical premises, assumptions, or necessary conditions are absent and this omission results in logical gaps, incomplete reasoning, or biased conclusions.
Deception Resistance
demands PRMs to detect the deception or trap within a reasoning process, that is, statements that appear to be correct but are subtly altered to introduce inaccuracies while maintaining the illusion of correctness.
Multi-Solution Consistency
expects PRMs to maintain consistency when faced with different solution paths of the same problem. Concretely, we utilize multiple correct reasoning processes of the same question to test whether the PRM can perform correctly.
3.2 Data Curation
We curate the dataset by extracting metadata and constructing test cases according to our category definitions. Detailed statistics of PRMBench are displayed in Table 2, with the curation procedure outlined below.
Meta Data Extraction
Our metadata is built upon PRM800K Lightman et al. (2023), which provides the questions (), ground truth answers (), and ground truth step-level solution processes (). We select completely correct solutions from both the training and test sets, filtering out low-quality instances to establish our ground truth answers.
Test Case Construction
Each test case instance is represented as , where denotes the test question and represents the test solution process, which may include errors. With class-specific prompts, as demonstrated in Appendix E.1, we query GPT-4o OpenAI (2024a) to modify the ground-truth reasoning process into versions containing erroneous steps. For the multi-solution, we leverage the newly proposed multi-step reasoning model QwQ11 1 Qwen/QwQ-32B-Preview: https://huggingface.co/Qwen/QwQ-32B-Preview Team (2024a) to generate candidate answers for the given questions. These answers are then filtered to exclude unreasonable or incorrect ones, resulting in multi-solution reasoning processes for a single question.
| Model | Overall | Simplicity | Soundness | Sensitivity | |||||||||
| NR. | NCL. | Avg. | ES | SC. | DC. | CI | Avg. | PS | DR. | MS. | Avg. | ||
| Human Performance | 83.8 | 80.2 | 81.7 | 81.0 | 84.8 | 85.0 | 81.3 | 86.1 | 84.3 | 81.6 | 82.1 | 96.2 | 86.0 |
| Open-source Process Level Reward Models | |||||||||||||
| Skywork-PRM-1.5B | 61.1 | 52.0 | 56.4 | 54.2 | 64.8 | 64.9 | 63.3 | 66.5 | 64.9 | 57.5 | 63.3 | 91.1 | 70.7 |
| Skywork-PRM-7B | 65.1 | 56.4 | 62.8 | 59.6 | 69.4 | 67.1 | 67.7 | 69.9 | 68.5 | 60.9 | 65.8 | 93.2 | 73.3 |
| Llemma-PRM800k-7B | 52.0 | 49.3 | 53.4 | 51.4 | 56.4 | 47.1 | 46.7 | 53.3 | 50.9 | 51.0 | 53.5 | 93.6 | 66.0 |
| Llemma-MetaMath-7B | 50.5 | 50.2 | 50.5 | 50.3 | 51.9 | 47.6 | 44.4 | 52.1 | 49.0 | 50.5 | 51.3 | 96.0 | 66.0 |
| Llemma-oprm-7B | 50.3 | 48.7 | 49.3 | 49.0 | 54.2 | 46.8 | 44.5 | 53.5 | 49.8 | 49.2 | 51.3 | 91.8 | 64.1 |
| MATHMinos-Mistral-7B | 54.2 | 48.8 | 54.0 | 51.4 | 57.0 | 52.1 | 50.7 | 57.8 | 54.4 | 52.8 | 55.8 | 91.1 | 66.5 |
| MathShepherd-Mistral-7B | 47.0 | 44.0 | 50.3 | 47.1 | 49.4 | 44.5 | 41.3 | 47.7 | 45.7 | 47.2 | 48.6 | 86.1 | 60.7 |
| ReasonEval-7B | 60.1 | 61.0 | 50.1 | 55.6 | 62.1 | 65.9 | 61.5 | 66.0 | 63.9 | 55.7 | 58.0 | 99.5 | 71.1 |
| ReasonEval-34B | 60.5 | 54.8 | 48.1 | 51.5 | 66.4 | 60.3 | 57.8 | 67.5 | 63.0 | 57.7 | 64.3 | 97.2 | 73.1 |
| RLHFlow-PRM-Mistral-8B | 54.4 | 46.1 | 47.3 | 46.7 | 56.6 | 55.1 | 54.4 | 63.8 | 57.5 | 51.5 | 56.2 | 97.9 | 68.5 |
| RLHFlow-PRM-Deepseek-8B | 54.2 | 46.4 | 48.9 | 47.6 | 55.7 | 55.0 | 53.2 | 66.2 | 57.5 | 49.0 | 55.4 | 99.8 | 68.1 |
| Qwen2.5-Math-PRM-7B | 65.5 | 49.0 | 55.1 | 52.1 | 71.8 | 67.3 | 66.3 | 78.5 | 71.0 | 57.6 | 69.1 | 99.7 | 75.5 |
| Qwen2.5-Math-PRM-72B | 68.2 | 50.4 | 58.8 | 54.6 | 73.7 | 71.1 | 72.2 | 78.6 | 73.9 | 60.3 | 71.2 | 99.4 | 77.0 |
| Pure-PRM-7B | 65.3 | 49.2 | 55.2 | 52.2 | 71.1 | 68.8 | 64.0 | 76.9 | 70.2 | 60.3 | 69.2 | 98.0 | 75.8 |
| Avg. | 57.7 | 50.5 | 52.9 | 51.7 | 61.5 | 58.1 | 56.3 | 64.2 | 60.0 | 54.4 | 59.5 | 95.3 | 69.7 |
| Open LLMs, Prompted as Critic Models | |||||||||||||
| MetaMath-7B | 49.7 | 48.9 | 46.9 | 47.9 | 47.3 | 48.9 | 48.4 | 48.8 | 48.3 | 46.5 | 48.3 | 98.0 | 64.2 |
| MetaMath-13B | 49.4 | 50.3 | 44.4 | 47.3 | 47.8 | 47.4 | 49.4 | 48.1 | 48.2 | 49.0 | 48.1 | 99.5 | 65.5 |
| Qwen2.5-Math-72B | 57.4 | 55.3 | 54.9 | 55.1 | 55.5 | 71.6 | 58.1 | 59.1 | 61.1 | 47.4 | 53.8 | 100.0 | 67.1 |
| QwQ-Preview-32B | 63.6 | 57.2 | 55.6 | 56.4 | 67.4 | 72.3 | 66.2 | 66.9 | 68.2 | 57.8 | 62.7 | 100.0 | 73.5 |
| R1-Distill-Llama3.1-70B | 57.5 | 49.5 | 48.1 | 48.8 | 61.4 | 65.5 | 65.8 | 61.1 | 63.4 | 48.8 | 54.1 | 100.0 | 67.6 |
| R1-Distill-Qwen-7B | 52.6 | 32.9 | 37.9 | 35.4 | 47.3 | 54.1 | 48.4 | 48.0 | 49.4 | 45.6 | 46.8 | 100.0 | 64.1 |
| DeepSeek-R1† | 67.8 | 63.0 | 62.7 | 62.9 | 68.2 | 68.5 | 73.5 | 75.4 | 71.4 | 63.3 | 68.0 | 100.0 | 77.1 |
| Avg. | 56.8 | 51.0 | 50.1 | 50.5 | 56.4 | 61.2 | 58.5 | 58.2 | 58.6 | 51.2 | 54.5 | 99.6 | 68.5 |
| Proprietary LLMs, Prompted as Critic Models | |||||||||||||
| GPT-4o | 66.8 | 57.0 | 62.4 | 59.7 | 72.0 | 69.7 | 70.7 | 71.1 | 70.9 | 62.5 | 65.7 | 99.2 | 75.8 |
| o1-mini† | 68.8 | 65.6 | 63.7 | 64.6 | 74.5 | 67.7 | 73.8 | 72.3 | 72.1 | 61.8 | 64.8 | 100.0 | 75.5 |
| Gemini-2.0-flash-exp | 66.0 | 67.2 | 58.1 | 62.7 | 70.4 | 65.7 | 66.0 | 67.3 | 67.3 | 61.8 | 66.2 | 98.2 | 75.4 |
| Gemini-2.0-thinking-exp-1219 | 68.8 | 68.5 | 63.8 | 66.2 | 72.9 | 71.3 | 71.0 | 71.8 | 71.8 | 60.3 | 65.7 | 99.8 | 75.3 |
| Avg. | 67.6 | 64.6 | 62.0 | 63.3 | 72.4 | 68.6 | 70.4 | 70.7 | 70.5 | 61.6 | 65.6 | 99.3 | 75.5 |
3.3 Quality Control
To ensure a high-quality dataset, we implement a series of steps to filter out unqualified data and maintain data integrity. The specific procedures are outlined below:
Feature Filtering
Our data curation procedure imposes strict structural requirements on the generated responses, where any outputs that do not satisfy these specifications cannot be considered valid for accurately assessing the performance of PRMs. However, even with detailed instructions, LLMs cannot consistently generate outputs that fully adhere to the required structure Asai et al. (2024); Zeng et al. (2024a); Su et al. (2024a). To maintain high data quality, we define stringent filtering rules to exclude instances that fail to meet the necessary structural criteria. Detailed structural requirements are provided in Appendix E.1, and the full description of our data generation process can be found in the supplementary materials.
Human Verification
Furthermore, to further ensure the quality of the data, we manually evaluate 10% of the total instances. We focus on two key qualities for each data instance: ❶ Correctness of modification: Whether the modifications made to the data instance are correct and reasonable. ❷ Difference in the modification: Whether the modified data instance differs from the original. We recruited five volunteers to evaluate our proposed PRMBench and observe over 92% qualification rate on the correctness metric and over 98% qualification rate on the difference metric. The details of human annotation are provided in Appendix A.3.1, and instructions for annotators are provided in Appendix D. This validation ensures the overall quality of our dataset and its suitability for studying process-level language reward models.
4 Experiments
4.1 Models
To provide a comprehensive evaluation of various models on PRMBench, we select a wide range of models, including open-source PRMs like QwenPRM Zhang et al. (2025) and RLHFlowPRMs Xiong et al. (2024), as well as LLMs prompted as critic models, such as o1-mini OpenAI (2024b) and DeepSeek R1 Guo et al. (2025). A complete list of these models can be found in Appendix B.2. Additionally, we present the human evaluation results, with details available in Appendix A.3.2.
All PRMs and LLMs are evaluated on the complete PRMBench dataset, except for o1-mini and DeepSeek-R1, which are evaluated on a subset of PRMBench comprising 394 samples, proportionally selected to reflect the class distribution, in order to reduce evaluation costs.
Considering the complexity of the task, which involves question comprehension, evaluation of the provided processes, and adherence to format constraints, few-shot demonstration setups are employed to help the model adapt to the output format through In-Context Learning (ICL) examples. Specifically, we use two-shot examples when prompting general-purpose LLMs. The impact of few-shot settings is discussed in Section 5.4
4.2 Evaluation Metrics
Given our emphasis on evaluating the error detection capabilities, we use the negative F1 score as a metric for error detection performance. However, this metric may be affected by the inherent biases of models. To mitigate this and provide a unified, normalized score that reflects the overall competency of the evaluated model, following Zheng et al. (2024), we introduce a metric called PRMScore, defined formally in Equation 1.
|
|
(1) |
Where F1 and F1neg refer to F1 scores and negative F1 scores respectively. and are weights that are designed to maximize the differentiation between different models. The detailed evaluation procedure is provided in Appendix B.3. Besides, we also provide results of all evaluation categories in fine-grained metrics in Appendix B.4.
4.3 Main Results
The main results are shown in Table 3. Some observations can be summarized as follows:
| Model | Accuracy | PRM Score | Sim. | |
|---|---|---|---|---|
| Pos. | Neg. | |||
| ReasonEval-7B | 95.5 | 21.2 | 60.0 | 91.6 |
| ReasonEval-34B | 79.1 | 48.4 | 60.5 | 82.8 |
| Skywork-7B | 30.1 | 79.7 | 36.2 | 74.3 |
| RLHFlow-DeepSeek-8B | 95.0 | 13.0 | 54.2 | 95.0 |
| GPT-4o | 82.9 | 58.2 | 66.8 | 76.6 |
| Gemini-2-thinking | 89.0 | 49.8 | 68.8 | 82.0 |
| Random | 50.0 | 50.0 | 50.0 | 79.4 |
The PRMs partially grasp multi-step process evaluation
Our analysis indicates that, although Gemini-2-Thinking achieves the highest performance among all evaluated models, its score is still significantly lower than human performance (68.8 vs. 83.8), highlighting substantial room for improvement in multi-step process evaluation. Some models even perform worse than random guessing, highlighting their limited reliability and potential training biases. Notably, the best open-source PRMs fail to match the performance of general-purpose proprietary LLMs, which suggests that even specifically trained PRMs still lag behind leading general-purpose models. We provide a detailed error analysis in Appendix C.1.
Simplicity is more challenging for PRMs
Our analysis highlights significant variations in model reasoning capabilities across evaluation categories. For instance, in the Sensitivity category, ReasonEval-34B performs relatively well, achieving an average score of 73.1. Especially in the Multi-Solutions sub-category, it excels with a PRMScore of 97.2, approaching near-perfect classification accuracy. This suggests models perform relatively better on correct instance judgment. However, its performance declines markedly in more complex scenarios. In the Simplicity category, ReasonEval-34B’s PRMScore drops to 51.5, suggesting partially reliable performance.
Furthermore, to broaden the domain coverage of PRMBench, we additionally collect STEM-related data and construct PRMBench-STEM, which is designed to evaluate PRM performance across other domains. The data construction methodology and experimental results for PRMBench-STEM are provided in the appendix B.5.
5 Detailed Analysis
5.1 Inference Bias within PRMs
As shown in Table 3, most open-source PRMs exhibit significant bias during evaluation, with some models performing worse than random guessing. suggesting the potential presence of bias within the inference procedure for our test cases. To validate this assumption, we compare the difference of models’ performance on positive and negative instances. As shown in Table 4, some models exhibit a clear bias during evaluation, often favoring positive rewards. For instance, ReasonEval-7B and RLHFlow-DeepSeek-8B achieve over 95% accuracy on positive-labeled steps but only attain an average of 17% accuracy on negative-labeled steps. Although proprietary LLMs outperform open-source PRMs, they also exhibit bias with a comparatively milder reward tendency.
Additionally, to further investigate inference bias, we evaluate the reward similarity of models’ performance between completely correct reasoning processes and our test cases. The solution-level similarity is defined as , where denotes the average step accuracy within a solution. The results, shown in Table 4, reveal that certain models, such as ReasonEval-7B and RLHFlow-DeepSeek-8B, exhibit significantly higher similarity than the normal similarity score (79.4), showcasing potential limitations in differentiating positive and negative steps.
| Model | 0-shot | 1-shot | 2-shot |
|---|---|---|---|
| GPT-4o | 68.1 | 68.2 | 66.8 |
| Gemini-2-flash | 65.3 | 64.9 | 66.0 |
| Gemini-2-thinking | 67.8 | 67.8 | 68.8 |
5.2 Performance across Different Steps
PRMBench includes a wide range of error step positions. The distribution of error positions is illustrated in Figure 3. While differences exist across categories, the overall pattern remains consistent: all categories peak in frequency at step 5 and gradually decrease thereafter. This raises an interesting question: Does the variation in step positions affect model performance? To investigate, we focus on error steps to assess how erroneous step positions influence model accuracy. As depicted in Figure 4, proprietary LLMs maintain stable performance across different error step positions. In contrast, PRMs, including Math-Shepherd-7B and ReasonEval-7B, show a gradual improvement in performance as error step positions increase.
5.3 Impacts of ICL Settings
In this section, we investigate the impact of different ICL few-shot numbers on models’ performance. We vary the number of ICL few-shots to 0, 1, and 2 to examine whether increasing the few-shot number enhances the performance of generative models prompted as critic models. As shown in Table 5, for the Gemini-series models, a subtle improvement in performance is observed with a few-shot setup. However, for GPT-4o, no significant improvement is detected, and in some cases, a larger few-shot number even results in a decline in performance. These findings suggest that a few-shot approach exerts only a subtle impact on model performance on PRMBench.
| Method | MATH | OlymBen | Avg. | PRMScore |
|---|---|---|---|---|
| Pass@8 | 96.2 | 79.8 | 88.0 | - |
| Maj@8 | 71.8 | 40.3 | 56.1 | - |
| SkyworkPRM-7B | 90.0 | 60.1 | 75.1 | 65.1 |
| LlemmaPRM-7B | 87.4 | 58.3 | 72.8 | 52.0 |
| MATHMinos-7B | 88.3 | 59.1 | 73.7 | 54.2 |
| MathShepherd-7B | 88.6 | 60.0 | 74.3 | 47.0 |
| ReasonEval-7B | 87.0 | 58.4 | 72.7 | 60.1 |
| RLHFlowPRM-8B | 87.6 | 58.5 | 73.0 | 54.2 |
| Qwen2.5-PRM-7B | 88.0 | 58.7 | 73.4 | 65.5 |
| Standard Dev () | 0.91 | 0.71 | 0.81 | 6.40 |
| Somers’ D | -0.05 | 0.05 | -0.05 | 1.00 |
5.4 Comparison between BoN Evaluation and PRMBench
We compare the results between our PRMBench and Best-of-N (BoN) evaluation to observe the correlation. Following Zhang et al. (2025); Yang et al. (2024a), we sampled eight responses (i.e., N=8) from Qwen-QwQ across multiple mathematical benchmarks, including GSM8K Cobbe et al. (2021), MATH Hendrycks et al. (2021), Olympiad Bench He et al. (2024a) and MMLU Hendrycks et al. (2020). During evaluation, the PRMs are tasked with assigning a validity score to each step within every candidate response. The overall score for each candidate response is calculated by multiplying the individual step scores, as outlined in Lightman et al. (2023). We also provide majority voting as an baseline and pass@8 as the upper bound. The experiment setting and full BoN evaluation results are shown in Appendix C.2.
Although PRMs excel at selecting correct outcomes, they struggle with step-level reward hacking. As shown in Table 6, the average Somers’ D correlation between PRMBench and BoN is only -0.05, highlighting the inconsistency between step-level and outcome-level evaluation. For instance, Math-Shepherd-7B achieves a PRMScore of 47.0 with 51.3% accuracy in false-positive scenarios within PRMBench, but outperforms most PRMs, including the state-of-the-art QwenPRM-7B, in the BoN evaluation (74.3 vs. 73.4). This inconsistency reveals that PRMs are suboptimal at detecting step-level errors and false positives, exposing potential reward hacking (Gao et al., 2025). Compared to BoN, PRMBench provides a better distinction between models, with a higher standard deviation (6.40 vs. 0.81), indicating its greater sensitivity to fine-grained differences in reasoning steps. Furthermore, based on the discoveries from PRMBench, we provide further discussions including several promising directions for future exploration, which are included in the appendix F.
6 Conclusion
In this paper, we investigate a crucial question: Can existing PRMs detect various types of erroneous reasoning steps and provide reasonable rewards? To address this, we introduce PRMBench, a benchmark characterized by its fine-grained evaluation categories and challenging requirements. We carefully curate 6,216 data samples with 83,456 step-level labels through LLMs and human filtering. PRMBench can be used to evaluate different process-labeling models, ensuring its general applicability. Through a comprehensive evaluation of existing PRMs and generative LLMs prompted as critic models, we can observe that PRMs exhibit partial capability in multi-step process evaluation, showcasing significant room for improvement. Furthermore, we highlight the critical need for detecting detailed error types and conducting comprehensive evaluations of PRMs. Despite these advances, enhancing the reward accuracy of PRMs and improving models’ reasoning abilities remain open research challenges. We encourage future work to leverage and expand upon PRMBench to address these issues.
7 Acknowledgement
We gratefully acknowledge the support and resources provided by the Shanghai Artificial Intelligence Laboratory, which were essential to the successful completion of this research. We are also grateful to Zhiyi Li at The Hong Kong University of Science and Technology for offering valuable examples and inspiration for this study.
8 Limitations
There are still some limitations in our work, which are summarized below:
- •
While the PRMBench is large and comprehensive, comprising 6,216 samples and 83,456 step-level labels, a larger dataset could provide more robust evaluation and training opportunities. As the data construction method is flexible and data-agnostic, it can be adapted to more different data sources. We will continuously expand our dataset and explore PRM training in future versions.
- •
We evaluate the error-detection capabilities in terms of accuracy. However, a more detailed analysis, including the activation of the model’s neurons and hidden states Zhang et al. (2023), would offer a deeper insight into improvements for PRMs. This limitation is not unique to our study and is common in most evaluations of Large Language Models.
- •
PRMBench currently focuses on textual reasoning. In future work, we will further explore its generalization to multimodal reasoning processes, such as those involving the integration of text and images Su et al. (2025); Xia et al. (2024a), to assess error detection capabilities in cross-modal scenarios.
References
- Asai et al. (2024) Akari Asai, Jacqueline He*, Rulin Shao*, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Tian, D’arcy Mike, David Wadden, Matt Latzke, Minyang, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Dan Weld, Graham Neubig, Doug Downey, Wen-tau Yih, Pang Wei Koh, and Hannaneh Hajishirzi. 2024. OpenScholar: Synthesizing scientific literature with retrieval-augmented language models. Arxiv.
- Cheng et al. (2025) Jie Cheng, Lijun Li, Gang Xiong, Jing Shao, and Yisheng Lv. 2025. Pure: Prm is still effective and compute-efficient for llm math reasoning. https://github.com/CJReinforce/PURE.
- Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
- DeepMind (2024) DeepMind. 2024. Gemini 2.0 flash experimental. https://deepmind.google/technologies/gemini/flash/. Accessed: 2024-12-25.
- Gao et al. (2024) Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Junyang Lin, Chang Zhou, Wen Xiao, et al. 2024. Llm critics help catch bugs in mathematics: Towards a better mathematical verifier with natural language feedback. CoRR.
- Gao et al. (2025) Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang, and Yi Wu. 2025. On designing effective RL reward at training time for LLM reasoning.
- Golovneva et al. (2022) Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2022. Roscoe: A suite of metrics for scoring step-by-step reasoning. arXiv preprint arXiv:2212.07919.
- Guo et al. (2025) Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
- Guo et al. (2024) Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196.
- Hao et al. (2025) Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. 2025. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444.
- He et al. (2024a) Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024a. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008.
- He et al. (2024b) Conghui He, Wei Li, Zhenjiang Jin, Chao Xu, Bin Wang, and Dahua Lin. 2024b. Opendatalab: Empowering general artificial intelligence with open datasets. arXiv preprint arXiv:2407.13773.
- Hendrycks et al. (2020) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
- Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
- Li et al. (2024a) Lei Li, Yuancheng Wei, Zhihui Xie, Xuqing Yang, Yifan Song, Peiyi Wang, Chenxin An, Tianyu Liu, Sujian Li, Bill Yuchen Lin, et al. 2024a. Vlrewardbench: A challenging benchmark for vision-language generative reward models. arXiv preprint arXiv:2411.17451.
- Li et al. (2024b) Xiaoyuan Li, Wenjie Wang, Moxin Li, Junrong Guo, Yang Zhang, and Fuli Feng. 2024b. Evaluating mathematical reasoning of large language models: A focus on error identification and correction. arXiv preprint arXiv:2406.00755.
- Lightman et al. (2023) Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
- Lin et al. (2024) Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, and Yujiu Yang. 2024. Criticbench: Benchmarking llms for critique-correct reasoning. arXiv preprint arXiv:2402.14809.
- Liu et al. (2024) Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. 2024. Rm-bench: Benchmarking reward models of language models with subtlety and style. arXiv preprint arXiv:2410.16184.
- Luo et al. (2023) Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583.
- McAleese et al. (2024) Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. 2024. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215.
- o1 Team (2024) Skywork o1 Team. 2024. Skywork-o1 open series. https://huggingface.co/Skywork.
- OpenAI (2024a) OpenAI. 2024a. Gpt-4o system card. https://cdn.openai.com/gpt-4o-system-card.pdf. Accessed: 2024-09-26.
- OpenAI (2024b) OpenAI. 2024b. Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/.
- Qin et al. (2024) Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. 2024. O1 replication journey: A strategic progress report–part 1. arXiv preprint arXiv:2410.18982.
- Qu et al. (2025) Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. 2025. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.21614.
- Su et al. (2025) Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617.
- Su et al. (2024a) Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. 2024a. Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in llm. arXiv preprint arXiv:2408.12076.
- Su et al. (2024b) Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, and Yu Cheng. 2024b. Timo: Towards better temporal reasoning for language models. arXiv preprint arXiv:2406.14192.
- Team (2024a) Qwen Team. 2024a. Qwq: Reflect deeply on the boundaries of the unknown.
- Team (2024b) ScalableMath Team. 2024b. Easy-to-hard generalization models. https://huggingface.co/ScalableMath.
- Uesato et al. (2022) Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275.
- Wang et al. (2023) Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. 2023. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. CoRR, abs/2312.08935.
- Wen et al. (2025) Xueru Wen, Jie Lou, Yaojie Lu, Hongyu Lin, XingYu, Xinyu Lu, Ben He, Xianpei Han, Debing Zhang, and Le Sun. 2025. Rethinking reward model evaluation: Are we barking up the wrong tree? In The Thirteenth International Conference on Learning Representations.
- Xia et al. (2024a) Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. 2024a. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXiv:2410.13085.
- Xia et al. (2024b) Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. 2024b. Evaluating mathematical reasoning beyond accuracy. arXiv preprint arXiv:2404.05692.
- Xiong et al. (2024) Wei Xiong, Hanning Zhang, Nan Jiang, and Tong Zhang. 2024. An implementation of generative prm. https://github.com/RLHFlow/RLHF-Reward-Modeling.
- Yan et al. (2024) Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, et al. 2024. Errorradar: Benchmarking complex mathematical reasoning of multimodal large language models via error detection. arXiv preprint arXiv:2410.04509.
- Yang et al. (2024a) An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.
- Yang et al. (2024b) An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024b. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122.
- Yu et al. (2023) Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284.
- Zeng et al. (2024a) Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024a. Evaluating large language models at evaluating instruction following. In The Twelfth International Conference on Learning Representations.
- Zeng et al. (2023) Zhongshen Zeng, Pengguang Chen, Shu Liu, Haiyun Jiang, and Jiaya Jia. 2023. Mr-gsm8k: A meta-reasoning benchmark for large language model evaluation. arXiv preprint arXiv:2312.17080.
- Zeng et al. (2024b) Zhongshen Zeng, Yinhong Liu, Yingjia Wan, Jingyao Li, Pengguang Chen, Jianbo Dai, Yuxuan Yao, Rongwu Xu, Zehan Qi, Wanru Zhao, Linling Shen, Jianqiao Lu, Haochen Tan, Yukang Chen, Hao Zhang, Zhan Shi, Bailin Wang, Zhijiang Guo, and Jiaya Jia. 2024b. Mr-ben: A meta-reasoning benchmark for evaluating system-2 thinking in llms. CoRR, abs/2406.13975.
- Zhang et al. (2024a) Di Zhang, Jingdi Lei, Junxian Li, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, et al. 2024a. Critic-v: Vlm critics help catch vlm errors in multimodal reasoning. arXiv preprint arXiv:2411.18203.
- Zhang et al. (2024b) Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. 2024b. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884.
- Zhang et al. (2025) Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301.
- Zhang et al. (2023) Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923.
- Zheng et al. (2024) Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2024. Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559.
- Zhou et al. (2024) Zihao Zhou, Shudong Liu, Maizhen Ning, Wei Liu, Jindong Wang, Derek F Wong, Xiaowei Huang, Qiufeng Wang, and Kaizhu Huang. 2024. Is your model really a good math reasoner? evaluating mathematical reasoning with checklist. arXiv preprint arXiv:2407.08733.
Appendix A Detailed Information for PRMBench
A.1 Evaluation Categories
In this section, We provide detailed information on our evaluation categories. The hierarchical categories, corresponding descriptions, and illustrations are shown in Figure 5. We carefully curated 6,216 data samples and 83,456 step-level labels. The benchmark spreads across three main evaluation categories: simplicity, soundness, and sensitivity. Among them, Simplicity comprises two sub-categories: non-redundancy and non-circular Logic. Soundness includes four main sub-categories: empirical soundness, step consistency, domain consistency, and confidence invariance. Finally, Sensitivity mainly evaluates models in three main parts: prerequisite sensitivity, deception resistance, and multi-solution consistency. The detailed descriptions and illustrations of each sub-category are shown in Figure 5.
A.1.1 Simplicity
Specifically, the Simplicity evaluation category is divided into two sub-categories: Non-Redundancy and Non-Circular Logic, with detailed descriptions provided below:
Non-Redundancy
requires PRM to detect the redundancy within the reasoning procedure. The redundancy situation refers to a process that is not the most concise or efficient, as it includes one or more redundant steps that can be removed without affecting the correctness of the overall solution path. For example, as shown in Figure 5, if represents a correct inference chain, the redundant reasoning procedure can be displayed as . where represents one or more redundant steps .
Non-Circular Logic
In this sub-category, PRMs are required to detect the potential circular logic within the reasoning process. Circular logic is a specific form of redundancy, distinct from general redundancy, in that it finally loops back to a previous reasoning step. For example, as shown in Figure 2, if represents a correct inference chain, circular logic can be formulated as , where the reasoning starts at step , progresses through a sequence of steps, and ultimately loops back to . We list Non-Circular Logic separately due to its common occurrence in reasoning processes.
| Step Descriptions | GT | ReasonE | MathS | GPT-4o | Gemini | ||||
| Question |
|
- | - | - | - | - | |||
| Step 1 |
|
✓ | 89.1 | 87.5 | 100.0 | 100.0 | |||
| Step 2 |
|
✓ | 77.5 | 82.8 | 100.0 | 100.0 | |||
| Step 3 |
|
✓ | 71.4 | 76.7 | 100.0 | 100.0 | |||
| Step 4 |
|
✓ | 20.5 | 76.9 | 100.0 | 100.0 | |||
| Step 5 |
|
✓ | 56.0 | 83.7 | 100.0 | 100.0 | |||
| Step 6 |
|
✓ | 63.5 | 80.4 | 100.0 | 100.0 | |||
| Step 7 |
|
✓ | 84.1 | 79.1 | 50.0 | 50.0 | |||
| Step 8 |
|
✗ | 73.6 | 74.1 | 0.0 | 0.0 | |||
| Step 9 |
|
✗ | 79.1 | 78.5 | 0.0 | 100.0 | |||
| Step 10 |
|
✗ | 67.4 | 78.0 | 0.0 | 0.0 | |||
| Step 11 |
|
✗ | 68.1 | 72.1 | 0.0 | 100.0 | |||
| Step 12 |
|
✗ | 49.2 | 73.8 | 0.0 | 100.0 | |||
| Step 13 |
|
✗ | 35.1 | 73.8 | 0.0 | 100.0 | |||
| Step 14 |
|
✗ | 72.5 | 64.3 | -50.0 | 100.0 | |||
| Step 15 |
|
✗ | 6.3 | 35.8 | 0.0 | -100.0 | |||
| Step 16 |
|
✓ | 22.6 | 43.5 | -100.0 | -100.0 | |||
| Final Acc. | - | 100 | 56.2 | 50.0 | 93.8 | 62.5 | |||
| Reason |
| ||||||||
| Abbr. | Full Name | Evaluation Category |
|---|---|---|
| NR. | Non-Redundancy | Simplicity |
| NCL. | Non-Circular Logic | Simplicity |
| ES. | Empirical Soundness | Soundness |
| SC. | Step Consistency | Soundness |
| DC. | Domain Consistency | Soundness |
| CI. | Confidence Invariance | Soundness |
| PS. | Prerequisite Sensitivity | Sensitivity |
| DR. | Deception Resistance | Sensitivity |
| MS. | Multi-Solution Consistency | Sensitivity |
A.1.2 Soundness
We divide the Soundness category into four sub-categories due to its complexity: Empirically Soundness, Step Consistency, Domain Consistency, and Confidence Invariance. The definition of each sub-category is discussed below.
Empirically Soundness
demands PRM to detect the implicit counterfactual mistakes within the reasoning process. A counterfactual step refers to a statement within a reasoning chain that contradicts established ground truth . Such contradictions can arise from relying on outdated theories, omitting critical constraints in theory, or incorporating erroneous assumptions.
Step Consistency
expects PRM to detect the implicit step-wise contradiction, which means a conflict between a specific step and other steps within a reasoning path. Given a reasoning path , a step contradiction exists if , where and .
Domain Consistency
Under this circumstance, PRMs are required to detect potential domain inconsistency mistakes, where domain inconsistency is a special type of counterfactual. It refers to a step within the reasoning chain that uses a statement or theory valid in other domains or cases but is not valid within the current reasoning chain.
Confidence Invariance
demands the PRM to detect over-confident hallucinations, a type of counterfactual where an incorrect statement is made with unwarranted certainty, contradicting established ground truth.
A.1.3 Sensitivity
This category includes three sub-categories: Prerequisite Sensitivity, Deception Resistance, and Multi-Solution Consistency, with detailed descriptions provided below.
Prerequisite Sensitivity
requires the PRM to maintain sensitivity to missing conditions or prerequisite mistakes, which means a flaw in the reasoning chain where critical premises, assumptions, or necessary conditions are absent. This omission results in logical gaps, incomplete reasoning, or biased conclusions. For example, when a missing condition occurs, the model is required to solve the problem through case analysis or further investigation. However, the answer becomes incorrect if the model overlooks the missing condition and proceeds with standard reasoning methods.
Deception Resistancy
demands the PRM to detect the implicit deception or trap within a reasoning process, that is, statements that appear to be correct or align with ground truth but are subtly altered to introduce inaccuracies while maintaining the illusion of correctness.
Multi-Solution Consistency
expects the PRM to maintain consistency when faced with different solution paths of the same problem. Concretely, to evaluate the sensitivity and the generalizability of PRMs, we utilize multiple correct reasoning processes of the same question to test whether the PRM can perform correctly.
A.2 Examples For Different Evaluation Categories
In this section, we provide detailed examples of the various evaluation categories and their corresponding sub-categories. The data instance examples are displayed in Figure 7-18. All datasets used in this work are publicly available and have been released by their original creators, who are responsible for ensuring privacy protection. These datasets are utilized in accordance with their respective licenses and intended purposes, without introducing any harmful or sensitive information.
A.3 Human Annotation Settings
A.3.1 Quality Control
In the second stage of the quality control process, we recruited five volunteers, each holding a bachelor’s degree or an equivalent qualification, to assess the correctness and validity of PRMBench. To facilitate high-quality labeling, we utilize LabelLLM He et al. (2024b) to help the data annotation procedure, as shown in Figure 6. The annotators’ instructions are shown in Appendix D.
A.3.2 Human Performance Evaluation
For human performance evaluation, we recruited three volunteers, each holding a bachelor’s degree or an equivalent qualification, to assist with data annotation. Following Hao et al. (2025), we randomly selected 50 instances from each sub-category, resulting in a mini-test set of 450 samples. Each annotator was responsible for three subsets, and the results are presented in Table 3.
Appendix B Detailed Experiment Results
B.1 Abbreviation Of Sub-Categories
The full names of abbreviations used in our tables are shown in Table 8.
B.2 Models
To provide a comprehensive evaluation of various models on PRMBench, we select a diverse set of models, including both open-source PRMs and different types of LLMs configured as critic models. Specifically, the open-source PRMs include Skywork-PRM-1.5B/7B o1 Team (2024), LlemmaPRMs Team (2024b), MathMinosPRM Gao et al. (2024), MathShepherd-Mistral-7B Wang et al. (2023), ReasonEval-7B/34B Xia et al. (2024b), Pure-PRM-7B Cheng et al. (2025) and Qwen-PRM-7B/72B Zhang et al. (2025). Additionally, we evaluate state-of-the-art general-purpose LLMs, including the open-source MetaMath-7B/34B Yu et al. (2023), Qwen2.5-Math-72B Yang et al. (2024b), DeepSeek-R1 and its distill series Guo et al. (2025), as well as closed-source LLMs such as GPT-4o OpenAI (2024a), Gemini-2.0-flash DeepMind (2024), and multi-step reasoning-enhanced LLMs like the o1 series models OpenAI (2024b) and Gemini-2-Thinking DeepMind (2024).
All models used in this work are publicly available and have been released by their original authors, who are responsible for ensuring privacy protection. These models are utilized in accordance with their respective licenses and intended purposes, without any modifications.
B.3 Evaluation Procedure
For each annotated question-solution pair, the reward models are tasked with evaluating the correctness and redundancy of each step, assigning a step-level validity score and a step-level redundancy score to each step. We subsequently utilize the specified threshold of each model to obtain the prediction indicating whether the step is correct or redundant. This task is therefore framed as a binary classification problem. Thus we can utilize the evaluation metric defined in Section 4.2 to evaluate the performance of models on PRMBench.
B.4 Detailed Results of PRMBench
B.5 Detailed Results of PRMBench-STEM
We extend our benchmark by collecting additional data from various scientific domains, including physics, chemistry, and biology, and construct PRMBench-STEM using a similar data curation methodology as described in the main paper. The statistics of PRMBench-STEM are presented in Table 9. We then evaluate several representative PRMs on PRMBench-STEM, and the results are summarized in Tables 10, 11, and 12 for the Biology, Chemistry, and Physics subsets, respectively. As shown, PRMs exhibit weaker performance on PRMBench-STEM compared to their performance on the original PRMBench. Notably, the Simplicity category remains the most challenging, consistent with our earlier observations.
| Domain | # of Instances | Avg. Step Num | Avg. Error Num |
|---|---|---|---|
| Physics | 1619 | 7.32 | 2.74 |
| Chemistry | 1543 | 7.74 | 2.78 |
| Biology | 2342 | 6.76 | 2.49 |
| Model | Overall | Simplicity | Soundness | Sensitivity | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NR. | NCL. | Avg. | ES | SC. | DC. | CI | Avg. | PS | DR. | Avg. | ||
| Open-source Process Level Reward Models | ||||||||||||
| Skywork-PRM-1.5B | 47.3 | 46.7 | 41.6 | 44.2 | 45.0 | 54.1 | 55.3 | 49.6 | 51.0 | 43.2 | 41.7 | 42.4 |
| Skywork-PRM-7B | 67.4 | 54.6 | 53.6 | 54.1 | 71.6 | 76.2 | 77.9 | 74.4 | 75.0 | 64.0 | 65.3 | 64.6 |
| Llemma-PRM800k-7B | 52.7 | 49.1 | 52.4 | 50.7 | 52.3 | 49.4 | 53.9 | 55.1 | 52.7 | 51.1 | 52.6 | 51.9 |
| Llemma-MetaMath-7B | 41.5 | 43.7 | 39.1 | 41.4 | 39.3 | 44.8 | 44.3 | 42.0 | 42.6 | 38.5 | 38.2 | 38.4 |
| Llemma-oprm-7B | 50.7 | 44.4 | 37.3 | 40.9 | 54.5 | 50.3 | 53.0 | 56.8 | 53.6 | 51.0 | 53.6 | 52.3 |
| MATHMinos-Mistral-7B | 47.1 | 43.0 | 37.9 | 40.4 | 48.0 | 51.6 | 47.1 | 53.6 | 50.1 | 45.1 | 45.3 | 45.2 |
| MathShepherd-Mistral-7B | 53.1 | 47.2 | 43.1 | 45.1 | 52.3 | 62.1 | 63.5 | 57.5 | 58.9 | 50.0 | 46.4 | 48.2 |
| ReasonEval-7B | 67.5 | 74.5 | 62.9 | 68.7 | 63.5 | 74.5 | 71.8 | 71.8 | 70.4 | 59.9 | 59.3 | 59.6 |
| ReasonEval-34B | 60.7 | 68.2 | 61.9 | 65.0 | 61.0 | 53.8 | 52.0 | 59.9 | 56.7 | 57.8 | 62.6 | 60.2 |
| RLHFlow-PRM-Mistral-8B | 45.5 | 43.0 | 37.4 | 40.2 | 44.5 | 53.1 | 49.0 | 50.3 | 49.2 | 42.6 | 42.2 | 42.4 |
| RLHFlow-PRM-Deepseek-8B | 47.0 | 42.9 | 37.4 | 40.1 | 44.7 | 59.0 | 57.7 | 52.6 | 53.5 | 41.6 | 41.6 | 41.6 |
| Qwen2.5-Math-PRM-7B | 65.8 | 42.9 | 49.0 | 46.0 | 69.6 | 75.1 | 72.5 | 75.7 | 73.2 | 63.9 | 65.4 | 64.7 |
| Qwen2.5-Math-PRM-72B | 73.7 | 42.8 | 50.6 | 46.7 | 82.3 | 79.6 | 77.6 | 82.9 | 80.6 | 75.5 | 76.6 | 76.1 |
| Avg. | 55.4 | 49.5 | 46.5 | 48.0 | 56.1 | 60.3 | 59.7 | 60.2 | 59.0 | 52.6 | 53.1 | 52.9 |
| Open LLMs, Prompted as Critic Models | ||||||||||||
| MetaMath-7B | 45.1 | 45.3 | 35.8 | 40.6 | 45.8 | 48.8 | 48.8 | 47.1 | 47.6 | 43.6 | 43.7 | 43.6 |
| MetaMath-13B | 43.6 | 44.7 | 35.8 | 40.2 | 42.2 | 46.3 | 47.1 | 45.0 | 45.2 | 42.4 | 41.1 | 41.7 |
| Qwen2.5-Math-72B | 42.3 | 49.5 | 35.6 | 42.5 | 38.0 | 56.1 | 45.7 | 42.8 | 45.6 | 37.0 | 35.6 | 36.3 |
| QwQ-Preview-32B | 54.5 | 54.8 | 43.3 | 49.0 | 53.6 | 63.9 | 64.4 | 58.5 | 60.1 | 51.8 | 48.3 | 50.1 |
| R1-Distill-Llama3.1-70B | 53.8 | 49.8 | 41.7 | 45.7 | 53.4 | 63.6 | 71.7 | 57.8 | 61.6 | 46.7 | 44.5 | 45.6 |
| R1-Distill-Qwen-7B | 50.4 | 39.9 | 41.8 | 40.9 | 46.4 | 48.7 | 50.9 | 47.4 | 48.3 | 45.6 | 43.3 | 44.5 |
| Avg. | 48.3 | 47.3 | 39.0 | 43.2 | 46.6 | 54.6 | 54.8 | 49.8 | 51.4 | 44.5 | 42.7 | 43.6 |
| Model | Overall | Simplicity | Soundness | Sensitivity | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NR. | NCL. | Avg. | ES | SC. | DC. | CI | Avg. | PS | DR. | Avg. | ||
| Open-source Process Level Reward Models | ||||||||||||
| Skywork-PRM-1.5B | 48.1 | 45.9 | 43.9 | 44.9 | 46.3 | 52.5 | 54.0 | 51.3 | 51.0 | 45.3 | 43.3 | 44.3 |
| Skywork-PRM-7B | 64.3 | 51.2 | 51.1 | 51.1 | 65.2 | 70.4 | 71.6 | 73.3 | 70.1 | 64.2 | 64.6 | 64.4 |
| Llemma-PRM800k-7B | 52.6 | 47.1 | 53.2 | 50.2 | 56.5 | 48.8 | 47.9 | 55.7 | 52.2 | 52.6 | 55.2 | 53.9 |
| Llemma-MetaMath-7B | 45.2 | 45.3 | 41.5 | 43.4 | 44.9 | 48.2 | 46.3 | 47.6 | 46.7 | 42.3 | 42.8 | 42.5 |
| Llemma-oprm-7B | 48.3 | 44.7 | 38.1 | 41.4 | 53.5 | 46.6 | 41.6 | 56.2 | 49.5 | 49.0 | 52.4 | 50.7 |
| MATHMinos-Mistral-7B | 49.6 | 44.0 | 38.8 | 41.4 | 50.8 | 51.9 | 46.3 | 58.2 | 51.8 | 49.7 | 50.4 | 50.0 |
| MathShepherd-Mistral-7B | 52.5 | 47.8 | 43.4 | 45.6 | 55.7 | 56.3 | 54.3 | 58.4 | 56.2 | 50.2 | 49.5 | 49.8 |
| ReasonEval-7B | 62.9 | 69.4 | 60.1 | 64.7 | 59.2 | 66.3 | 64.1 | 70.3 | 65.0 | 57.4 | 55.5 | 56.4 |
| ReasonEval-34B | 63.2 | 63.5 | 55.8 | 59.6 | 65.6 | 58.6 | 57.8 | 66.9 | 62.2 | 61.9 | 65.6 | 63.7 |
| RLHFlow-PRM-Mistral-8B | 45.6 | 43.3 | 38.5 | 40.9 | 44.2 | 50.9 | 47.7 | 51.2 | 48.5 | 44.1 | 42.3 | 43.2 |
| RLHFlow-PRM-Deepseek-8B | 47.3 | 43.5 | 38.3 | 40.9 | 46.9 | 51.8 | 50.5 | 56.3 | 51.4 | 45.4 | 43.1 | 44.2 |
| Qwen2.5-Math-PRM-7B | 66.3 | 43.5 | 49.9 | 46.7 | 70.9 | 66.8 | 68.8 | 80.0 | 71.6 | 67.0 | 68.1 | 67.5 |
| Qwen2.5-Math-PRM-72B | 71.4 | 44.8 | 53.2 | 49.0 | 77.3 | 71.6 | 74.6 | 81.0 | 76.1 | 73.1 | 77.5 | 75.3 |
| Avg. | 55.2 | 48.8 | 46.6 | 47.7 | 56.7 | 57.0 | 55.8 | 62.0 | 57.9 | 54.0 | 54.6 | 54.3 |
| Open LLMs, Prompted as Critic Models | ||||||||||||
| MetaMath-7B | 45.6 | 44.2 | 37.5 | 40.9 | 46.0 | 48.5 | 48.0 | 47.3 | 47.4 | 45.6 | 45.5 | 45.5 |
| MetaMath-13B | 44.3 | 42.9 | 37.0 | 40.0 | 44.7 | 48.6 | 47.0 | 45.3 | 46.4 | 45.2 | 42.2 | 43.7 |
| Qwen2.5-Math-72B | 43.2 | 50.5 | 34.8 | 42.6 | 37.6 | 61.7 | 43.8 | 43.3 | 46.6 | 37.5 | 36.6 | 37.0 |
| R1-Distill-Llama3.1-70B | 50.6 | 45.9 | 41.0 | 43.5 | 48.2 | 65.7 | 58.1 | 57.8 | 57.5 | 46.7 | 42.8 | 44.8 |
| R1-Distill-Qwen-7B | 51.8 | 35.6 | 41.0 | 38.3 | 47.5 | 51.0 | 50.1 | 49.2 | 49.5 | 47.1 | 46.2 | 46.7 |
| Avg. | 47.1 | 43.8 | 38.2 | 41.0 | 44.8 | 55.1 | 49.4 | 48.6 | 49.5 | 44.4 | 42.7 | 43.5 |
| Model | Overall | Simplicity | Soundness | Sensitivity | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NR. | NCL. | Avg. | ES | SC. | DC. | CI | Avg. | PS | DR. | Avg. | ||
| Open-source Process Level Reward Models | ||||||||||||
| Skywork-PRM-1.5B | 50.3 | 44.7 | 41.8 | 43.3 | 51.8 | 57.7 | 55.2 | 56.1 | 55.2 | 46.8 | 46.8 | 46.8 |
| Skywork-PRM-7B | 65.2 | 51.1 | 51.9 | 51.5 | 68.3 | 70.3 | 72.7 | 76.4 | 71.9 | 64.8 | 65.1 | 65.0 |
| Llemma-PRM800k-7B | 53.8 | 49.1 | 52.0 | 50.6 | 57.3 | 49.6 | 49.4 | 57.3 | 53.4 | 53.8 | 58.6 | 56.2 |
| Llemma-MetaMath-7B | 46.9 | 46.4 | 42.0 | 44.2 | 46.5 | 47.2 | 44.2 | 51.7 | 47.4 | 47.8 | 45.4 | 46.6 |
| Llemma-oprm-7B | 47.1 | 42.8 | 39.7 | 41.2 | 52.2 | 42.7 | 40.9 | 54.3 | 47.5 | 48.1 | 51.6 | 49.8 |
| MATHMinos-Mistral-7B | 46.5 | 43.6 | 38.9 | 41.2 | 46.9 | 47.8 | 44.2 | 57.8 | 49.2 | 45.3 | 43.3 | 44.3 |
| MathShepherd-Mistral-7B | 53.5 | 48.0 | 44.8 | 46.4 | 55.9 | 56.3 | 54.8 | 62.8 | 57.4 | 48.6 | 53.5 | 51.1 |
| ReasonEval-7B | 62.5 | 71.5 | 57.2 | 64.4 | 60.1 | 66.9 | 62.2 | 70.6 | 65.0 | 56.9 | 53.9 | 55.4 |
| ReasonEval-34B | 64.2 | 68.5 | 52.0 | 60.2 | 68.5 | 59.3 | 56.4 | 69.8 | 63.5 | 63.7 | 66.6 | 65.2 |
| RLHFlow-PRM-Mistral-8B | 46.5 | 42.7 | 37.5 | 40.1 | 46.0 | 51.0 | 50.5 | 55.7 | 50.8 | 44.2 | 42.2 | 43.2 |
| Qwen2.5-Math-PRM-7B | 66.9 | 43.7 | 49.3 | 46.5 | 73.6 | 66.4 | 70.3 | 80.0 | 72.6 | 69.6 | 68.8 | 69.2 |
| Qwen2.5-Math-PRM-72B | 71.1 | 44.2 | 56.2 | 50.2 | 78.9 | 71.1 | 71.1 | 80.3 | 75.4 | 72.9 | 77.6 | 75.2 |
| Avg. | 55.6 | 45.9 | 46.3 | 46.1 | 58.1 | 56.8 | 55.7 | 64.2 | 58.7 | 54.5 | 55.2 | 54.8 |
| Open LLMs, Prompted as Critic Models | ||||||||||||
| MetaMath-7B | 45.6 | 43.5 | 36.9 | 40.2 | 47.9 | 48.9 | 49.1 | 48.1 | 48.5 | 47.3 | 43.6 | 45.4 |
| MetaMath-13B | 43.8 | 42.7 | 36.6 | 39.7 | 44.5 | 46.7 | 48.0 | 46.3 | 46.4 | 41.5 | 41.9 | 41.7 |
| Qwen2.5-Math-72B | 44.8 | 50.6 | 37.8 | 44.2 | 39.8 | 64.5 | 45.5 | 47.5 | 49.3 | 37.9 | 36.2 | 37.1 |
| R1-Distill-Llama3.1-70B | 53.3 | 44.5 | 40.4 | 42.5 | 53.0 | 63.8 | 59.4 | 64.6 | 60.2 | 49.6 | 48.3 | 48.9 |
| R1-Distill-Qwen-7B | 51.0 | 29.7 | 34.8 | 32.3 | 44.1 | 48.1 | 45.7 | 48.9 | 46.7 | 44.0 | 40.7 | 42.3 |
| Avg. | 47.7 | 42.2 | 37.3 | 39.8 | 45.9 | 54.4 | 49.5 | 51.1 | 50.2 | 44.0 | 42.2 | 43.1 |
Appendix C Details for Further Analysis
C.1 Error Analysis
A representative test case and the corresponding model performances are presented in Table 7. This example involves a counterfactual reasoning process, where steps eight through thirteen contain information that contradicts the correct computational principles and should be classified as “negative”. However, most models fail to identify these erroneous reasoning steps and assign relatively positive rewards, except for GPT-4o. While GPT-4o provides a relatively accurate reward, its judgments for key steps are only marginally negative, reflecting low confidence. This highlights a significant room for improvement in PRMs’ detailed error-detection capabilities.
| Method | GSM8K | MATH | OlymBen | MMLU | Avg. | PRMScore |
|---|---|---|---|---|---|---|
| Pass@8 | 98.9 | 96.2 | 79.8 | 96.7 | 92.9 | - |
| Maj@8 | 95.4 | 71.8 | 40.3 | 85.8 | 73.3 | - |
| Skywork-PRM-1.5B | 96.7 | 89.2 | 58.8 | 90.2 | 83.7 | 61.1 |
| Skywork-PRM-7B | 97.1 | 90.0 | 60.1 | 90.3 | 84.4 | 65.1 |
| Llemma-PRM800k-7B | 96.0 | 87.4 | 58.3 | 90.0 | 82.9 | 52.0 |
| Llemma-MetaMath-7B | 96.0 | 88.2 | 58.6 | 90.0 | 83.2 | 50.5 |
| Llemma-oprm-7B | 96.4 | 86.6 | 58.0 | 89.9 | 82.7 | 50.3 |
| MATHMinos-Mistral-7B | 95.8 | 88.3 | 59.1 | 89.1 | 83.1 | 54.2 |
| MathShepherd-Mistral-7B | 96.6 | 88.6 | 60.0 | 90.0 | 83.8 | 47.0 |
| ReasonEval-7B | 96.4 | 87.0 | 58.4 | 90.0 | 82.9 | 60.1 |
| RLHFlow-PRM-Mistral-8B | 96.5 | 88.4 | 59.1 | 90.4 | 83.6 | 54.4 |
| RLHFlow-PRM-Deepseek-8B | 96.4 | 87.6 | 58.5 | 90.2 | 83.2 | 54.2 |
| ReasonEval-34B | 96.4 | 86.9 | 56.9 | 90.0 | 82.6 | 60.5 |
| Qwen2.5-Math-PRM-7B | 96.7 | 88.0 | 58.7 | 89.8 | 83.3 | 65.5 |
| Qwen2.5-Math-PRM-72B | 96.7 | 89.3 | 60.5 | 90.0 | 84.2 | 68.2 |
| Standard Deviation () | 0.35 | 0.97 | 0.93 | 0.29 | 0.53 | 6.41 |
| Somers’ D correlation | 0.50 | 0.26 | 0.22 | 0.19 | 0.28 | 1.00 |
| Method | GSM8K | MATH | OlymBen | MMLU | Avg. | PRMScore |
|---|---|---|---|---|---|---|
| Pass@8 | 99.0 | 96.0 | 77.0 | 94.0 | 91.5 | - |
| Maj@8 | 96.5 | 68.0 | 41.0 | 86.0 | 72.9 | - |
| QwQ-Preview-32B | 96.5 | 83.5 | 56.5 | 87.5 | 81.0 | 63.6 |
| MetaMath-7B | 95.5 | 82.0 | 58.0 | 88.0 | 80.9 | 49.7 |
| MetaMath-13B | 95.0 | 84.5 | 57.5 | 86.5 | 80.9 | 49.4 |
| MetaMath-70B | 96.5 | 85.0 | 58.5 | 86.0 | 81.5 | 45.9 |
| Qwen2.5-Math-7B | 96.0 | 85.5 | 59.0 | 86.5 | 81.8 | 49.2 |
| Qwen2.5-Math-72B | 97.5 | 82.0 | 53.5 | 88.5 | 80.4 | 57.4 |
| R1-Distill-Llama3.1-8B | 96.5 | 84.5 | 53.5 | 87.5 | 80.5 | 52.7 |
| R1-Distill-Llama3.1-70B | 95.0 | 83.0 | 59.5 | 86.5 | 81.0 | 57.5 |
| R1-Distill-Qwen-7B | 97.0 | 86.5 | 59.0 | 86.5 | 82.2 | 52.6 |
| R1-Distill-Qwen-32B | 97.0 | 82.0 | 60.5 | 84.5 | 81.0 | 60.2 |
| WizardMath-7B | 96.5 | 82.5 | 60.0 | 87.0 | 81.5 | 49.2 |
| Gemini-2.0-flash-exp | 97.0 | 81.5 | 56.5 | 86.5 | 80.4 | 66.0 |
| Gemini-2.0-thinking-exp-1219 | 98.0 | 87.5 | 60.5 | 89.5 | 83.9 | 68.8 |
| Standard Deviation () | 0.87 | 1.83 | 2.25 | 1.19 | 0.91 | 7.02 |
| Somers’ D correlation | 0.36 | -0.21 | 0.03 | 0.24 | -0.15 | 1.00 |
C.2 Details for BoN Evaluation
Following Zhang et al. (2025), we sample eight responses (i.e., N=8) from Qwen-QwQ-Preview 32B Team (2024a). During evaluation, the PRMs are tasked with assigning a validity score to each step within every candidate response. The overall score for each candidate response is calculated by multiplying the individual step scores, as outlined in Lightman et al. (2023). We then select the highest-ranked candidate response, compare it with the correct answer, and calculate the accuracy, which we refer to as prm@8. Additionally, we report the result of majority voting among the eight sampled responses (maj@8) as the baseline, and we define pass@8 as the proportion of test samples where any of the eight samplings lead to the correct final answer, which serves as the upper bound.
We conduct BoN evaluation across all models on GSM8K Cobbe et al. (2021), MATH Hendrycks et al. (2021), Olympiad Bench He et al. (2024a) and MMLU Hendrycks et al. (2020). Due to the limit of the space, We omit some results in Table 6. Thus we provide the full results of all PRMs in Table 13. Moreover, due to cost constraints, we select a subset of 200 samples for each benchmark and obtain a total of 800 samples to evaluate generative LLMs of their BoN performance. The results are shown in Table 14.
Appendix D Instructions for Human Annotators
D.1 Backgrounds
With the emergence of multi-step reasoning enhanced language models such as OpenAI o1, and Deepmind Gemini-thinking, these models demonstrate the ability to decompose complex problems and solve them step by step. However, while their solutions often appear correct, they may contain errors in understanding, calculation, or reasoning logic, which is also known as false positive situations. A popular way to evaluate the results generated by these models is by utilizing process-level reward models (PRMs). Nevertheless, PRMs are fallible and not always correct. Existing benchmarks are not adequate for evaluating PRMs on different error types. Therefore, we are building a comprehensive evaluation benchmark for PRMs that can have a fine-grained detection of PRMs.
D.2 Task Definition
We begin by collecting completely correct multi-solution data and leveraging state-of-the-art LLMs to introduce various types of errors into these correct solutions, thereby generating our test cases. The detailed error types are described in Section 3. All synthesized data instances undergo an initial filtering process based on specific features.
Your task is to identify whether the modification taken is reasonable and whether the modified data instance is different from the original data instance.
Sub-task 1
The first sub-task is a binary classification task whose options include yes and no. Your task is to decide whether the modified step-by-step solution generated by LLMs is reasonable. The word “reasonable” has two aspects for evaluation.
- •
The modified process generated by LLMs seems like a possible solution path that could happen.
- •
The modified process generated by LLMs is exactly wrong and the type of error is suitable for the current “classification”.
Please assign a “yes” for this sub-task if both of the answers to the above two questions are “yes”. Otherwise, assign a “no” for this sub-task.
D.2.1 Sub-task 2
The second sub-task is a binary classification task whose options include yes and no. Your task is to decide whether the modified step-by-step solution generated by LLMs is different from the original solution process. The word “different” means the modified solution process is logically different from the original one, or there exist different statements compared to the original process.
Please assign a “yes” to this sub-task if your answer to the above question is “yes”. Otherwise, assign a “no” for this sub-task.
D.2.2 Error Types
Redundancy
refers to a process that is not the most concise or efficient, as it includes one or more redundant steps that can be removed without affecting the correctness of the overall solution path. For example, if represents a correct inference chain, your task is to introduce one or more redundant steps and reformulate the solution chain as .
Circular logic
is a specific form of redundancy, characterized by a reasoning chain that starts at a step , progresses through a sequence of steps, and ultimately loops back to . Symbolically, this can be expressed as , where , , and represent individual reasoning steps. Your task is to modify the reasoning process to introduce such circular logic.
Counterfactual
A counterfactual step refers to a statement within a reasoning chain that contradicts ground truth or established theories. Such contradictions can arise from relying on outdated theories, omitting critical constraints in a theory, or incorporating erroneous assumptions. Your task is to modify the reasoning process to introduce such counterfactual steps.
Step contradiction
refers to a conflict between a specific step and other steps within a reasoning path. Given a reasoning path , a step contradiction exists if , where and . Your task is to modify the reasoning process to introduce such step contradiction steps.
Domain inconsistency
is a special type of counterfactual. It refers to a step or a few steps within the reasoning chain that uses a statement or theory valid in other domains or cases but is not valid within the current reasoning chain. Your task is to modify the reasoning process to introduce steps for such domain inconsistency.
Confident hallucination
is a special type of counterfactual. It refers to a statement within the reasoning chain that contradicts established ground truth and is presented with an overly confident tone. In other words, it involves stating an incorrect statement with unwarranted certainty. Your task is to modify the reasoning process to introduce such confident hallucination steps.
Missing condition or prerequisite
refers to a flaw in the reasoning chain where critical premises, assumptions, or necessary conditions are absent. This omission results in logical gaps, incomplete reasoning, or biased conclusions. For example, when a missing condition occurs, the model must solve the problem through case analysis or further investigation. However, the answer becomes incorrect if the model overlooks the missing condition and proceeds with standard reasoning methods. Your task is to modify the reasoning process to introduce such missing condition errors.
Deception or traps
refer to statements that appear to be correct or align with ground truth but are subtly altered to introduce inaccuracies while maintaining the illusion of correctness. Your task is to modify the reasoning process to introduce such deception or trap error steps.
Appendix E Prompts
E.1 Prompts For Generating Data
As introduced in Section 3.2, we query GPT-4o OpenAI (2024a) to synthesize the metadata at the very first step of our test case construction procedure. To better prompt LLMs to generate high-quality data instances, we carefully designed our prompts, which are displayed in Figure 19-22. We display only one example here due to limitations in space. And the prompts can be found in our supplementary materials.
E.2 Prompts For Evaluating Generative LLMs
As introduced in Section 4.1, we prompt some state-of-the-art generative LLMs as critic models to evaluate their rewarding capabilities on PRMBench. To make a fair comparison between different models, we carefully design the prompts and utilize a unified prompt to query them. The prompt used is displayed in Figure 23 and 24.
Appendix F Further Discussion
Inspired by the results and discoveries on PRMBench, we further propose several promising directions for future research, which we hope can offer valuable insights and contribute to the advancement of the research community.
Anti-redundancy training:
As stated in Section 4.3, our work highlights a specific weakness of current PRMs in identifying redundant reasoning steps. To mitigate this, one possible approach is to modify the label distribution during training. PRM training data is typically labeled as Correct, Neutral, or Incorrect, where the Neutral label often corresponds to redundant steps. By reducing the proportion of Neutral samples in the training data, we can train PRMs with stronger anti-redundancy capabilities.
Contrastive training:
A high-quality data curation pipeline is introduced in Section 3.1, which can also be adapted to curate training samples labeled with fine-grained error types. By leveraging contrastive learning or preference alignment with the curated data, the error sensitivity and detection capabilities of PRMs can be further improved.
Step-level evaluation for LLMs:
As introduced in Section 5.4, the inconsistency between PRMBench and BoN evaluation reveals the false positive situation and the risk of reward hacking within LM post-training. Therefore, traditional outcome-level label-based evaluation is not enough, highlighting the need for a comprehensive step-level evaluation of LLM’s reasoning procedure.
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 61.1 | 88.6 | 33.6 | 80.6 | 88.8 | 33.2 | 44.7 | 91.7 |
| Skywork-PRM-7B | 65.1 | 89.2 | 40.9 | 81.7 | 88.5 | 42.7 | 56.6 | 90.1 |
| Llemma-PRM800k-7B | 52.0 | 75.7 | 28.3 | 63.7 | 66.4 | 48.5 | 22.2 | 79.5 |
| Llemma-MetaMath-7B | 50.5 | 80.4 | 20.7 | 68.5 | 75.6 | 27.7 | 15.1 | 84.2 |
| Llemma-oprm-7B | 50.3 | 77.3 | 23.3 | 64.9 | 69.9 | 36.1 | 16.6 | 83.7 |
| MATHMinos-Mistral-7B | 54.2 | 79.2 | 29.1 | 67.9 | 72.8 | 41.7 | 38.0 | 84.6 |
| MathShepherd-Mistral-7B | 47.0 | 64.9 | 29.2 | 53.0 | 51.5 | 61.1 | 54.6 | 83.5 |
| ReasonEval-7B | 60.1 | 90.8 | 29.3 | 83.8 | 95.5 | 21.2 | 30.3 | 91.6 |
| ReasonEval-34B | 60.5 | 83.8 | 37.2 | 74.2 | 79.1 | 48.4 | 50.8 | 82.8 |
| RLHFlow-PRM-Mistral-8B | 54.4 | 87.7 | 21.1 | 78.8 | 90.2 | 17.9 | 22.1 | 92.8 |
| RLHFlow-PRM-Deepseek-8B | 54.2 | 89.9 | 18.6 | 82.0 | 95.0 | 13.0 | 17.0 | 95.0 |
| Qwen2.5-Math-PRM-7B | 65.5 | 91.5 | 39.4 | 85.1 | 95.4 | 30.6 | 37.8 | 89.0 |
| Qwen2.5-Math-PRM-72B | 68.2 | 91.4 | 45.1 | 85.1 | 93.8 | 38.7 | 48.5 | 86.8 |
| Pure-PRM-7B | 65.3 | 90.1 | 40.5 | 83.0 | 91.8 | 36.6 | 41.8 | 86.6 |
| Avg. | 57.7 | 84.3 | 31.2 | 75.2 | 82.4 | 35.5 | 35.4 | 87.3 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 49.7 | 88.2 | 11.2 | 79.1 | 90.9 | 9.1 | 7.6 | 91.6 |
| MetaMath-13B | 49.4 | 89.6 | 9.1 | 81.4 | 94.4 | 6.3 | 4.8 | 94.8 |
| Qwen2.5-Math-72B | 57.4 | 90.3 | 24.4 | 82.9 | 96.9 | 15.9 | 19.8 | 92.4 |
| QwQ-Preview-32B | 63.6 | 87.6 | 39.6 | 79.4 | 89.2 | 36.4 | 40.2 | 83.1 |
| R1-Distill-Llama3.1-70B | 57.5 | 91.4 | 23.5 | 84.6 | 97.4 | 15.2 | 19.0 | 93.8 |
| R1-Distill-Qwen-7B | 52.6 | 83.2 | 22.0 | 72.3 | 80.5 | 26.0 | 23.9 | 79.3 |
| DeepSeek-R1† | 67.8 | 87.2 | 48.4 | 79.5 | 83.1 | 60.5 | 63.7 | 77.9 |
| Avg. | 56.8 | 88.2 | 25.4 | 79.9 | 90.3 | 24.2 | 25.6 | 87.5 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 66.8 | 86.9 | 46.7 | 79.0 | 82.9 | 58.2 | 64.4 | 76.6 |
| o1-mini† | 68.8 | 89.2 | 48.3 | 82.1 | 86.9 | 55.4 | 56.4 | 80.4 |
| Gemini-2.0-flash-exp | 66.0 | 86.5 | 45.5 | 78.4 | 82.3 | 57.2 | 64.3 | 80.0 |
| Gemini-2.0-thinking-exp-1219 | 68.8 | 89.7 | 47.8 | 82.8 | 89.0 | 49.8 | 57.0 | 82.0 |
| Avg. | 67.6 | 88.1 | 47.1 | 80.6 | 85.3 | 55.2 | 60.5 | 79.7 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 52.0 | 88.4 | 15.6 | 79.6 | 88.8 | 15.1 | 18.7 | 95.0 |
| Skywork-PRM-7B | 56.4 | 89.0 | 23.9 | 80.7 | 88.8 | 24.2 | 31.3 | 94.5 |
| Llemma-PRM800k-7B | 49.3 | 77.7 | 20.9 | 65.2 | 69.2 | 36.9 | 20.5 | 86.4 |
| Llemma-MetaMath-7B | 50.2 | 85.3 | 15.0 | 75.0 | 83.2 | 17.6 | 15.3 | 94.4 |
| Llemma-oprm-7B | 48.7 | 80.3 | 17.2 | 68.1 | 74.1 | 26.5 | 16.0 | 92.0 |
| MATHMinos-Mistral-7B | 48.8 | 82.7 | 14.9 | 71.2 | 79.2 | 19.0 | 20.8 | 95.7 |
| MathShepherd-Mistral-7B | 44.0 | 67.7 | 20.2 | 54.0 | 55.6 | 43.7 | 39.7 | 93.8 |
| ReasonEval-7B | 61.0 | 91.5 | 30.5 | 84.9 | 94.0 | 25.1 | 32.8 | 89.8 |
| ReasonEval-34B | 54.8 | 86.7 | 22.9 | 77.4 | 85.3 | 25.3 | 31.6 | 83.8 |
| RLHFlow-PRM-Mistral-8B | 46.1 | 89.1 | 3.2 | 80.3 | 92.3 | 2.4 | 3.4 | 98.5 |
| RLHFlow-PRM-Deepseek-8B | 46.4 | 91.0 | 1.9 | 83.5 | 96.1 | 1.2 | 1.5 | 99.0 |
| Qwen2.5-Math-PRM-7B | 49.0 | 92.1 | 5.9 | 85.4 | 98.0 | 3.5 | 3.3 | 98.2 |
| Qwen2.5-Math-PRM-72B | 50.4 | 91.9 | 8.8 | 85.2 | 97.4 | 5.4 | 6.2 | 98.2 |
| Pure-PRM-7B | 49.2 | 90.8 | 7.6 | 83.3 | 95.2 | 5.2 | 5.4 | 96.2 |
| Avg. | 50.5 | 86.0 | 14.9 | 76.7 | 85.5 | 17.9 | 17.6 | 94.0 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 48.9 | 80.7 | 17.0 | 68.7 | 74.9 | 25.5 | 23.2 | 76.3 |
| MetaMath-13B | 50.3 | 86.5 | 14.0 | 76.7 | 85.7 | 14.9 | 11.8 | 85.4 |
| Qwen2.5-Math-72B | 55.3 | 90.3 | 20.3 | 82.7 | 93.9 | 15.4 | 17.0 | 90.1 |
| QwQ-Preview-32B | 57.2 | 85.8 | 28.7 | 76.3 | 84.3 | 31.4 | 33.9 | 81.9 |
| R1-Distill-Llama3.1-70B | 49.5 | 92.5 | 6.5 | 86.1 | 98.8 | 3.6 | 5.4 | 98.0 |
| R1-Distill-Qwen-7B | 32.9 | 43.9 | 21.9 | 34.7 | 29.4 | 70.1 | 68.6 | 31.5 |
| DeepSeek-R1† | 63.0 | 86.9 | 39.2 | 78.4 | 82.2 | 53.3 | 62.5 | 76.5 |
| Avg. | 51.0 | 80.9 | 21.1 | 71.9 | 78.5 | 30.6 | 31.8 | 77.1 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 57.0 | 77.8 | 36.3 | 67.0 | 66.5 | 70.4 | 77.1 | 68.9 |
| o1-mini† | 65.6 | 90.8 | 40.4 | 84.1 | 90.4 | 41.5 | 45.8 | 84.1 |
| Gemini-2.0-flash-exp | 67.2 | 91.5 | 42.9 | 85.1 | 91.8 | 41.7 | 49.7 | 82.4 |
| Gemini-2.0-thinking-exp-1219 | 68.5 | 91.4 | 45.6 | 85.1 | 90.9 | 47.1 | 56.5 | 84.7 |
| Avg. | 64.6 | 87.9 | 41.3 | 80.4 | 84.9 | 50.2 | 57.3 | 80.0 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 35.8 | 34.7 | 36.9 | 35.8 | 22.7 | 75.2 | 71.4 | 72.9 |
| Skywork-PRM-7B | 41.2 | 44.2 | 38.1 | 41.3 | 30.9 | 72.6 | 71.6 | 73.5 |
| Llemma-PRM800k-7B | 53.4 | 65.5 | 41.3 | 56.6 | 54.9 | 61.4 | 15.7 | 65.6 |
| Llemma-MetaMath-7B | 50.5 | 70.8 | 30.3 | 58.8 | 66.4 | 36.0 | 15.7 | 74.8 |
| Llemma-oprm-7B | 49.3 | 66.3 | 32.2 | 55.0 | 59.0 | 42.9 | 10.2 | 73.3 |
| MATHMinos-Mistral-7B | 54.0 | 70.8 | 37.2 | 60.2 | 66.6 | 43.2 | 40.6 | 79.1 |
| MathShepherd-Mistral-7B | 50.3 | 60.4 | 40.2 | 52.3 | 49.9 | 58.6 | 50.3 | 79.0 |
| ReasonEval-7B | 50.1 | 80.8 | 19.4 | 69.0 | 89.8 | 13.6 | 19.0 | 88.1 |
| ReasonEval-34B | 48.1 | 76.7 | 19.5 | 63.9 | 81.9 | 16.0 | 19.9 | 83.4 |
| RLHFlow-PRM-Mistral-8B | 47.3 | 79.8 | 14.9 | 67.3 | 88.7 | 10.5 | 18.9 | 93.7 |
| RLHFlow-PRM-Deepseek-8B | 48.9 | 82.3 | 15.4 | 70.7 | 93.6 | 9.8 | 15.6 | 94.6 |
| Qwen2.5-Math-PRM-7B | 55.1 | 84.5 | 25.7 | 74.3 | 96.1 | 16.3 | 22.7 | 91.6 |
| Qwen2.5-Math-PRM-72B | 58.8 | 84.5 | 33.1 | 74.8 | 94.3 | 22.8 | 33.5 | 89.4 |
| Pure-PRM-7B | 55.2 | 82.6 | 27.8 | 72.0 | 91.6 | 19.7 | 24.3 | 88.0 |
| Avg. | 49.9 | 70.3 | 29.4 | 60.9 | 70.5 | 35.6 | 30.7 | 81.9 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 46.9 | 74.9 | 19.0 | 61.7 | 75.2 | 18.7 | 13.6 | 77.8 |
| MetaMath-13B | 44.4 | 76.4 | 12.4 | 62.8 | 81.0 | 10.2 | 7.3 | 83.8 |
| Qwen2.5-Math-72B | 54.9 | 81.2 | 28.6 | 70.2 | 89.5 | 21.1 | 27.7 | 81.5 |
| QwQ-Preview-32B | 55.6 | 76.1 | 35.2 | 65.0 | 77.9 | 33.1 | 37.2 | 75.8 |
| R1-Distill-Llama3.1-70B | 48.1 | 84.4 | 11.8 | 73.5 | 98.1 | 6.6 | 8.5 | 95.6 |
| R1-Distill-Qwen-7B | 37.9 | 38.2 | 37.7 | 37.9 | 25.8 | 73.4 | 73.3 | 27.2 |
| DeepSeek-R1† | 62.7 | 76.4 | 49.0 | 67.7 | 71.6 | 57.3 | 61.7 | - |
| Avg. | 50.1 | 72.5 | 27.7 | 62.7 | 74.2 | 31.5 | 32.8 | 73.6 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 62.4 | 73.5 | 51.3 | 65.6 | 65.4 | 66.2 | 80.6 | 59.9 |
| o1-mini† | 63.7 | 80.4 | 47.0 | 71.4 | 80.6 | 46.6 | 47.9 | - |
| Gemini-2.0-flash-exp | 58.1 | 81.8 | 34.5 | 71.5 | 88.1 | 27.4 | 34.7 | 79.4 |
| Gemini-2.0-thinking-exp-1219 | 63.8 | 81.2 | 46.4 | 72.2 | 82.8 | 44.0 | 54.8 | 74.6 |
| Avg. | 62.0 | 79.2 | 44.8 | 70.2 | 79.2 | 46.1 | 54.5 | 71.3 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 32.4 | 32.9 | 32.0 | 32.4 | 20.3 | 85.4 | 76.4 | 72.2 |
| Skywork-PRM-7B | 36.7 | 39.1 | 34.4 | 36.8 | 24.9 | 88.9 | 82.5 | 67.6 |
| Llemma-PRM800k-7B | 56.4 | 76.8 | 36.1 | 65.9 | 69.2 | 51.7 | 20.0 | 81.2 |
| Llemma-MetaMath-7B | 51.9 | 78.0 | 25.8 | 66.0 | 73.9 | 31.7 | 15.4 | 83.3 |
| Llemma-oprm-7B | 54.2 | 77.3 | 31.1 | 65.9 | 71.4 | 41.6 | 15.6 | 83.5 |
| MATHMinos-Mistral-7B | 57.0 | 77.3 | 36.7 | 66.6 | 71.1 | 48.6 | 39.7 | 82.0 |
| MathShepherd-Mistral-7B | 49.4 | 62.4 | 36.4 | 52.7 | 48.9 | 68.0 | 58.0 | 81.6 |
| ReasonEval-7B | 62.1 | 89.7 | 34.6 | 82.2 | 96.6 | 23.7 | 37.4 | 91.2 |
| ReasonEval-34B | 66.4 | 83.3 | 49.4 | 74.9 | 78.1 | 61.9 | 61.1 | 81.5 |
| RLHFlow-PRM-Mistral-8B | 56.6 | 85.8 | 27.4 | 76.2 | 89.6 | 22.5 | 24.5 | 90.5 |
| RLHFlow-PRM-Deepseek-8B | 55.7 | 87.8 | 23.5 | 79.0 | 94.6 | 16.2 | 21.4 | 93.5 |
| Qwen2.5-Math-PRM-7B | 71.8 | 90.8 | 52.8 | 84.6 | 94.8 | 43.3 | 56.2 | 83.7 |
| Qwen2.5-Math-PRM-72B | 73.7 | 90.7 | 56.8 | 84.7 | 93.2 | 50.6 | 66.9 | 81.4 |
| Pure-PRM-7B | 71.1 | 89.5 | 52.6 | 82.9 | 91.5 | 48.0 | 55.1 | 82.6 |
| Avg. | 56.8 | 75.8 | 37.8 | 67.9 | 72.7 | 48.7 | 45.0 | 82.6 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 47.3 | 89.8 | 4.7 | 81.6 | 97.3 | 2.7 | 1.3 | 97.2 |
| MetaMath-13B | 47.8 | 90.0 | 5.6 | 81.9 | 98.4 | 3.1 | 3.1 | 98.6 |
| Qwen2.5-Math-72B | 55.5 | 88.6 | 22.4 | 80.2 | 99.1 | 13.1 | 16.2 | 94.7 |
| QwQ-Preview-32B | 67.4 | 87.6 | 47.2 | 79.9 | 92.8 | 38.1 | 42.5 | 83.5 |
| R1-Distill-Llama3.1-70B | 61.4 | 89.8 | 33.0 | 82.3 | 97.1 | 22.1 | 26.8 | 90.9 |
| R1-Distill-Qwen-7B | 47.3 | 90.2 | 4.4 | 82.3 | 99.1 | 2.4 | 3.3 | 98.2 |
| DeepSeek-R1† | 68.2 | 84.3 | 52.0 | 76.3 | 78.0 | 69.0 | 77.1 | - |
| Avg. | 56.4 | 88.6 | 24.2 | 80.6 | 94.5 | 21.5 | 24.3 | 93.9 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 72.0 | 88.9 | 55.2 | 82.2 | 88.8 | 55.4 | 63.2 | 78.7 |
| o1-mini† | 74.5 | 88.9 | 60.0 | 82.7 | 85.6 | 69.8 | 75.0 | - |
| Gemini-2.0-flash-exp | 70.4 | 85.3 | 55.4 | 77.9 | 80.0 | 69.4 | 76.3 | 77.1 |
| Gemini-2.0-thinking-exp-1219 | 72.9 | 89.4 | 56.4 | 83.0 | 89.8 | 55.5 | 64.8 | 79.5 |
| Avg. | 72.4 | 88.1 | 56.7 | 81.4 | 86.0 | 62.5 | 69.8 | 78.4 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 64.9 | 91.1 | 38.7 | 84.4 | 88.7 | 47.1 | 58.3 | 91.2 |
| Skywork-PRM-7B | 67.1 | 91.0 | 43.2 | 84.4 | 87.7 | 56.6 | 69.6 | 88.9 |
| Llemma-PRM800k-7B | 47.1 | 75.2 | 18.9 | 62.1 | 64.3 | 42.5 | 24.1 | 80.5 |
| Llemma-MetaMath-7B | 47.6 | 81.3 | 13.9 | 69.2 | 74.5 | 23.9 | 15.5 | 85.0 |
| Llemma-oprm-7B | 46.8 | 77.6 | 16.1 | 64.6 | 68.3 | 32.6 | 16.8 | 83.4 |
| MATHMinos-Mistral-7B | 52.1 | 80.7 | 23.5 | 69.2 | 72.6 | 42.4 | 39.4 | 85.5 |
| MathShepherd-Mistral-7B | 44.5 | 64.9 | 24.2 | 52.0 | 49.9 | 68.6 | 63.0 | 81.9 |
| ReasonEval-7B | 65.9 | 94.1 | 37.6 | 89.2 | 96.6 | 29.4 | 40.1 | 92.0 |
| ReasonEval-34B | 60.3 | 83.7 | 36.9 | 74.1 | 74.8 | 68.2 | 70.6 | 81.5 |
| RLHFlow-PRM-Mistral-8B | 55.1 | 90.0 | 20.2 | 82.2 | 90.0 | 20.1 | 22.9 | 92.3 |
| RLHFlow-PRM-Deepseek-8B | 55.0 | 92.4 | 17.7 | 86.1 | 95.2 | 13.4 | 16.6 | 95.1 |
| Qwen2.5-Math-PRM-7B | 67.3 | 93.4 | 41.2 | 88.1 | 94.4 | 37.6 | 43.8 | 87.9 |
| Qwen2.5-Math-PRM-72B | 71.1 | 92.9 | 49.2 | 87.5 | 91.6 | 54.5 | 61.6 | 83.1 |
| Pure-PRM-7B | 68.8 | 91.7 | 45.8 | 85.7 | 89.6 | 54.5 | 61.0 | 83.5 |
| Avg. | 58.1 | 85.7 | 30.5 | 77.1 | 81.3 | 42.2 | 43.1 | 86.6 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 48.9 | 92.6 | 5.1 | 86.3 | 94.7 | 4.0 | 4.1 | 96.3 |
| MetaMath-13B | 47.4 | 93.6 | 1.2 | 88.1 | 97.8 | 0.7 | 0.9 | 99.6 |
| Qwen2.5-Math-72B | 71.6 | 93.5 | 49.6 | 88.5 | 95.1 | 43.8 | 49.5 | 86.9 |
| QwQ-Preview-32B | 72.3 | 91.3 | 53.3 | 85.3 | 88.8 | 62.4 | 65.0 | 79.7 |
| R1-Distill-Llama3.1-70B | 65.5 | 93.4 | 37.6 | 88.1 | 95.3 | 31.6 | 37.2 | 89.6 |
| R1-Distill-Qwen-7B | 54.1 | 94.0 | 14.1 | 88.8 | 98.9 | 8.2 | 9.2 | 97.0 |
| DeepSeek-R1† | 68.5 | 90.5 | 46.5 | 83.8 | 86.0 | 65.7 | 63.2 | 75.0 |
| Avg. | 61.2 | 92.7 | 29.6 | 87.0 | 93.8 | 30.9 | 32.7 | 89.2 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 69.7 | 89.9 | 49.6 | 83.1 | 84.3 | 74.3 | 76.9 | 76.3 |
| o1-mini† | 67.7 | 89.7 | 45.7 | 82.7 | 84.4 | 68.6 | 70.8 | 74.2 |
| Gemini-2.0-flash-exp | 65.7 | 86.0 | 45.4 | 77.7 | 77.0 | 83.1 | 85.6 | 77.3 |
| Gemini-2.0-thinking-exp-1219 | 71.3 | 91.4 | 51.2 | 85.4 | 87.6 | 68.5 | 72.8 | 81.0 |
| Avg. | 68.6 | 89.3 | 48.0 | 82.3 | 83.3 | 73.6 | 76.5 | 77.2 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 63.3 | 89.6 | 37.0 | 82.2 | 87.9 | 42.1 | 49.9 | 90.5 |
| Skywork-PRM-7B | 67.7 | 90.2 | 45.2 | 83.3 | 87.3 | 55.4 | 65.4 | 88.3 |
| Llemma-PRM800k-7B | 46.7 | 71.6 | 21.9 | 58.3 | 59.9 | 47.1 | 20.5 | 74.3 |
| Llemma-MetaMath-7B | 44.4 | 76.0 | 12.9 | 62.3 | 68.0 | 22.5 | 12.9 | 79.1 |
| Llemma-oprm-7B | 44.5 | 73.4 | 15.6 | 59.6 | 63.8 | 30.2 | 13.6 | 79.4 |
| MATHMinos-Mistral-7B | 50.7 | 76.1 | 25.3 | 63.8 | 66.6 | 45.8 | 39.0 | 80.6 |
| MathShepherd-Mistral-7B | 41.3 | 59.2 | 23.3 | 46.7 | 44.6 | 60.5 | 52.2 | 79.8 |
| ReasonEval-7B | 61.5 | 92.1 | 31.0 | 85.8 | 95.4 | 23.8 | 30.5 | 91.2 |
| ReasonEval-34B | 57.8 | 80.3 | 35.3 | 69.7 | 71.0 | 61.9 | 57.9 | 77.5 |
| RLHFlow-PRM-Mistral-8B | 54.4 | 87.8 | 21.0 | 78.9 | 87.8 | 20.9 | 18.8 | 90.4 |
| RLHFlow-PRM-Deepseek-8B | 53.2 | 90.4 | 15.9 | 82.8 | 93.7 | 12.2 | 11.6 | 94.2 |
| Qwen2.5-Math-PRM-7B | 66.3 | 91.5 | 41.2 | 85.1 | 92.2 | 39.1 | 42.0 | 85.0 |
| Qwen2.5-Math-PRM-72B | 72.2 | 91.7 | 52.7 | 85.9 | 90.1 | 58.7 | 65.5 | 80.5 |
| Pure-PRM-7B | 64.0 | 89.8 | 38.3 | 82.5 | 89.0 | 40.7 | 42.5 | 84.2 |
| Avg. | 56.3 | 82.8 | 29.8 | 73.4 | 78.4 | 40.1 | 37.3 | 83.9 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 48.4 | 90.9 | 5.8 | 83.5 | 96.5 | 3.6 | 3.8 | 96.7 |
| MetaMath-13B | 49.4 | 92.1 | 6.7 | 85.4 | 98.1 | 3.9 | 3.8 | 98.8 |
| Qwen2.5-Math-72B | 58.1 | 91.8 | 24.3 | 85.3 | 97.0 | 16.4 | 15.0 | 92.6 |
| QwQ-Preview-32B | 66.2 | 87.0 | 45.5 | 78.9 | 82.9 | 57.0 | 51.6 | 74.4 |
| R1-Distill-Llama3.1-70B | 65.8 | 91.9 | 39.7 | 85.7 | 93.4 | 35.4 | 35.5 | 86.1 |
| R1-Distill-Qwen-7B | 48.4 | 92.1 | 4.7 | 85.4 | 98.4 | 2.7 | 2.8 | 98.4 |
| DeepSeek-R1† | 73.5 | 91.3 | 55.6 | 85.5 | 85.7 | 84.1 | 77.1 | 75.4 |
| Avg. | 58.5 | 91.0 | 26.0 | 84.2 | 93.1 | 29.0 | 27.1 | 88.9 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 70.7 | 88.2 | 53.3 | 81.2 | 81.3 | 80.4 | 81.4 | 71.4 |
| o1-mini† | 73.8 | 92.2 | 55.5 | 86.7 | 87.7 | 77.8 | 74.5 | 80.0 |
| Gemini-2.0-flash-exp | 66.0 | 83.6 | 48.4 | 75.1 | 73.2 | 87.5 | 87.3 | 70.8 |
| Gemini-2.0-thinking-exp-1219 | 71.0 | 88.8 | 53.2 | 81.9 | 82.6 | 77.2 | 79.2 | 73.7 |
| Avg. | 70.4 | 88.2 | 52.6 | 81.2 | 81.2 | 80.7 | 80.6 | 74.0 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 66.5 | 91.3 | 41.7 | 84.8 | 89.7 | 47.3 | 62.2 | 92.5 |
| Skywork-PRM-7B | 69.9 | 91.5 | 48.2 | 85.4 | 88.8 | 59.3 | 71.5 | 90.8 |
| Llemma-PRM800k-7B | 53.3 | 79.9 | 26.6 | 68.5 | 70.9 | 50.0 | 29.4 | 86.2 |
| Llemma-MetaMath-7B | 52.1 | 83.7 | 20.4 | 73.0 | 78.5 | 30.3 | 15.6 | 88.9 |
| Llemma-oprm-7B | 53.5 | 81.7 | 25.4 | 70.6 | 74.0 | 43.8 | 27.2 | 89.0 |
| MATHMinos-Mistral-7B | 57.8 | 82.5 | 33.1 | 72.3 | 74.5 | 56.3 | 53.8 | 87.3 |
| MathShepherd-Mistral-7B | 47.7 | 66.6 | 28.7 | 54.5 | 51.7 | 75.1 | 72.2 | 87.3 |
| ReasonEval-7B | 66.0 | 93.7 | 38.3 | 88.6 | 96.9 | 29.0 | 40.6 | 92.8 |
| ReasonEval-34B | 67.5 | 87.7 | 47.2 | 80.1 | 81.0 | 73.3 | 73.3 | 86.9 |
| RLHFlow-PRM-Mistral-8B | 63.8 | 91.0 | 36.7 | 84.2 | 90.7 | 37.5 | 45.0 | 90.9 |
| RLHFlow-PRM-Deepseek-8B | 66.2 | 93.0 | 39.5 | 87.5 | 94.9 | 33.5 | 43.3 | 92.1 |
| Qwen2.5-Math-PRM-7B | 78.5 | 94.9 | 62.0 | 91.0 | 95.2 | 60.5 | 70.4 | 85.4 |
| Qwen2.5-Math-PRM-72B | 78.6 | 94.7 | 62.5 | 90.7 | 94.3 | 64.1 | 73.7 | 84.8 |
| Pure-PRM-7B | 76.9 | 93.9 | 59.9 | 89.5 | 92.9 | 64.7 | 70.8 | 86.3 |
| Avg. | 64.2 | 87.6 | 40.7 | 80.0 | 83.9 | 51.8 | 53.5 | 88.7 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 48.8 | 93.9 | 3.7 | 88.5 | 97.1 | 2.4 | 2.1 | 97.7 |
| MetaMath-13B | 48.1 | 94.7 | 1.5 | 89.9 | 99.1 | 0.8 | 0.6 | 99.0 |
| Qwen2.5-Math-72B | 59.1 | 93.1 | 25.1 | 87.4 | 99.6 | 14.7 | 19.8 | 96.2 |
| QwQ-Preview-32B | 66.9 | 91.7 | 42.2 | 85.4 | 94.8 | 34.2 | 43.1 | 89.1 |
| R1-Distill-Llama3.1-70B | 61.1 | 93.9 | 28.2 | 88.8 | 98.3 | 18.5 | 22.0 | 94.8 |
| R1-Distill-Qwen-7B | 48.0 | 94.6 | 1.3 | 89.8 | 99.8 | 0.7 | 0.7 | 99.5 |
| DeepSeek-R1† | 75.4 | 92.7 | 58.1 | 87.6 | 88.8 | 78.1 | 82.4 | 82.7 |
| Avg. | 58.2 | 93.5 | 22.9 | 88.2 | 96.8 | 21.4 | 24.4 | 94.2 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 71.1 | 92.1 | 50.2 | 86.3 | 90.4 | 56.5 | 60.4 | 85.2 |
| o1-mini† | 72.3 | 91.6 | 53.1 | 85.7 | 87.3 | 72.7 | 75.0 | 80.1 |
| Gemini-2.0-flash-exp | 67.3 | 88.3 | 46.4 | 80.8 | 82.4 | 68.8 | 76.0 | 84.9 |
| Gemini-2.0-thinking-exp-1219 | 71.8 | 92.6 | 51.1 | 87.1 | 91.4 | 55.8 | 60.5 | 86.1 |
| Avg. | 70.7 | 91.1 | 50.2 | 85.0 | 87.9 | 63.5 | 68.0 | 84.1 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 33.1 | 36.2 | 30.1 | 33.3 | 23.2 | 78.4 | 73.9 | 70.2 |
| Skywork-PRM-7B | 36.8 | 42.9 | 30.8 | 37.4 | 28.8 | 76.0 | 72.8 | 70.2 |
| Llemma-PRM800k-7B | 51.0 | 71.7 | 30.3 | 59.7 | 62.3 | 47.9 | 24.4 | 75.0 |
| Llemma-MetaMath-7B | 50.5 | 78.7 | 22.4 | 66.5 | 75.5 | 26.5 | 15.6 | 82.8 |
| Llemma-oprm-7B | 49.2 | 74.4 | 24.1 | 61.7 | 68.0 | 33.4 | 16.1 | 81.2 |
| MATHMinos-Mistral-7B | 52.8 | 77.5 | 28.0 | 65.7 | 73.3 | 34.2 | 29.5 | 83.1 |
| MathShepherd-Mistral-7B | 47.2 | 63.6 | 30.9 | 52.3 | 51.8 | 54.5 | 45.0 | 81.4 |
| ReasonEval-7B | 55.6 | 88.4 | 22.9 | 79.8 | 95.4 | 15.4 | 17.3 | 92.8 |
| ReasonEval-34B | 57.7 | 80.5 | 35.0 | 70.0 | 76.8 | 41.6 | 35.2 | 82.6 |
| RLHFlow-PRM-Mistral-8B | 51.5 | 85.3 | 17.6 | 75.0 | 89.9 | 13.7 | 16.2 | 93.1 |
| RLHFlow-PRM-Deepseek-8B | 49.0 | 87.0 | 10.9 | 77.4 | 94.4 | 7.1 | 6.7 | 95.9 |
| Qwen2.5-Math-PRM-7B | 57.6 | 88.8 | 26.5 | 80.5 | 95.6 | 18.1 | 17.7 | 91.7 |
| Qwen2.5-Math-PRM-72B | 60.3 | 88.5 | 32.2 | 80.3 | 93.8 | 24.1 | 24.9 | 89.4 |
| Pure-PRM-7B | 60.3 | 87.4 | 33.1 | 78.8 | 91.4 | 27.0 | 24.0 | 87.3 |
| Avg. | 50.9 | 75.0 | 26.8 | 65.6 | 72.9 | 35.6 | 29.9 | 84.0 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 46.5 | 88.0 | 5.0 | 78.7 | 96.3 | 3.0 | 2.3 | 96.2 |
| MetaMath-13B | 49.0 | 88.6 | 9.4 | 79.7 | 97.6 | 5.4 | 3.1 | 96.7 |
| Qwen2.5-Math-72B | 47.4 | 88.5 | 6.3 | 79.5 | 99.9 | 3.3 | 3.1 | 98.8 |
| QwQ-Preview-32B | 57.8 | 86.4 | 29.1 | 77.2 | 92.8 | 21.5 | 18.3 | 88.3 |
| R1-Distill-Llama3.1-70B | 48.8 | 88.9 | 8.7 | 80.3 | 98.7 | 4.8 | 4.8 | 97.2 |
| R1-Distill-Qwen-7B | 45.6 | 89.5 | 1.6 | 81.0 | 99.3 | 0.8 | 1.1 | 99.1 |
| DeepSeek-R1† | 63.1 | 82.7 | 43.5 | 73.5 | 82.6 | 43.7 | 34.8 | - |
| Avg. | 51.2 | 87.5 | 14.8 | 78.6 | 95.3 | 11.8 | 9.6 | 96.1 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 62.5 | 86.8 | 38.3 | 78.2 | 88.8 | 34.7 | 33.3 | 83.8 |
| o1-mini† | 61.8 | 84.9 | 38.7 | 75.7 | 88.6 | 33.1 | 18.8 | - |
| Gemini-2.0-flash-exp | 61.8 | 83.2 | 40.4 | 73.8 | 80.5 | 45.7 | 44.2 | 81.4 |
| Gemini-2.0-thinking-exp-1219 | 60.3 | 87.1 | 33.5 | 78.4 | 90.5 | 28.0 | 27.0 | 87.0 |
| Avg. | 61.6 | 85.5 | 37.7 | 76.5 | 87.1 | 35.4 | 30.8 | 84.0 |
| Model Name | PRMScore | F1 | Negative F1 | Acc | Positive Acc | Negative Acc | First | similarity |
|---|---|---|---|---|---|---|---|---|
| Open-source Process Level Reward Models | ||||||||
| Skywork-PRM-1.5B | 32.3 | 36.3 | 28.2 | 32.5 | 22.8 | 84.0 | 79.1 | 74.5 |
| Skywork-PRM-7B | 37.4 | 44.9 | 29.9 | 38.3 | 29.8 | 83.3 | 77.8 | 73.3 |
| Llemma-PRM800k-7B | 53.5 | 77.8 | 29.1 | 66.2 | 70.4 | 43.8 | 23.1 | 85.1 |
| Llemma-MetaMath-7B | 51.3 | 80.6 | 22.1 | 68.9 | 76.6 | 27.9 | 15.1 | 85.6 |
| Llemma-oprm-7B | 51.3 | 78.4 | 24.1 | 66.4 | 72.5 | 33.8 | 17.2 | 86.6 |
| MATHMinos-Mistral-7B | 55.8 | 79.1 | 32.4 | 68.1 | 72.8 | 45.2 | 41.4 | 83.7 |
| MathShepherd-Mistral-7B | 48.6 | 65.6 | 31.7 | 54.2 | 52.5 | 62.8 | 57.1 | 84.0 |
| ReasonEval-7B | 58.0 | 90.6 | 25.4 | 83.2 | 96.7 | 16.9 | 24.6 | 93.8 |
| ReasonEval-34B | 64.3 | 84.4 | 44.3 | 75.6 | 79.4 | 57.3 | 56.8 | 83.6 |
| RLHFlow-PRM-Mistral-8B | 56.2 | 87.5 | 24.9 | 78.6 | 90.3 | 21.0 | 27.1 | 92.2 |
| RLHFlow-PRM-Deepseek-8B | 55.4 | 89.5 | 21.4 | 81.5 | 95.1 | 14.9 | 19.4 | 95.0 |
| Qwen2.5-Math-PRM-7B | 69.1 | 91.7 | 46.6 | 85.6 | 95.4 | 37.3 | 46.9 | 87.0 |
| Qwen2.5-Math-PRM-72B | 71.2 | 91.5 | 50.9 | 85.5 | 93.9 | 44.3 | 55.9 | 85.2 |
| Pure-PRM-7B | 69.2 | 90.2 | 48.3 | 83.6 | 91.3 | 45.3 | 51.7 | 84.5 |
| Avg. | 55.3 | 77.7 | 32.8 | 69.2 | 74.3 | 44.1 | 42.4 | 85.3 |
| Open LLMs, Prompted as Critic Models | ||||||||
| MetaMath-7B | 48.3 | 90.4 | 6.2 | 82.5 | 95.3 | 4.1 | 3.5 | 96.6 |
| MetaMath-13B | 48.1 | 91.4 | 4.8 | 84.3 | 98.7 | 2.6 | 4.0 | 99.6 |
| Qwen2.5-Math-72B | 53.8 | 90.5 | 17.1 | 82.9 | 99.3 | 9.6 | 11.5 | 96.3 |
| QwQ-Preview-32B | 62.7 | 89.2 | 36.1 | 81.6 | 94.9 | 26.6 | 29.8 | 88.8 |
| R1-Distill-Llama3.1-70B | 54.1 | 91.0 | 17.2 | 83.7 | 99.0 | 9.9 | 12.0 | 96.1 |
| R1-Distill-Qwen-7B | 46.8 | 91.8 | 1.9 | 84.9 | 99.5 | 1.0 | 1.2 | 99.3 |
| DeepSeek-R1† | 69.2 | 89.0 | 49.5 | 81.9 | 87.3 | 54.1 | 59.5 | - |
| Avg. | 54.7 | 90.5 | 19.0 | 83.1 | 96.3 | 15.4 | 17.4 | 96.1 |
| Proprietary LLMs, Prompted as Critic Models | ||||||||
| GPT-4o | 65.7 | 89.2 | 42.2 | 81.8 | 90.5 | 39.3 | 41.3 | 84.8 |
| o1-mini† | 64.8 | 86.7 | 42.9 | 78.4 | 84.5 | 48.2 | 43.8 | - |
| Gemini-2.0-flash-exp | 66.2 | 86.3 | 46.1 | 78.1 | 82.7 | 55.5 | 60.2 | 83.4 |
| Gemini-2.0-thinking-exp-1219 | 65.7 | 89.7 | 41.8 | 82.5 | 91.8 | 37.0 | 40.2 | 86.4 |
| Avg. | 65.6 | 88.0 | 43.2 | 80.2 | 87.4 | 45.0 | 46.4 | 84.9 |