olver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team
Abstract
Despite impressive progress on complex reasoning, current large language models (LLMs) typically operate in isolation—treating each problem as an independent attempt, without accumulating or integrating experiential knowledge. In contrast, expert problem solvers—such as Olympiad or programming contest teams—leverage a rich tapestry of experiences: absorbing mentorship from coaches, developing intuition from past problems, leveraging knowledge of tool usage and library functionality, adapting strategies based on the expertise and experiences of peers, continuously refining their reasoning through trial and error, and learning from other related problems even during competition. Inspired by this, we introduce olver—a training-free, multi-agent reasoning framework that equips a black-box LLM with a persistent, evolving memory of holistic experience. olver integrates diverse experience modalities, including external and self-retrieval, tool use, collaborative agent interactions, agent-driven evaluation, and iterative reasoning refinement. By learning from relevant strategies, code fragments, and abstract reasoning patterns at inference time, olver avoids generating solutions from scratch—marking a transition from isolated inference toward experience-aware language agents. Built on both open-weight and proprietary models, olver consistently outperforms specialized reasoning agents (e.g., OctoTools, CheatSheet, Search-o1). Even when instantiated with lightweight backbones (e.g., QWQ-32B), it often surpasses the most advanced models to date—including Qwen3-235B, Gemini 2.5 Pro, o3, and o4-mini-high. With a stronger backbone like o3-mini-high, it achieves a new best result—98.1% on GSM8K, 94.4% on AIME’24, 93.7% on AIME’25, 99.8% on Math-500, and 91.6% on LiveCodeBench—highlighting holistic experience learning as a key step toward dynamic, generalist agents capable of expert-level reasoning. We open-source all code, and data of olver at https://kagnlp.github.io/xolver.github.io/.
1 Introduction
Recent advances in large language models (LLMs) have made remarkable progress in complex reasoning and problem solving across domains such as mathematics [6, 15, 26] and programming [4, 3, 21]. Yet despite these impressive capabilities, conventional LLM reasoning approaches remain fundamentally limited: they standalone each problem instance, generating solutions from scratch without accumulating or transferring insights from rich, diverse experiential knowledge.
This isolated reasoning paradigm marks a significant departure from how expert human problem solvers operate. Expert problem solvers—such as an Olympiad or programming contest teams—rarely approach problems in a vacuum. Instead, they draw upon a rich tapestry of cumulative experiences: absorbing mentorship from coaches, developing intuition from past problems, leveraging knowledge of tool usage and library functionality (e.g., calculator), adapting strategies based on peers’ expertise and experiences, gaining insights through iterative trial and error, and learning from related problems even during competition. This holistic experience empowers them to tackle new challenges not from scratch, but by dynamically applying accumulated knowledge and adaptive strategies.
While numerous prior studies have enhanced LLM reasoning and problem solving through various forms of experiential knowledge augmentation, they have predominantly operated within discrete modalities—retrieving similar problems or relevant contexts [46, 25, 14], leveraging external tools [33, 32], or facilitating multi-agent collaboration [18, 16, 62]. Despite their individual strengths, these approaches address distinct facets of experiential knowledge independently, preventing LLMs from accumulating and synthesizing a comprehensive repertoire of learning signals across diverse experiential dimensions, thereby limiting the development of the rich, interconnected knowledge structures that characterize human expertise.
In this paper, we introduce olver, a unified, memory-augmented, multi-agent inference framework that emulates the holistic experience-driven, collaborative reasoning of expert teams. olver dynamically orchestrates a roster of specialized agents—such as mathematicians, programmers, verifiers—that iteratively tackle complex problems. Unlike conventional LLM pipelines, olver seamlessly integrates planning, episodic retrieval—both from external or self-parametric long-term memory—an evolving intermediate shared memory, tool invocation, multi-agent collaboration, agent-driven evaluation, and iterative self-refinement into a single adaptive architecture.
Each agent’s reasoning begins with exemplars drawn from episodic memory. From the second iteration onward, agents rely exclusively on an evolving shared memory that records the highest-quality reasoning paths, solutions, and evaluation feedback generated so far—thereby accumulating symbolic experience over time. This shared repository guides agents to build on successful strategies, correct mistakes, and improve solution quality. When needed, agents invoke external tools (e.g., code execution), and a dedicated judge agent reviews all outputs—selecting top responses, issuing feedback, and enriching the intermediate shared memory with curated traces and collective evaluations for future rounds. Iterations continue until outputs converge or a preset limit is reached, followed by a final verification or external debugging phase to ensure correctness. Additionally, by updating its episodic store with each newly solved problem and its reasoning trace, olver can continually expand its knowledge base. Through this closed loop of collaborative agents, memory-guided refinement, and tool guided precision, olver features a more holistic experience learning and transcends static LLM inference, delivering adaptive, expert-level reasoning over time. Figure 2 illustrates the workflow.
We conduct large-scale experiments across a range of math and programming benchmarks—including GSM8K, Math-500, AIME (2024 and 2025), and LiveCodeBench (v5)—using both proprietary (o3-mini-medium) and open-weight (QWQ-32B) backbone models. olver consistently outperforms specialized reasoning systems such as OctoTools [33], CheatSheet [55], and Search-o1 [28]. Remarkably, even when instantiated with lightweight models, olver often surpasses significantly larger state-of-the-art LLMs, including Qwen3-235B [56], Gemini 2.5 Pro [7], o1, o3, and o4-mini-high [41]. As in Figure 1, olver (m) achieves 91.6% average accuracy on the AIME ’24 and ’25 benchmarks—an 18.5-point gain over o3-mini-medium—while olver (h) reaches 94.1%, outperforming o3-mini-high by 7.2 points. On LiveCodeBench, olver (m) improves upon its base by 21 points (66.3% to 87.3%), with olver (h) achieving 91.6%, a 22.1-point lift over o3-mini-high.
Our analysis reveals how olver’s experiential components contribute to its performance. Accuracy improves consistently with more agents and iterations, reflecting the benefits of experience accumulation, though at increased cost. While external retrieval remains powerful, we find that self-retrieval—drawing from the model’s own parametric memory—can serve as an alternative with some performance drop. For tasks involving symbolic reasoning and complex arithmetic, multi-agent, multi-iterative refinement is more beneficial than tool use (e.g., Python execution). Our experiments confirm that even without updating episodic memory during inference, olver retains substantial performance gains, emphasizing the strength of its intermediate memory and iterative refinement. Together, these findings highlight olver ’s ability to accumulate, refine, and reuse symbolic experience through collaborative, memory-guided reasoning.
2 The olver Framework
Given a problem query and a pretrained language model , a conventional approach generates a solution via single-step inference: . In contrast, olver executes a dynamic, multi-agent reasoning process that iteratively accumulates and leverages symbolic experience to solve complex problems more effectively.
To support structured collaborative reasoning, olver maintains two complementary forms of memory: an episodic memory , which stores a library of past problems, solutions, and reasoning traces; and an intermediate dynamic shared memory , which evolves during inference to retain high-quality agent trajectories—comprising reasoning thoughts, responses, agent metadata, and feedback. In olver, a multi-agent team is orchestrated adaptively by a planner agent , which assigns roles and configures memory access. During inference, agents leverage an external toolset (e.g., Python interpreter) to support accurate computation. Finally, a verifier or external debugger is invoked to extract and format the final answer, and to validate correctness for executable outputs.
Below, we first describe the olver agents and tools in Section 2.1, followed by the memory components in Section 2.2, and the inference cycle in Section 2.3.
2.1 Agents and Tools
Planner Agent . The planner agent is responsible for initiating, planning, and orchestrating the olver multi-agent architecture. Given the problem and the number of agents , it constructs a team of dynamic agents, each assigned a distinct expert role (e.g., algebra solver, mathematician, theorist, programmer, algorithm designer) tailored to the demands of . To ensure sufficient task coverage and role diversity, first prompts the underlying LLM to over-generate candidate agents, from which it then selects the most effective subset such that . A summary of the most frequently generated and selected roles is provided in Appendix D.4.
Dynamic Reasoning Agents . The set represents a team of dynamic reasoning agents constructed by the planner agent . Each agent is assigned a distinct expert role (e.g., algebra solver, programmer, counter-example generator) tailored to the task query . Agents are instantiated using a standardized prompting template (see Appendix A) that incorporates the task description, assigned role, retrieved examples, prior reasoning attempts, and shared memory feedback—enabling iterative self-correction and role specialization.
At each iteration , agent receives a context and generates a structured reasoning trace and a response . For the first iteration (), the context is initialized using the task query and relevant retrieved exemplars:
For subsequent iterations (), the context evolves by incorporating its prior generation (history) and the shared memory:
Judge Agent .
The judge agent evaluates intermediate outputs from each agent and returns structured feedback to guide refinement and memory updates. Given a query , a reasoning trace , and a response , it produces a feedback tuple , where is a natural language explanation (e.g., critique, justification, correction), and is a scalar quality score. The interpretation of is task-dependent: for math problems, reflects an LLM-estimated correctness probability; for code tasks, , where denotes the total number of test cases including problem-provided samples and 10 synthesized test cases generated using AceCode-RM-32B [67]. To avoid compiler interaction latency and maintain symbolic traceability, test case outcomes are determined by simulating execution through LLM prompting within the judge agent , following the CodeSim protocol [18]. This structured feedback enables agents to identify failures, receive localized corrections, and improve reasoning over iterations.
Verifier Agent .
Due to linguistic complexity and varying answer specification formats, a response may be incorrect even when the underlying reasoning or open-ended response is valid. For instance, answer formats may require multiple-choice letters (e.g., “(A)” or “Choice B”), boxed numerical values (e.g., “”), or final answers in specific units (e.g., “5 km” or “12%”). An additional round of answer extraction and formatting helps reduce such mispredictions [44]. This challenge is even more pronounced in code generation tasks, where predicted code may fail to execute or not pass all test cases. To mitigate this, olver includes a Verifier Agent , which operates differently based on the output type. For math and QA problems, extracts the final reasoning , response , and answer from the response associated with the top-ranked entry BestResponse in , ensuring adherence to the expected output format. For executable code, olver invokes an external debugger (LDB [70]), where interacts with a Python runtime to capture execution feedback and iteratively fix runtime errors.
Tools .
Integrating natural language reasoning with tools like Python execution is a proven way to boost performance on complex reasoning tasks [37, 57]. We observe that even advanced reasoning models often make mistakes in intermediate steps, particularly when computations become non-trivial. To address this, each dynamic agent is explicitly instructed to use Python execution during reasoning when needed. While olver currently limits to Python, our prompting strategy is tool-agnostic, allowing an interface for future extensions to richer toolsets [32, 33].
All agents are built using the underlying LLM. All prompts are 0-shot and provided in Appendix A.
2.2 Memory Components
Episodic Memory . olver maintains two forms of episodic (long-term) memory: (1) an external memory corpus , which consists of past problem instances , their corresponding reasoning traces (optional), and solution responses ; and (2) the internal parametric memory encoded in the weights of the agent-specific language model .
We define a general retrieval operator that returns a set of examples relevant to the query . When is available, retrieval is conducted using similarity-based search (e.g., BM25):
Otherwise, olver falls back to internal self-retrieval by sampling from the agent model itself:
In the case of an external episodic memory, can also be updated with UpdateEpisodicMemory by adding the top-ranked reasoning and response from , paired with the problem , into the external corpus . That is, where is the top-ranked entry in .
Intermediate Shared Memory .
The shared memory maintains a fixed-size set of high-quality intermediate reasoning, responses, and metadata generated by the dynamic agents during inference on the current query . For simplicity and to preserve the dynamic nature of the framework, we constrain , where is the number of dynamic agents in . Initially, . At each iteration , each agent produces a reasoning trace , response , and receives structured feedback from the judge agent , where is a natural language explanation and is a scalar score reflecting the quality of the tuple . After collecting the new outputs
we form the candidate pool We then update the fixed-size shared memory by keeping only the top- tuples by score
where extracts the scalar score from .
This replacement mechanism ensures that always contains exactly entries with the highest observed scores across all iterations. By maintaining only the strongest reasoning-response-feedback tuples, the shared memory facilitates knowledge transfer between agents and across iterations, enabling collaborative improvement through exposure to diverse high-quality solutions.
2.3 Inference Protocol
Algorithm 1 summarizes the olver inference protocol, which operates in three structured stages. Stage-1, which emulates initialization with prior experience, involves the planner constructing a team of agents (lines 2–3). Stage-2, embodying symbolic experience accumulation and refinement, iterates for rounds (lines 4–10). In each round, all agents receive access to and , build their contexts, and generate structured trajectories and responses ( is only used for context construction at the first iteration). These are evaluated by the judge agent , and is updated with the resulting feedback tuples (line 7). Upon convergence or after rounds, Stage-3 invokes the verifier agent , which extracts the final answer from the top-ranked entry in (line 11), and updates with the new experience.
3 Experiments
3.1 Evaluation Setup
Evaluation Benchmarks We evaluate olver across five diverse and challenging benchmarks covering both mathematical and coding reasoning. For math, we use GSM8K [6], Math-500[15], and the AIME 2024 [34] and 2025 [35], comprising high-school level competition problems requiring multi-step symbolic reasoning. For coding, we use LiveCodeBench (v5) [20], a dynamic benchmark that ensures no data leakage by periodically releasing new problems. These benchmarks span arithmetic, algebra, number theory, geometry, combinatorics, and algorithmic problem solving.
Baselines and Metrics We compare olver against directly using leading reasoning models– (a) proprietary models: Gemini 2.5 (Pro and Flash Think) [7], Grok-3 Beta Think and Grok-3 Mini (Beta) Think [63], Claude 3.7 Sonnet Think [2], o1 [41], o3-mini, o3, and o4-mini [42]; (b) open-weight LLMs, e.g., Qwen3-235B [48], QWQ-32B [49], and DeepSeek-R1 [8]; (c) math- and code-specialized models, e.g., AlphaOne [68], OpenMathReason [37], rStar-Math [12], rStar-Coder [30], OpenCodeReason [1], and Kimi-K1-1.6 [22]. We also compare with (d) agents or frameworks: Self-Reflexion [52], agentic search based framework Search-o1 [28], specialized tool based framework OctoTools [33] which excels general purpose agent platforms outperforming AutoGen or LangChain, cross-problem baseline framework CheatSheet [55], and multi-agent code generation framework CodeSim [18], which leverage refinement, retrieval or online search, fine-grained tool augmentation in addition to online search, dynamic memory updates after solving new problems, and multi-agent reasoning techniques respectively. For agent-based baselines (d), we reproduce results using the same backbone LLMs as olver for fair comparison; for model-based baselines (a–c), we report official results from their technical reports or corresponding benchmark leaderboards. As evaluation metric, we use accuracy using GPT-4o [40] for math problems, and pass@1 for code tasks.
Inference Details We use both open-weight QWQ-32B [48] and proprietary o3-mini (medium and high) [42] as the backbone. To mitigate performance variance inherent in single-run evaluations, we report the average accuracy and pass@1 metric, calculated by averaging 32 inference runs for competitive benchmarks LiveCodeBench and AIME ’25, and 16 runs for AIME ’24, ensuring standard deviation within (Appendix D.1). For simpler tasks GSM8K and Math-500, we follow DeepSeek-v3 [29], using a single greedy-decoded generation. By default, we set temperature to 0.2, number of agents , and max iterations . olver iteration terminates either when the maximum number of iterations is reached, or when all entries in the shared memory converge—i.e., they achieve perfect scores of 1.0 (correct) for math tasks, or pass all test cases (both sample and synthesized) for code tasks. As the external retrieval corpus in coding task, we collect a 9-million-token dataset of algorithmic code problems and their C++ solutions with explanations from GitHub11 1 https://github.com/cp-algorithms/cp-algorithms (details in Appendix C). For math, we use the OpenMathReason dataset [37] as . We evaluate two variants of olver: (i) olver with in-competition cross-problem experience (olver (+)), which dynamically updates the episodic memory after solving each problem to utilize accumulated knowledge across problems; and (ii) olver (–), which keeps the episodic memory static, focusing solely on problem-specific experience. By default, we refer to olver (+) as our method if not specified otherwise.
3.2 Main Results
| Model | Appr. | GSM8K | AIME ’24 | AIME ’25 | Math -500 | LiveCodeBench (v5) |
|---|---|---|---|---|---|---|
| Proprietary Models | ||||||
| Claude 3.7 Sonnet T. | LongCoT | – | 61.3 | 49.5 | 96.2 | 51.4 |
| Grok-3 (Beta) T. | Direct | – | 83.9 | 77.3 | – | 70.6 |
| Grok-3-mini (Beta) T. | LongCoT | – | 89.5 | 82.0 | – | - |
| Gemini 2.5 Flash T. | LongCoT | – | 88.0 | 78.0 | – | 63.5 |
| o1 | LongCoT | 96.4 | 74.3 | 79.2 | 96.4 | 71.0 |
| o3-mini-high | LongCoT | – | 87.3 | 86.5 | – | 69.5 |
| Gemini 2.5 Pro. | Direct | – | 92.0 | 86.7 | – | 70.4 |
| o3 | LongCoT | 96.7 | 91.6 | 88.9 | – | – |
| o4-mini-high | LongCoT | – | 93.4 | 92.7 | – | 69.5 |
| Open Weights Models | ||||||
| DeepSeek-R1 | LongCoT | – | 79.8 | 70.0 | 97.3 | 64.3 |
| Qwen3-235B-A22B | LongCoT | – | 85.7 | 81.5 | – | 70.7 |
| Math/Code Specialized Models | ||||||
| rStar-Math (Best) | – | 95.2 | 53.3 | – | 90.0 | – |
| OpenMathReason (Best) | – | – | 93.3 | 80.0 | – | – |
| AlphaOne (Best) | – | – | 53.3 | – | 89.4 | 75.8 |
| OpenCodeReason (Best) | – | – | – | – | – | 61.8 |
| rStar-Coder (Best) | – | – | – | – | – | 62.5 |
| Kimi-k1.6-IOI-high | – | – | – | – | – | 73.8 |
| Reasoning Agents/Frameworks | ||||||
| o3-mini-medium | LongCoT | 95.2 | 75.8 | 70.4 | 97.3 | 66.3 |
| Self-Refl. | 93.1 | 79.4 | 76.5 | 95.2 | 73.2 | |
| OctoTools | 95.4 | 81.7 | 75.3 | 97.5 | – | |
| Search-o1 | 95.8 | 81.8 | 76.7 | 97.6 | 73.6 | |
| CheatSheet | 95.9 | 82.2 | 75.8 | 97.7 | – | |
| CodeSim | – | – | – | – | 73.8 | |
| olver (–) | 95.6 | 87.2 | 85.1 | 97.7 | 79.6 | |
| olver (+) | 97.1 | 93.8 | 89.4 | 99.2 | 87.3 | |
| QWQ-32B | LongCoT | 96.1 | 78.1 | 65.8 | 83.2 | 63.4 |
| Self-Refl. | 94.0 | 79.3 | 66.3 | 80.4 | 69.2 | |
| OctoTools | 96.3 | 83.0 | 71.7 | 86.1 | – | |
| Search-o1 | 96.4 | 84.4 | 71.8 | 87.1 | 69.3 | |
| CheatSheet | 96.8 | 83.5 | 72.2 | 86.5 | – | |
| CodeSim | – | – | – | – | 70.5 | |
| olver (–) | 96.5 | 89.9 | 79.5 | 93.1 | 76.2 | |
| olver (+) | 98.0 | 93.6 | 82.7 | 95.5 | 79.2 | |
| o3-mini-high | olver (+) | 98.1 | 94.4 | 93.7 | 99.8 | 91.6 |
Table 1 evaluates olver across diverse mathematical and coding reasoning benchmarks, highlighting its effectiveness compared to state-of-the-art LLMs, specialized models, and other frameworks.
Strong Gains Across Benchmarks Overall, olver consistently delivers significant improvements over the backbone LLMs’ standard LongCoT prompting. Both the problem-specific olver (–) and the cross-problem olver (+) variants outperform their respective backbone LLM (LongCoT) baselines across all datasets. For example, with the o3-mini-medium backbone, olver (+) improves from 75.8 to 93.8 on AIME’24, and from 66.3 to 79.6 on LiveCodeBench, while the QWQ-32B backbone sees gains from 78.1 to 89.9 on AIME’24 and from 63.4 to 76.2 on LiveCodeBench.
Surpassing Prior Agents Compared to previous frameworks such as Search-o1, OctoTools, and CheatSheet, olver demonstrates consistent and significant gains. With o3-mini-medium, olver (+) improves over the best baseline by +12.7 points on AIME’25 and +13.5 points on LiveCodeBench, highlighting its superior reasoning capabilities by integrating diverse forms of experience.
In Comparison to Leading LLMs Despite using weaker backbones, olver, specifically (+) variant, matches or surpasses proprietary frontier LLMs like o3 and o4-mini-high on key benchmarks. With o3-mini-medium, olver (+ outperforms o4-mini-high on AIME’24 (93.8 vs. 93.4) and substantially exceeds it on LiveCodeBench (87.3 vs. 69.5), demonstrating that structured reasoning and dynamic memory can rival even the strongest closed-source models.
Backbone Agnostic Improvements from olver are consistent across different backbone LLMs. Both o3-mini-medium and QWQ-32B benefit substantially from the framework, demonstrating its model-agnostic design. For example, on GSM8K, olver (+) achieves 97.1 (o3-mini-medium) and 98.0 (QWQ-32B), both surpassing baseline variants by significant margins.
Effectiveness of Dynamic Episodic Memory While both variants excel, the cross-problem variant olver (+) consistently outperforms the problem-specific version olver (-) in all benchmarks. On average, episodic memory integration yields a +3.5 point improvement across both backbones and datasets where the largest gain is +7.7 points with o3-mini-medium on coding (LiveCodeBench).
Scales with Backbone LLM’s Strength olver’s performance scales consistently with the strength of its backbone LLM. With o3-mini-high, it sets new state-of-the-art results across all benchmarks (98.1 on GSM8K, 94.4 on AIME’24, 93.7 on AIME’25, 99.8 on Math-500, and 91.6 on LiveCodeBench).
4 Ablation and Analyses
Ablations: Quantifying Component Impact In Figure 3, we present an ablation study quantifying the contribution of individual components in olver to overall performance, measured by the average performance drop on math reasoning (Math Avg) and programming (LiveCodeBench) tasks.
Each component plays a necessary role, with the most significant degradation observed when removing Multi-iteration and Multi-Agent followed by Judge Agent, highlighting their central importance in complex reasoning and code synthesis. In contrast, removing components like Verifier/Debugger and Tool leads to comparatively smaller drops, suggesting a more auxiliary role in the overall system. Likewise self-retrieval can also work in-place of external retrieval with some drop in accuracy.
Impact of Agent Count and Iterations, and Emerging Benefits of Collaboration We analyze the effect of varying the number of agents and reasoning iterations on olver’s performance. In a controlled setup, we fix one variable (e.g., 3 agents or 2 iterations) and incrementally increase the other. As shown in Figure 4, performance improves consistently on both AIME ’25 and LiveCodeBench with more agents or iterations, highlighting the advantage of collaborative and iterative problem solving.
To probe deeper, we conduct a budget-controlled experiment on the AIME ’25 dataset, where the total reasoning budget (i.e., number of agents number of iterations) is fixed. While iterative reasoning remains a crucial factor for olver’s performance, we find that increasing the number of agents—particularly beyond a minimum of three—yields additional, emergent improvements, leading to over a 4% performance gain. This suggests that agent diversity and parallelism complement iterative depth, together producing stronger collaborative problem-solving benefits than either alone.
Effect of Retrieval Strategies on olver Performance. We evaluate the impact of different retrieval strategies on olver by comparing three settings: (1) External Retrieval, where the model retrieves the top- (e.g., ) most similar problems and their solutions from an external corpus using a BM25 retriever; (2) Self-Retrieval, where the model recalls the top- most similar problems and solutions from its own internal memory; and (3) No Retrieval, where neither external nor self-retrieval is used. As shown in Figure 5, performance on both AIME ’25 and LiveCodeBench follows the trend: External Retrieval Self-Retrieval No Retrieval, indicating that external retrieval significantly enhances olver’s performance. We note that for code tasks, although the external retrieval corpus contains solutions written in C++—a different language from the target Python—external retrieval still provides a substantial performance boost. Nonetheless, while self-retrieval results in a notable performance drop compared to external retrieval, it still outperforms the no-retrieval baseline with notable margins, serving as a viable alternative when external resources are unavailable.
Fine-grained Performance Analysis We perform a fine-grained analysis of olver’s performance across both Math-500 and LiveCodeBench, as shown in Figure 6 and Figure 7.On Math-500, olver (both o3-mini-medium and QWQ-32B) consistently outperforms CheatSheet across nearly all seven subject categories, despite the latter relying on costly per-problem memory updates. The only exception is in Number Theory, where o3-mini-medium scores 99.2 compared to CheatSheet’s 99.5. As for QWQ-32B, olver achieves substantial accuracy gains over CheatSheet across all categories, with improvements of +9.0% in Prealgebra, +8.5% in Algebra, +11.0% in Number Theory, +8.5% in Counting and Probability, +8.8% in Geometry, +10.0% in Intermediate Algebra, and +7.5% in Precalculus. These consistent gains highlight olver’s strong performance across both symbolic and numerical reasoning.
On LiveCodeBench, olver demonstrates even more pronounced gains. The o3-mini-medium variant achieves 95.6%, 90.4%, and 85.8% accuracy on Easy, Medium, and Hard problems respectively, significantly outperforming CodeSim by +4.5%, +11.9%, and a striking +32.3% margin on hard examples. Even with a weaker QWQ-32B backbone, olver (95.2%, 87.5%, 70.0%) surpasses all baselines and achieves similar gains. In contrast to CheatSheet and CodeSim, olver leverages multi-agent collaborations and holistic experience learning. These consistent and backbone-agnostic gains across different reasoning tasks underscore olver’s robustness and position it as a breakthrough in retrieval and tool-augmented, multi-agent and evolving reasoning systems.
Can a Self-Judge Replace a Judge Agent? We analyze the effect of different judging mechanisms on olver’s performance by comparing two setups: (1) self-judging, where each dynamic agent evaluates its own response through self-reflection without altering its role, and (2) external judging, where a separate judge agent is used to assess the responses. We find that self-judging agents tend to be biased in favor of their own outputs, occasionally validating incorrect solutions. This self-bias leads to a noticeable drop in overall performance—specifically, a 9.9% decrease in coding tasks and a 3.88% decrease in math tasks, on average.
Cost Analysis and How Long Do olver Agents Think? We perform a detailed analysis of token usage in Figure 8, reporting input, reasoning, and output statistics for olver (QWQ-32B) across all datasets. Our LLM token usage has computational complexity of , where is the number of agents and is the number of reasoning iterations. However, the runtime complexity remains since the dynamic agents operate in parallel. This is significantly more efficient than the self-consistency [59], which typically require 32–64 generations per example, as well as the baseline CheatSheet framework, which incurs a memory update complexity of —quadratic in the test dataset size—due to usefulness estimation over all previous examples after solving each new example. As a multi-agent system, olver allocates a majority of its tokens to context sharing and inter-agent communication, while approximately 25% are spent on actual reasoning steps.
Nonetheless in Figure 8, we also compare the total token usage of olver with a single agent reasoning framework Search-o1 using tiktoken for o3-mini-medium and AutoTokenizer for QWQ-32B for token count. As expected, olver incurs higher token costs—approximately that of Search-o1—due to its collaborative and iterative multi-agent reasoning. However, this moderate increase represents a highly efficient trade-off given the substantial performance improvements observed. As shown in Figure 6 and Figure 7, olver achieves remarkable gains across both domains, including a +32.3% absolute improvement on hard coding problems with o3-mini-medium and 9.05% accuracy boosts across all Math-500 categories with QWQ-32B. These findings demonstrate that olver ’s slightly higher reasoning cost is well-justified by its superior, generalist performance across diverse problem-solving scenarios.
Does Data Shuffling Affect olver (+) Performance? olver (+) updates its external memory incrementally after solving each new problem. To examine whether the order of test instances impacts performance, we conduct an ablation study by randomly shuffling the sequence of problems in each task. This helps determine if there is any dependency on the data order. Results in Appendix D.3 show that olver exhibits minimal performance variation across different shuffles, with a standard deviation of approximately within only 5 runs, indicating that its performance is largely stable regardless of data ordering.
Qualitative Examples In Appendix B, we present qualitative examples along with all the prompts of full-cycle olver on both math and code reasoning tasks. These examples illustrate how olver initiates reasoning from external or self-retrieved exemplars, engages in multi-agent collaboration, and incrementally accumulates experiences through inter-agent propagation and refinement. The full interaction trace highlights olver’s ability to iteratively decompose, solve, and adapt solutions across reasoning steps, showcasing its capacity for dynamic knowledge construction and generalizable problem solving.
More Error Analysis in Math and Code In Figure 9, we present an error analysis across both math and code tasks that goes beyond simple accuracy or pass@1 metrics. While olver significantly improves reasoning and generation capabilities in both domains, both (o3-mini-medium and QWQ-32B) backbone LLMs can still produce solutions that are syntactically correct yet semantically flawed, resulting in failed executions due to incorrect reasoning, incomplete logic, unoptimized implementations, or misaligned tool usage. In code tasks, failure modes include incorrect final code, time limit exceeded (TLE), runtime errors (RTE), and syntax issues. In math tasks, remaining errors are primarily due to flawed logical derivations or faulty intermediate calculations. Although Python-based tools are available, such calculation errors often occur when agents choose not to invoke these tools—highlighting that tool usage remains decoupled from the model’s core reasoning process (see Appendix A for our prompt design). These findings provide insights for future improvements by exposing the variety of failure modes across domains, and further emphasize the importance of robust self-verification and refinement mechanisms, as employed by olver.
Dynamics of Reasoning Patterns in olver Traces To understand how olver adapts its reasoning process to perform complex reasoning, we analyze the dynamics of reasoning pattern frequencies across difficulty levels in LiveCodeBench, as shown in Table 2. Detailed description of how we collected the reasoning patterns is provided in the Appendix D.1. Our analysis reveals that olver dynamically increases self-evaluation and exploratory strategies (e.g., trying new approaches) as problem difficulty grows. Correct solutions demonstrate a declining need for problem rephrasing and subgoal decomposition, indicating more direct and confident reasoning. In contrast, incorrect solutions show increased subgoal setup and rephrasing attempts—suggesting that the system recognizes failure and attempts recovery through restructuring. Compared to OpenCodeReasoning, which shows stagnation or regression in key patterns (e.g., self-evaluation), olver exhibits robust and adaptive reasoning behavior, supported by multi-agent collaboration and judge feedback. This behavior highlights the generality and flexibility of olver ’s reasoning model.
Reasoning Pattern Correct Solutions Incorrect Solutions Easy Medium Medium High Easy Medium Medium High Self-Evaluation () 0.35 0.37 New Approach () 0.18 0.21 0.21 0.24 0.17 0.24 0.24 0.26 Problem Rephrasing () 0.20 0.17 0.18 0.18 0.23 0.24 0.24 0.25 Subgoal Setup () 0.14 0.13 0.13 0.11 0.11 0.11
5 Case-Study: How olver Enhances Reasoning
To further understand the reasoning and problem-solving strategies behind our multi-agent, iterative framework olver, we conduct an in-depth analysis combining qualitative runtime inspection with controlled experiments. We begin by manually studying olver’s agent interaction traces on AIME ’25 and LiveCodeBench. These case studies reveal that at each iteration, dynamic agents attempt to improve upon earlier failures by leveraging Judge agent feedback and by aligning with top-ranked outputs stored in the shared memory . This process results in progressively refined outputs, increased agent alignment, and eventual convergence toward correct solutions.
To verify this behavior systematically, we conduct a controlled experiment across both math and code tasks. We instantiate two dynamic agents with complementary strengths: a Coder agent and a Mathematician agent, each proficient in one domain but suboptimal in the other. We then measure their performance and agreement across iterations—defined as the percentage of problems in which both agents independently produce the same correct answer (for math) or code that passes the same test cases (for code). As shown in Figure 10, both agents demonstrate consistent accuracy improvements over time, accompanied by a rising agreement rate. This not only illustrates mutual influence and learning-by-alignment but also validates the emergence of collaborative synergy.
Crucially, we observe that the presence of the Judge agent plays a vital role in this convergence process. When the Judge agent is removed—as shown in our first ablation—performance degrades significantly. These findings collectively affirm that olver’s iterative memory-sharing, feedback-driven refinement, and role-specialized agents contribute to its strong reasoning performance across domains, making it a compelling framework for general-purpose, self-improving problem solving.
6 Related Work
Memory-Augmented and Retrieval-Augmented LLMs. Memory-augmented language models have evolved from static retrieval systems like RAG [25] and REALM [14] to dynamic approaches such as Reflexion [53], MemGPT [43], and Scratchpads [39]. However, these systems operate on isolated tasks, lack cross-problem experience accumulation, and employ single-agent architectures. olver addresses these limitations through a novel dual-memory architecture combining episodic long-term memory with dynamic intermediate memory, enabling specialized agents to collectively build and refine experiential knowledge. While prior work has explored cross-trial information sharing [69, 53] and multi-source memory integration [66], these approaches remain confined to single-agent settings. Our framework creates a persistent knowledge base through multi-agent collaboration [10], allowing agents to accumulate expertise from solved problems and leverage collective experience for future tasks.
Multi-Agent Problem Solving. Multi-agent LLM systems address the limitations of single models by leveraging collaborative approaches for improved reliability and task specialization [13, 10]. From early frameworks like CAMEL [27] with fixed role assignments, the field progressed to dynamic role adjustment in AgentVerse [5] and code execution in AutoGen [62]. Recent advances include layered agent networks in DyLAN [31], multi-agent code generation and problem solving [17, 18] and multi-agent debate frameworks [9, 50, 54]. While these systems demonstrate effective collaboration, they operate on isolated problems without cross-task experience accumulation. olver introduces dual-memory architecture, holistic experience integration, judge-mediated selection, and continuous episodic corpus expansion—transforming single-problem solvers into experience-aware agents.
LLM Reasoning Enhancement Techniques. Various techniques have emerged to enhance LLM reasoning capabilities beyond standard prompting. Chain-of-Thought [61] introduced step-by-step reasoning, Self-Consistency [58] explores multiple reasoning paths with majority voting, and Tree of Thoughts [64] enables exploration of reasoning branches—yet all remain limited to single-pass generation. Self-reflective approaches like Reflexion [53] enable iterative improvement but operate within single tasks, while retrieval-enhanced methods like CheatSheet [55] and Search-o1 [28] remain confined to single-agent architectures. These approaches share fundamental limitations: no cross-problem learning, no persistent memory, and no multi-agent collaboration. olver unifies these enhancements within a multi-agent framework where agents collaboratively refine solutions through judge-mediated iterations and leverage dual memory systems for cross-problem learning.
Tool-Augmented Reasoning. Tool integration extends LLM capabilities beyond language processing. Early systems like WebGPT [38] introduced single-tool integration, while PAL [11] enabled code execution for mathematical reasoning. Multi-tool frameworks evolved with ReAct [65] interleaving reasoning with actions, Chameleon [32] composing multiple tools, and OctoTools [33] standardizing tool planning—yet all remain limited to single-agent execution without iterative refinement or cross-problem learning. olver transforms tool use into a collaborative, memory-enriched ecosystem where agents collectively execute tools, share outcomes, and accumulate successful strategies across problems—creating an adaptive framework that evolves with experience.
7 Conclusion
We propose olver, an open-source, multi-agent inference framework for complex reasoning tasks that enables holistic experience learning. olver integrates (1) episodic retrieval from external or self-parametric memory, (2) an evolving intermediate shared memory that accumulates and reuses high-quality reasoning traces, (3) tool invocation for complex computations, (4) collaborative multi-agent reasoning, (5) self-evaluation and iterative refinement, (6) verification or external debugging, and (7) propagation of learned strategies across problems. These components collectively support adaptive, experience-informed problem solving. Despite its strong performance, olver faces limitations in computational efficiency, with substantially higher token consumption than traditional approaches, and remains dependent on the quality of backbone LLMs. Future work aims to optimize agent interactions to reduce resource requirements, enhance robustness to variations in model quality, improve retrieval filtering [60, 51, 45], develop better RAG strategies [19, 47, 24], and extend the framework to more diverse reasoning domains beyond mathematics and programming. In addition, we plan to integrate advanced external verifiers of reasoning [36] to further enforce validity through structured guardrails. By addressing these challenges, we aim to further advance the development of experience-aware reasoning systems that can approach the adaptability and integrated knowledge use of human experts.
References
- [1] Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jocelyn Huang, Vahid Noroozi, and Boris Ginsburg. Opencodereasoning: Advancing data distillation for competitive coding, 2025. URL https://arxiv.org/abs/2504.01943.
- [2] Anthropic. Claude 3.7 Sonnet, 2025. URL https://www.anthropic.com/news/claude-3-7-sonnet.
- [3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021.
- [4] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
- [5] Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2(4):6, 2023.
- [6] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
- [7] Google DeepMind. Gemini 2.5, 2025. URL https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/.
- [8] DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
- [9] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325, 2023.
- [10] Shangbin Feng, Wenxuan Ding, Alisa Liu, Zifeng Wang, Weijia Shi, Yike Wang, Zejiang Shen, Xiaochuang Han, Hunter Lang, Chen-Yu Lee, Tomas Pfister, Yejin Choi, and Yulia Tsvetkov. When one llm drools, multi-llm collaboration rules. ArXiv, abs/2502.04506, 2025. URL https://api.semanticscholar.org/CorpusID:276235808.
- [11] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764–10799. PMLR, 2023.
- [12] Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small llms can master math reasoning with self-evolved deep thinking, 2025. URL https://arxiv.org/abs/2501.04519.
- [13] Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680, 2024.
- [14] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. REALM: Retrieval-augmented language model pre-training. In International Conference on Machine Learning. JMLR.org, 2020. URL https://dl.acm.org/doi/abs/10.5555/3524938.3525306.
- [15] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021.
- [16] Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 3(4):6, 2023.
- [17] Md. Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. MapCoder: Multi-agent code generation for competitive problem solving. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4912–4944, Bangkok, Thailand, August 2024a. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.269. URL https://aclanthology.org/2024.acl-long.269/.
- [18] Md. Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. CodeSim: Multi-agent code generation and problem solving through simulation-driven planning and debugging. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Computational Linguistics: NAACL 2025, pages 5113–5139, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195-7. doi: 10.18653/v1/2025.findings-naacl.285. URL https://aclanthology.org/2025.findings-naacl.285/.
- [19] Shayekh Bin Islam, Md Asib Rahman, K S M Tozammel Hossain, Enamul Hoque, Shafiq Joty, and Md Rizwan Parvez. Open-RAG: Enhanced retrieval augmented reasoning with open-source large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14231–14244, Miami, Florida, USA, November 2024b. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-emnlp.831. URL https://aclanthology.org/2024.findings-emnlp.831/.
- [20] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
- [21] Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, and Shafiq Joty. XCodeEval: An execution-based large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6766–6805, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.367. URL https://aclanthology.org/2024.acl-long.367/.
- [22] Kimi Team. Kimi k1.5: Scaling reinforcement learning with llms, 2025.
- [23] Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, and Jimmy Huang. A systematic survey and critical review on evaluating large language models: Challenges, limitations, and recommendations. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13785–13816, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.764. URL https://aclanthology.org/2024.emnlp-main.764/.
- [24] Ahmed Lekssays, Utsav Shukla, Husrev Taha Sencar, and Md Rizwan Parvez. Techniquerag: Retrieval augmented generation for adversarial technique annotation in cyber threat intelligence text, 2025. URL https://arxiv.org/abs/2505.11988.
- [25] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
- [26] Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843–3857, 2022.
- [27] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991–52008, 2023.
- [28] Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366, 2025.
- [29] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
- [30] Yifei Liu, Li Lyna Zhang, Yi Zhu, Bingcheng Dong, Xudong Zhou, Ning Shang, Fan Yang, and Mao Yang. rstar-coder: Scaling competitive code reasoning with a large-scale verified dataset. arXiv preprint arXiv:2505.21297, 2025.
- [31] Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170, 2023.
- [32] Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36, 2024.
- [33] Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. Octotools: An agentic framework with extensible tools for complex reasoning. arXiv preprint arXiv:2502.11271, 2025.
- [34] MAA. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME 2024, February 2024. URL https://maa.org/math-competitions/american-invitational-mathematics-examination-aime.
- [35] MAA. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME 2025, February 2025. URL https://maa.org/math-competitions/american-invitational-mathematics-examination-aime.
- [36] Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832, 2024.
- [37] Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset, 2025. URL https://arxiv.org/abs/2504.16891.
- [38] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021.
- [39] Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021.
- [40] OpenAI. Hello GPT-4o, 2024a. URL https://openai.com/index/hello-gpt-4o/.
- [41] OpenAI. Learning to reason with llms, 2024b. URL https://openai.com/index/learning-to-reason-with-llms/.
- [42] OpenAI. Introducing openai o3 and o4-mini, 2025. URL https://openai.com/index/introducing-o3-and-o4-mini/.
- [43] Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, and Joseph Gonzalez. Memgpt: Towards llms as operating systems. ArXiv, abs/2310.08560, 2023. URL https://api.semanticscholar.org/CorpusID:263909014.
- [44] Md Rizwan Parvez. Chain of evidences and evidence to generate: Prompting for context grounded and retrieval augmented reasoning. In Weijia Shi, Wenhao Yu, Akari Asai, Meng Jiang, Greg Durrett, Hannaneh Hajishirzi, and Luke Zettlemoyer, editors, Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 230–245, Albuquerque, New Mexico, USA, May 2025. Association for Computational Linguistics. ISBN 979-8-89176-229-9. doi: 10.18653/v1/2025.knowledgenlp-1.21. URL https://aclanthology.org/2025.knowledgenlp-1.21/.
- [45] Md Rizwan Parvez and Kai-Wei Chang. Evaluating the values of sources in transfer learning. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5084–5116, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.naacl-main.402. URL https://aclanthology.org/2021.naacl-main.402/.
- [46] Md Rizwan Parvez, Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. Retrieval augmented code generation and summarization. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2719–2734, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-emnlp.232. URL https://aclanthology.org/2021.findings-emnlp.232/.
- [47] Md Rizwan Parvez, Jianfeng Chi, Wasi Uddin Ahmad, Yuan Tian, and Kai-Wei Chang. Retrieval enhanced data augmentation for question answering on privacy policies. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 201–210, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.eacl-main.16. URL https://aclanthology.org/2023.eacl-main.16/.
- [48] Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, 2024. URL https://qwenlm.github.io/blog/qwq-32b-preview/.
- [49] Qwen Team. QwQ-32B: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/.
- [50] Salman Rahman, Sheriff Issaka, Ashima Suvarna, Genglin Liu, James Shiffer, Jaeyoung Lee, Md Rizwan Parvez, Hamid Palangi, Shi Feng, Nanyun Peng, et al. Ai debate aids assessment of controversial claims. arXiv preprint arXiv:2506.02175, 2025.
- [51] Mobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki, Arsalan Gundroo, Bingqing Wang, Rakesh Menon, Md Parvez, and Zhe Feng. DelucionQA: Detecting hallucinations in domain-specific question answering. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 822–835, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-emnlp.59. URL https://aclanthology.org/2023.findings-emnlp.59/.
- [52] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
- [53] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
- [54] Vighnesh Subramaniam, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=JtGPIZpOrz.
- [55] Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test-time learning with adaptive memory. arXiv preprint arXiv:2504.07952, 2025.
- [56] Qwen Team. Qwen3, April 2025. URL https://qwenlm.github.io/blog/qwen3/.
- [57] Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. Openmathinstruct-1: A 1.8 million math instruction tuning dataset. arXiv preprint arXiv:2402.10176, 2024.
- [58] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
- [59] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Huai hsin Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. ArXiv, abs/2203.11171, 2022.
- [60] Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, and Graham Neubig. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377, 2023.
- [61] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
- [62] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155, 3(4), 2023.
- [63] xAI. Grok 3 beta — the age of reasoning agents, 2025. URL https://x.ai/news/grok-3.
- [64] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023a.
- [65] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023b.
- [66] Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil L Mui, Haiquan Wang, Caiming Xiong, and Silvio Savarese. Retroformer: Retrospective large language agents with policy gradient optimization. ArXiv, abs/2308.02151, 2023c. URL https://api.semanticscholar.org/CorpusID:260611249.
- [67] Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. Acecoder: Acing coder rl via automated test-case synthesis. arXiv preprint arXiv:2502.01718, 2025.
- [68] Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, and Huan Zhang. Alphaone: Reasoning models thinking slow and fast at test time. 2025. URL https://arxiv.org/abs/2505.24863.
- [69] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Gaetan Lin, Y. Liu, and Gao Huang. Expel: Llm agents are experiential learners. In AAAI Conference on Artificial Intelligence, 2023. URL https://api.semanticscholar.org/CorpusID:261048772.
- [70] Li Zhong, Zilong Wang, and Jingbo Shang. Ldb: A large language model debugger via verifying runtime execution step-by-step. arXiv preprint arXiv:2402.16906, 2024.
Appendix A Lists of Prompts
This section provides the list of prompts for planning, dynamic, judge, verifier and reasoning segmentation we have used in the experimental period. These are crucial to ensure the reproducibility [23] of the framework olver.
A.1 Planner Agent
A.2 Dynamic Agent
A.3 Judge Agent
A.4 Verifier Agent
A.5 Reasoning Segmentation
Appendix B Qualitative Examples
The subsequent section presents some qualitative examples–agentic interaction breakdown on how olver can approach and solve a problem of math and coding tasks when it encounters.
Appendix C External Retrieval Corpus for Code
As shown in Figure 11, our coding external retrieval corpus has 156 fine grained algorithmic tutorials with relevant problems along with their solutions collections from 10 different categories (algebrea, data structure, dynamic programming, string processing, linear algebrea, combinatorics, numerical methods, geometry, graphs and miscellaneous).
Appendix D Additional Analysis
This section presents our multiple runs’ results which we tested and list of roles which LLMs selected during the experiments.
D.1 Patterns in olver Reasoning Traces
We employ the QWQ-32B model for the initial segmentation of the chain of thought, guided by the first prompt illustrated in Appendix A.5. Subsequently, for segments that remain unannotated, a second segmentation pass is performed using the second prompt depicted in Appendix A.5. Reasoning patterns are identified by extracting content enclosed within the <pattern> tags. Verification and reasoning patterns are combined into a unified self-evaluation category. If the model assigns multiple patterns to a single segment, that segment is excluded due to ambiguity in pattern classification. For each generated output, we calculate the proportion of occurrences of each pattern relative to the total patterns present, resulting in a frequency vector representing pattern distribution per generation. In examining the relationship between pattern usage and problem difficulty, we compute the mean frequencies separately for correct and incorrect generations and assess significance through a t-test. To evaluate pattern prevalence on a per-problem basis, a binary matrix is constructed where rows correspond to problems and entries indicate whether a pattern is more common in correct (1) or incorrect (0) solutions. The statistical significance of these findings is evaluated using a binomial test.
Reasoning Pattern Correct Solutions Incorrect Solutions Easy Medium Medium High Easy Medium Medium High (a) OpenCodeReasoning Self-Evaluation () 0.36 0.37 New Approach () 0.16 0.20 0.20 0.23 0.16 0.22 0.22 0.25 Problem Rephrasing () 0.21 0.20 0.20 0.20 0.21 0.22 0.22 0.23 Subgoal Setup () 0.13 0.12 0.12 0.10 0.10 0.10 (b) olver Self-Evaluation () 0.35 0.37 New Approach () 0.18 0.21 0.21 0.24 0.17 0.24 0.24 0.26 Problem Rephrasing () 0.20 0.17 0.18 0.18 0.23 0.24 0.24 0.25 Subgoal Setup () 0.14 0.13 0.13 0.11 0.11 0.11
D.2 Performance Variance Statistics
In this experiment on the variance of olver performance, we tested olver against multiple runs (16 for AIME ’24 and 32 for AIME ’25 and LiveCodeBench) in AIME and LiveCodeBench dataset. Results shows in Table 4 that it has small scale performance change with multiple runs which is a strong sign on the robustness of olver.
| Model | Appr. | AIME ’24 | AIME ’25 | LiveCodeBench (v5) |
|---|---|---|---|---|
| o3-mini-medium | Xolver (-) | 87.2 1.2 | 85.1 1.3 | 79.6 1.0 |
| Xolver (+) | 93.8 0.3 | 89.4 0.7 | 87.3 0.4 | |
| QWQ-32B | Xolver (-) | 89.9 0.8 | 79.5 1.1 | 76.2 0.9 |
| Xolver (+) | 93.6 0.2 | 82.7 0.8 | 79.2 0.5 | |
| o3-mini-high | Xolver (+) | 94.4 0.6 | 93.7 0.5 | 91.6 0.3 |
D.3 Impact of Data-Shuffling in olver (+) Performance
During this experiment on the impact of shuffling data on olver performance, we randomly shuffled the test instances and conducted the experiment with 5 runs. Results shows in Table 5 that olver has limited performance change (STD ) with shuffling data–a strong sign on the robustness of the framework.
| Mean STD | (With only 5 Runs, all STD ) | ||||
|---|---|---|---|---|---|
| Model | GSM8K | AIME ’24 | AIME ’25 | MATH-500 | LiveCodeBench (v5) |
| o3-mini-medium | 97.6 1.3 | 92.2 0.4 | 91.0 0.3 | 98.3 0.6 | 90.9 1.1 |
| QWQ-32B | 97.2 0.6 | 93.7 0.5 | 82.7 2.0 | 95.1 0.6 | 83.6 1.6 |
D.4 List of Roles of Selected by Dynamic Agents
Table 6 shows some selected specialized roles by the dynamic agents while testing on coding and math tasks along with their most frequently selected roles.
| (a) Specialized Roles | |
| Math | Coding |
| Problem Analyzer | Problem Analyzer |
| Mathematical Modeler | Algorithm Designer |
| Algorithm Designer | Solution Architect |
| Numerical Analyst | Implementation Specialist |
| Symbolic Solver | Data Structure Specialist |
| Mathematician | Optimization Engineer |
| Computational Tools Specialist | Unit Tester |
| Debugging Expert | |
| Programmer | |
| Debugging Expert | |
| Code Reviewer | |
| (b) Most Frequent Roles | |
| Math | Coding |
| Mathematical Modeler | Algorithm Designer |
| Numerical Analyst | Implementation Specialist |
| Symbolic Solver | Data Structure Specialist |
| Mathematician | Programmer |
| Computational Tools Specialist | Optimization Engineer |