Three tasks ran. Two passed. The tasks that passed averaged $0.006 each, but the run cost $0.009 per delivered result. The extra $0.003 per result came from a task that ran twice and still failed.
I built a trace replay in AICostBudget to see where that spend goes. Here's the small synthetic example I used to check the math:
| Task | What happened | Model cost |
|---|---|---|
| A | Passed on the second attempt | $0.006 |
| B | Passed on one call marked as a fallback | $0.006 |
| C | Failed after two attempts | $0.006 |
| Total | 2 successes across 5 model calls | $0.018 |
All five calls use GPT-5.4 mini. At $0.75 per million input tokens and $4.50 per million output tokens, four calls cost $0.003 each (1,000 input + 500 output tokens). The fallback call used twice as many tokens, so it cost $0.006.
Cost per successful task = $0.018 total spend ÷ 2 successes = $0.009. Looking only at A and B gives $0.006, which leaves a third of the run's spend out of the calculation.
The two retries cost $0.003 each. A's retry got the task through; C's didn't. C's second call shows up under both Retry tax and Failed-work cost. Those categories overlap, so adding them would count that call twice.
To reproduce it, open the trace cost report and click Load synthetic example. You should see $0.006 in failed-work cost and $0.009 per successful task. You can also import Agent Cost Evidence JSON v0.1 or a supported OpenInference / OTLP JSON export with a separate task outcomes file. The trace stays in your browser, and the estimate uses current standard rates.
For your own workflow, decide what counts as a pass before comparing retry rules. For invoice extraction, I'd count the invoice being accepted downstream. Then run both rules on the same tasks and compare how many passed and what each success cost.
If you're already measuring this, where do you record whether a task passed: in the trace itself, or in a separate eval or job record?
Top comments (0)