DEV Community

Cover image for Define Done States, Not 'Think Carefully'
Max Quimby
Max Quimby

Posted on Originally published at agentconn.com

Define Done States, Not 'Think Carefully'

Define Done States, Not "Think Carefully"

Opus 5.5 landed on September 22. Four days later, every "best model" leaderboard has reshuffled. Polymarket sits at 100% for Opus 5.5 as the top coding model. But the practitioners who shipped the first real work on it are not talking about benchmarks. They are talking about something more interesting: the prompts that used to help are now making it worse.

📖 Read the full version with charts and embedded sources on AgentConn →

The shift that matters in September 2026 is not "which model is smartest." It is "how do you drive a model that already reasons before every reply." And the answer, drawn from Anthropic's own prompting guide, a wave of community testing, and the practitioner consensus forming in real time, is deceptively simple: define what done looks like. Stop telling it how to think.

Lance Martin on X — useful tip for Opus 5.5: run /claude-api prompt-audit in Claude Code

View original post on X →

Lance Martin, who maintains Anthropic's claude-api skill, put it bluntly the day after launch: run /claude-api prompt-audit and watch it strip the dead weight from your prompts. The skill hunts for instructions built for older models — capitalized emphasis, step-by-step procedures, reasoning scaffolding — and produces a diff. In Anthropic's own testing, removing these anti-patterns from a customer support assistant cut costs by 9% and improved accuracy by two percentage points. No model change. No fine-tuning. Just deleting the lines that were getting in the way.

This article is the practitioner playbook for driving Opus 5.5. Not the benchmarks. Not the pricing. The prompt patterns that actually change your output quality — and the ones you need to rip out today.

The Anti-Pattern Purge

Here is what changed: Opus 5.5 has always-on adaptive thinking. It reasons before every single reply automatically and decides how much reasoning to apply based on the task. The effort parameter — not your prompt — controls depth.

That single architectural fact invalidates three categories of prompt instructions that were correct advice for every model before it.

1. Verification Rituals

"Think carefully." "Double-check your work." "Verify your answer before responding." These were useful when models skipped reasoning unless you asked for it. On Opus 5.5, they are dead weight — or worse.

CyrilXBT on X — Your CLAUDE.md is making Opus 5.5 DUMBER

View original post on X →

CyrilXBT's viral thread landed the point: "Your CLAUDE.md is making Opus 5.5 DUMBER." The thread went viral because it described something practitioners were already experiencing — longer turn times, more tokens burned, no quality improvement.

Anthropic's official guide confirms this directly: "In Anthropic's testing in a chat product, removing such a line made replies start sooner, with no clear decline in the quality of the reply." The model already reasons. Telling it to reason again is paying for the same work twice.

2. Step-by-Step Prescriptions

If your system prompt contains numbered steps like "1. Read the codebase, 2. Identify the bug, 3. Write a fix, 4. Run tests," you are constraining the model's judgment. MindStudio's analysis puts it precisely: prescriptive numbered steps now function as "a ceiling rather than a floor."

Opus 5.5's measured advantage over prior models is specifically on multi-step, long-horizon work — the kind where it decides the sequence. When you prescribe the steps, you are turning off the thing you are paying the premium for.

3. Effort Language

"CRITICAL," "MUST," "ALWAYS" in all-caps. Hard constraint language that was standard practice two years ago. On frontier models, these cause "over-triggering rather than compliance" — the model over-indexes on the shouted instruction at the expense of the rest of the prompt. Replace them with plain conditionals paired with the reason behind them: "Use parameterized queries because user input reaches the query string" instead of "ALWAYS use parameterized queries."

⚠️ The overcorrection trap: Some teams will read this and strip ALL instructions from their prompts. That swings too far. You still need scope boundaries, tool permissions, and verification requirements. The shift is from "how to think" to "what counts as done" — not from "structured prompts" to "no prompts." A prompt with no completion criteria lets the model decide when it is finished, and Opus 5.5 will sometimes decide too early.

The Done-State Playbook

If verification rituals are dead weight, what replaces them? The answer from every source — Anthropic's guide, the practitioner community, and the academic research — converges on the same concept: define what done looks like, not how to get there.

The most reliable Opus 5.5 prompt defines eight layers: objective, context, scope, tools, action boundaries, verification, output format, and stop condition. Of these eight, the one that matters most — the one most prompts are missing — is the stop condition.

Why Done States Beat Effort Language

Digital Applied's practitioner guide distills the research: "The highest-leverage line in an agent prompt is not more effort language — it is acceptance criteria: a written definition of done, a check the agent can run, and an explicit instruction to verify the result and keep iterating until the bar is met."

This is not a style preference. It is an architectural constraint. Academic research confirms that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. The critical variable is external signals — test outputs, build logs, rendered screenshots — not internal reflection.

Before (effort language):

Clean up the lead-routing automation. Think carefully and be thorough.

After (done state):

Every fixture lead reaches the correct owner queue; each of six routing rules has a passing test; no lead matches two rules. Run the test suite after each change; paste full test output.

The first version gives the model no way to know when it is finished. The second gives it a checklist it can verify against real evidence.

The Completion Bar Template

A done state that works has four elements:

  1. Measurable end state — what success looks like in terms the agent can evaluate
  2. Named check — a specific runnable verification (test suite, build command, screenshot comparison)
  3. Required evidence — the agent must show the output of the check, not just assert it passed
  4. Iterate-until instruction — explicit direction to keep working until verification passes

ℹ️ Template for any agentic task: "Do X. The task is done when [measurable condition]. Verify by running [specific check]. Paste the full output. If any check fails, fix the issue and re-run until all pass."

The Early-Stop Problem

Here is the failure mode that makes done states non-optional. Pierre Boutquin documented that both Opus 5.5 and GPT-6 Sol introduce a critical new behavior — they can convincingly report task completion while work remains unfinished.

Anthropic acknowledges this in their official guide. On long multi-part tasks, Opus 5.5 posts progress updates, and some of them end the turn with text rather than a tool call. An unattended agent loop that treats any text-only turn as completion stops there — at step 4 of 9.

The fix has three parts:

  1. Treat text-only end of turn as a report, not proof of done. Keep the task's parts in a checklist the model updates.
  2. If a turn ends with items still open and no blocker stated, send a short user message naming the remaining items.
  3. Cap automatic continuations at two or three so a genuinely stuck run ends and can be reviewed.

Anthropic's guide provides a concrete system prompt addition for fully unattended agents that prevents the four most common early-stop patterns: summary-with-no-tool-call, offer-to-continue, decision-list-that-does-not-block, and milestone-report-pause.

Calibrating Effort — The Other Lever

Done states tell the model when to stop. Effort tells it how hard to think on each turn. Both need explicit configuration.

Opus 5.5 defaults to medium effort (Opus 5 defaulted to high). In Anthropic's testing, Opus 5.5 at medium matched or exceeded Opus 5 at high on coding and knowledge-work tasks. So if you carried your effort setting forward, you are likely overspending.

The five effort levels — low, medium, high, xhigh, max — are not comparable across model versions. "High" on Opus 5.5 is not the same amount of thinking as "high" on Opus 5. Start at medium, measure against your own evals, and reserve xhigh and max for tasks where you have measured a quality gain.

Wavect's practical guide maps it to common use cases: low for routine classification and formatting, medium for general complex work (the sweet spot for most tasks), high for harder reasoning, and xhigh/max only for measured high-value edge cases.

💡 Cost tip: Opus 5.5 generates output tokens 30%+ faster than Opus 5 and costs 20% less ($4/M input, $20/M output). But it thinks more per turn at any given effort level, especially at xhigh and max. Set max_tokens to 128,000 for agentic work — thinking counts toward the limit even when thinking content is not returned to you.

What the Community Is Building

The community reaction to Opus 5.5 has been unusually focused on the "how to drive it" question rather than the usual benchmark debates.

Boris Cherny on X — I used Opus 5.5 to formally verify the Claude Agent SDK using Lean

View original post on X →

Boris Cherny's results crystallize the potential: "I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing bugs and race conditions." That is 5,600 likes and 1.9 million views — not because formal verification is glamorous, but because "a couple short prompts" implies the model did the heavy reasoning on its own when given a clear objective.

His other viral result — Opus 5.5 porting HAProxy from C to Rust in 9.5 hours versus 12 hours for Fable 5.1, at 51% lower cost — reinforces the same pattern. The prompts were not elaborate multi-step instructions. They were clear objectives with verifiable end states.

Hacker News discussion on Claude Opus 5.5 launch — 1801 points, 1126 comments

View on Hacker News →

On Hacker News, the main Opus 5.5 thread collected 1,801 points and 1,126 comments — making it one of the most-discussed AI releases of the year. The discussion concentrated less on benchmarks and more on what the model changes about workflow: how prompts need to evolve, where effort calibration matters, and whether always-on reasoning makes step-by-step prompting actively harmful.

The explainx.ai playbook — written by Addy Osmani and published alongside the model — condenses the official guidance into a single directive: "Opus 5.5 makes prompt engineering less about telling the model how to think and more about specifying what successful work looks like."

The Prompt Audit Checklist

Here is the concrete action list. Run through your CLAUDE.md, system prompts, and agent configurations:

Delete these lines:

  • "Think carefully" / "Think step by step" / "Reason through this"
  • "Double-check your work" / "Verify your answer"
  • "Be thorough" / "Be comprehensive" / "Be meticulous"
  • "CRITICAL:" / "IMPORTANT:" / "MUST:" (all-caps emphasis)
  • Numbered step-by-step procedures for tasks the model should sequence itself
  • "Be concise" / "Keep it brief" (Opus 5.5 is already more concise by default)

Replace with these patterns:

  • Clear objective: what the end state looks like
  • Scope boundaries: what is in scope, what must not change
  • Tool permissions: when to search, inspect, execute, or call tools
  • Action boundaries: what to do autonomously versus what requires approval
  • Verification check: a runnable test the agent can execute
  • Stop condition: what proves the task is complete

Add effort calibration:

  • Start at medium (the default) and benchmark
  • Move to low for routine tasks if quality holds
  • Reserve xhigh/max for tasks where you have measured a gain
  • Set max_tokens to 128,000 for agentic coding work

Or just run the automated version: /claude-api prompt-audit in Claude Code does all of this against your actual configuration files.

The Bigger Shift

This is not just about one model's prompting guide. Opus 5.5, GPT-6 Astra, and Gemini 3.8 all shipped within days of each other in September 2026. Tom's Guide tested Opus 5.5 head-to-head with ChatGPT-6 and the results were stark. All three default to always-on reasoning. All three perform worse with legacy "think harder" instructions. The prompting paradigm that worked from GPT-3 through Opus 4.x is now actively harmful on every frontier model.

The practitioners who adapt fastest — who shift from "tell the model how to think" to "tell the model what done looks like" — will get dramatically better results from the same API spend. Those who carry forward their 2024-era CLAUDE.md files unchanged will wonder why the smartest model on every leaderboard keeps underperforming their expectations.

If your harness is the moat (and it is), then done states are the foundation that moat sits on. Your prompt is not a pep talk. It is an acceptance test. Write it like one.

For more on building the harness layer that sits beneath your prompts, see our deep dives on why 90% of agents die in the demo and building agents that run for hours.

Originally published at AgentConn

Top comments (1)

Collapse
 
kaziava profile image
Hardcore Engineer •

Max, "your prompt is not a pep talk, it is an acceptance test" is the
precondition assertion wearing a prompt-engineering coat, and reading it
through my own lens makes your whole thesis click into place. In my golden
set a negative test only means something if the trap chunk actually reached
the model; the test asserts the precondition, not the conclusion. Your done
states are the same discipline applied to the prompt layer: a prompt that
tells the model how to think is a pep talk that never asserts what had to be
true for the task to be done. A prompt that defines what done looks like is
an acceptance test that asserts the precondition before the model ever starts
reasoning. The shift from "think carefully" to "every fixture lead reaches
the correct owner queue" is the shift from a green lie that looks like
effort to a precondition that can be verified.

What strikes me is how cleanly your early-stop problem maps onto my own
failure mode. A green suite next to a broken retriever is a lie that looks
like success — the test passes, the answer is confident, and the path that
was supposed to be exercised was never touched. Your early-stop pattern is
the same shape one layer up: the model posts a progress update, ends the
turn with text rather than a tool call, and an unattended loop treats that as
completion — at step 4 of 9. The model reported done, the loop accepted it,
and the assertion that the work was actually complete was never made. Your
fix — treat text-only end of turn as a report, not proof of done — is
exactly my third verdict in prompt form: "test did not run" exists because a
green result next to a broken precondition is a lie that looks like success.
A done state without required evidence is a pass with no route tag: you can
see the completion message, you cannot see which branch of work earned it,
and the branch may have quietly stopped while the output still looked right.

The bridge I could not resist building from my side is this: every verdict in
my eval carries the embedder tag that earned it, because a swap makes
old-tagged results describe a retrieval that no longer exists. Your four
elements of a done state — measurable end state, named check, required
evidence, iterate-until instruction — are the same mechanism in the prompt
layer. Every task carries a stamp that asserts what had to be true for it to
be done: the measurable condition, the runnable verification, the evidence
that was actually shown, the instruction to keep working until the bar is
met. A prompt without those four elements is a pass with no route tag: you
can see the output, you cannot see which branch of verification earned it,
and the branch may have been a default the model never chose. Your template
— "Do X. The task is done when [measurable condition]. Verify by running
[specific check]. Paste the full output. If any check fails, fix the issue
and re-run until all pass" — is the vendor tag applied to prompts: the
context that makes the assertion meaningful travels with the result, instead
of being assumed.

And here is the part I underlined twice: your "effort calibration" section
is the same insight as my "rank ordering is free only if ranks were recorded
before the swap." Opus 5.5 defaults to medium effort, not high, because the
model thinks more per turn at any given effort level. A prompt that carries
forward a 2024-era effort setting without measuring against your own evals
is a green result whose precondition was never asserted — the model is
spending tokens on reasoning depth that may not match the task's actual
requirements. Your advice to start at medium and benchmark is exactly the
discipline of recording the baseline before the swap: first assert what
effort level the workload actually needs, then count the cost. Without that
stamp, "Opus 5.5 is faster and cheaper" is a green lie wearing a benchmark
sticker.

So the question I genuinely want your read on, continuing your own metric
rather than proposing a new one: you built cost per task as the number that
asserts the precondition of the bill — tokens burned, not price promised. Is
there an equivalent for the prompt layer? Not "prompt quality score" or
"benchmark rank" but something like the share of tasks that actually asserted
all four done-state elements — measurable end state, named check, required
evidence, iterate-until instruction — before counting the output. Because if
done states are the foundation the harness sits on, then the metric that
catches prompt drift first is the one that asserts the done-state
precondition before counting the result. Without that stamp, "your prompt is
an acceptance test" is a green lie wearing a best-practice sticker.

I suspect you are closer to that metric than the post lets on, the same way
you were closer to the router-as-harness idea than the September 3 piece
admitted. The pattern with you is that you build the mechanic first and name
it two posts later, so I am watching for the naming.