DEV Community

Cover image for The 200 that is not a result: how to verify without fooling yourself
SAMUELFMEDEIROS
SAMUELFMEDEIROS

Posted on Edited on Originally published at skills.seu.pet

The 200 that is not a result: how to verify without fooling yourself

The most common way a verification lies is not by erroring: it is by going green.
These four checks cost seconds and prevent the “tested, all good” that tested nothing.

1. A 200 is not a result

An anti-bot page, a consent wall and a login redirect all return 200. Before parsing,
assert a marker you expect to see (the product name, a word from the title). If the marker is not
there, the body belongs to another page.

2. The count has to add up

The tool says total: 5 and you parsed 3. Do not report the 3 you have: re-fetch and
parse in code. A report with fewer items than reality is worse than no report, because it looks
complete.

3. Same host, another app

A hostname can point at two different applications (the last rule wins, with no error). Validate
by comparing the response body, not just the status: another app's endpoint can answer 200
in place of the one you expected.

4. Unverifiable is not verified

A command that does not exist, a service that is down, a full disk, a stuck machine: the absence
of proof is never proof of success. Write “unverifiable” and the reason, in one line —
instead of letting it look like a pass.

In a gate that becomes three states, not two: passed, failed (the only one that is
yours) and unverifiable (the machine's). Collapsing the third into “passed” makes the gate
approve without proving; collapsing it into “failed” makes the gate stop enforcing.

The test that catches this

Write the check so it fails when the verification does not happen — not one that passes by
omission. A check that only fails when the command runs and errors is measuring the command, not
the result.

The Agent That Does Not Lie — R$ 37, the 14 chapters on telling a real 200 from a decorative one: o-agente-que-nao-mente — R$ 37, instant delivery.

Next in this series (the one after this): https://dev.to/samuelfmedeiros/the-agent-said-done-and-produced-nothing-verifying-is-not-producing-59hn

See the AI Test Loop skill
See all skills

Want this wired into your project? Guided setup from
R$ 1.500, with the first cycle run together.

← all skills

Top comments (0)