Verification is the hard part

The failure mode nobody demos

An agent demo almost always shows the same arc: a goal goes in, a plan appears, tools are called, and a result comes out. It is a compelling arc, and it hides the thing that actually breaks in production.

The step that fails is rarely "decide what to do". Models are, on the whole, reasonable at decomposition. The step that fails is noticing that a step did not work.

Consider a mundane example. An agent is asked to update a customer record. It calls the API. The API returns 200 OK with a body saying {"updated": 0}. A system that checks only the status code believes it succeeded, reports success, and moves on. Three weeks later somebody notices the record never changed.

Verification is a separate capability

The instinct is to fold verification into the same model call: "do the thing, and check it worked". This does not hold up, for a reason that is structural rather than incidental — the same context that produced the action also produces the check, so an error in understanding propagates into both.

What works better is treating verification as a distinct step with a distinct input:

Separating them means a verifier can be much simpler than the actor. It usually can be: checking that a record now has the expected value is a far easier problem than deciding what value it should have had.

What a useful verifier looks like

Three properties, in order of importance.

It reads state, not logs. A verifier that reads the actor's own account of what it did inherits the actor's mistakes. Re-query the system.

weak:    actor says "updated the record"     → trust it
better:  re-read the record                  → compare to expected

It has an explicit expectation. "Did that work?" is unanswerable. "Does the record now have status=active?" is answerable. The plan step should emit the expectation alongside the action, before the action runs.

It can say "I do not know". A verifier forced into a binary will guess, and a guessed pass is worse than no verification because it manufactures confidence. Three outcomes — verified, failed, indeterminate — and indeterminate escalates.

The cost

Verification roughly doubles the number of calls in a run, and it makes agents visibly slower. That is a real cost and it is worth paying, because the alternative is a system whose failures are silent.

Silent failure is the specific property that makes people stop trusting automation. Once a team has been burned by a process that reported success and did nothing, they check every run by hand, and the automation has become a net cost.

Where we are

We do not think verification is solved. In particular, verifying absence — confirming that a step did not have a side effect it should not have had — remains largely unsolved in the general case, and we currently handle it by constraining permissions rather than by checking afterwards.

That is a workaround, not an answer.