Permissions are not prompts

The pattern

It shows up in almost every early agent deployment:

You are a helpful assistant. You have access to the customer database.
Never modify records belonging to enterprise customers.
Never run queries that affect more than 100 rows.

Those last two lines are doing security work. They cannot do security work.

Why not

A prompt is an input to a probabilistic system. It shapes behaviour; it does not constrain it. The constraint holds until something in the context makes it not hold — a user instruction that conflicts, an unusual phrasing, a long conversation where early context is compressed, or simply an unlucky sample.

This is not a robustness problem that will be engineered away by a better model. It is a category error. You cannot build an access-control boundary out of a suggestion, however firmly it is worded.

The test that makes this obvious: would you accept this control in a code review if a human were on the other side of it? "The engineer has full database access but we told them not to touch enterprise records" would not survive the sentence.

Why capable teams still do it

Not carelessness. Three real pressures:

• Prompt-level restrictions are fast. Enforcing a permission boundary means building the enforcement layer. A line of prompt takes thirty seconds.
• They mostly work. In testing, the model does follow the instruction, which produces evidence that the control is effective.
• The failure is rare and the blast radius is invisible until it is not.

That combination — cheap, apparently working, rarely failing — is a near-perfect recipe for shipping something unsound.

The alternative

The rule we hold to: the executing layer enforces the constraint, and the model is never asked to.

In practice:

1. An agent runs as an identity with a scoped grant, in exactly the way a service account does.
2. Tool calls are checked against that grant at call time, by code.
3. A call outside the grant fails and is logged. It does not get negotiated.
4. The prompt may also describe the constraint — that improves behaviour and reduces pointless failed calls — but nothing depends on it.

The cost is real. You need an identity model, a grant model, and enforcement at every tool boundary, before the interesting work starts. It is unglamorous infrastructure and it is the part that determines whether the system can be deployed anywhere that matters.

A useful heuristic

When reviewing an agent design, find every constraint and ask where it is enforced. If the answer is "in the prompt", it is not a constraint. It is a preference.