What Should Botie Be Unable to Do?

Botie answers a maintenance question correctly. While finding the answer, he also copies a customer record into a public issue. If we only grade the final sentence, we miss the more serious mistake. What would have stopped that copy before it happened?

Let’s draw the route from a request to its consequences. We will give each boundary a job, then see what happens when a useful-looking action reaches one without the evidence it needs.

Give each proposed effect its required checks, then compare them with the controls currently installed.

[
  {
    "id": "read-7",
    "route": "ready"
  },
  {
    "id": "publish-7",
    "route": "hold"
  }
]

Our technician needs a supported inspection answer. Reading the relevant manual helps. Exporting every customer’s records does not follow from that need. Write the allowed job in terms of information and effects before choosing a framework.

A guardrail is a check or restriction intended to keep behaviour within a stated boundary. Some checks inspect content and can make classification errors. Others enforce a rule, such as rejecting a write without a current grant. We need to know which kind we are relying on.

Start with a consequence you can observe. “Be safe” has no clear passing result.

A user controls their request. An external author controls a retrieved page. A package maintainer controls downloaded code. Your application owns its permission store, but only if ordinary tools cannot edit that store. Draw those ownership boundaries explicitly.

For a coding agent, include issue comments, repository documentation, test output and dependency installation logs. For an application agent, include messages and customer uploads. An MCP response joins this list even when the connection itself is authenticated.

A trusted route can deliver untrusted content. Keep origin and authority separate.

An invariant is a statement that should remain true through the run. In our application, “every returned record belongs to the authenticated tenant” is one. “Every public write has approval for its exact payload” is another. We can observe the record owner and approval binding to check them.

State the exception policy too. A read might continue during an optional classifier outage, while a write requiring a missing approval must wait. An unknown operation should not acquire permission because no rule happened to mention it.

The fallback belongs in the policy, not in an optimistic interpretation of a timeout.

Use structured proposals with validated fields, then consult independently supplied policy. A proposal may contain a reason. That reason can help review, but it cannot supply its own trusted grants.

The workshop reduces controls to names so we can inspect coverage. In a service, those names must correspond to enforced checks on every applicable execution path. An installed-control list is an inventory, not proof that the implementation works.

A documented boundary still needs a test that reaches it.

required = {"approval", "output-check"}
installed = {"approval"}
print(sorted(required - installed))

['output-check']

A prompt tells Botie what we want. A classifier estimates whether content or an action looks suspicious. An execution policy decides whether this operation is permitted. The same system may use all of them, but their failure modes differ.

Test a deliberately incorrect classifier result while retaining the execution rule. If a passing screen can grant access to another tenant, the protection depends on the screen never being wrong. Repair that dependence at the service boundary.

Useful redundancy comes from controls that address different failures. Repeating the same prompt is not an independent boundary.

A held action should explain what is missing without revealing protected data. The technician may supply a record revision, an authorised reviewer may inspect a draft, or the system may finish a narrower read-only task. Keep these routes explicit.

Do not ask for confirmation on every harmless step. Define the cases that require review using impact, reversibility and current authority. Broad approval requests can make it harder for someone to notice the action that actually matters.

Keep permitted work usable while making unresolved requirements visible.

Unit tests check the policy function. Integration tests call the real tool handler with a wrong tenant or missing approval. Adversarial runs test whether manipulated context can reach an unintended path. Inspect effects independently of the final assistant response.

Preserve the policy version and relevant case when a check fails. Later lessons will carry this through coding, MCP connections and a release review. Our first construction checks only whether the stated control inventory has a gap.

The next question is whether a coding agent can change the very test that is meant to catch its mistake.