A Cached Agent Answer Belongs to a Particular Context

A Cache Entry Needs a Reuse Contract

BOTIE

Reusing an earlier response can save a network request and the work behind it. Web caches also exposed the uncomfortable question: when is the earlier answer still an answer to this request? A page can change, and the same address can produce different responses for different callers.

The 1999 HTTP/1.1 specification documented rules for caching, including freshness and matching the request to the stored response. Later specifications refined those rules; RFC 9111 brought the caching specification forward in 2022. A cache key identifies a possible match, while reuse rules decide whether that match may actually be served. An address by itself cannot settle both questions.

An agent response has further dependencies, such as retrieved evidence and model configuration. We will identify those dependencies before deciding whether an old answer is reusable. This is an application-level contract. HTTP headers alone cannot decide whether the current user may see a previously generated answer.

Sources for this history

The Same Question Came from a Different Team

Two technicians ask Botie the same question. Their teams can read different manuals. A cache keyed only by the question returns the first team’s answer to the second. Faster inference has created a data boundary failure.

A response belongs to the context that produced it. We need a reuse key that includes the relevant authority and evidence versions, then permission checks at the point of use.

This is a reuse-key check. Permissions, expiry and sensitive response storage require separate enforcement.

Step 1 of 1

Identical wording can have different authority

The evidence snapshot and tenant are both part of the context.

Neither later request is equivalent to the first for this cache policy.
Request	Tenant	Evidence
First	team-A	manual-v1
Second	team-B	manual-v1
Third	team-A	manual-v2

First
Tenant
team-A
Evidence
manual-v1
Second
Tenant
team-B
Evidence
manual-v1
Third
Tenant
team-A
Evidence
manual-v2

Neither later request is equivalent to the first for this cache policy.

List the dependencies of the answer

The full declared dependency record determines equivalence.

Scope the cached response

The same wording reaches Botie under a different context.

BOTIE

The same wording reaches Botie under a different context.

Choose an example

Cached question

Current question

Cached context record

tenant

evidence

Current context record

tenant

evidence

Whether the full reuse context matches

true

Follow the operations

1. Cached question
```
why?
```
2. Current question
```
why?
```
3. Compare complete values
```
true
```
4. Cached context record
```
{
  "tenant": "A",
  "evidence": "v1"
}
```
5. Current context record
```
{
  "tenant": "A",
  "evidence": "v1"
}
```
6. Compare complete values
```
true
```
7. Require both checks
```
true
```
8. Result
```
true
```

This is a reuse-key check. Permissions, expiry and sensitive response storage require separate enforcement.

1. List the dependencies of the answer

The question text is only one input. Relevant context can include the customer organisation, its current permissions, the evidence snapshot, model version, prompt version and generation settings. The organisation is often called the tenant. Keeping it in the context prevents identical questions from making two customers’ answers interchangeable. A conversation-dependent answer also depends on the relevant history.

Do not put secrets into the key itself or assume a hashed key makes sensitive metadata public-safe.

A semantic similarity score is not evidence that two requests have the same authorisation or evidence.

2. Compare complete context, not one convenient field

In the workshop, each context record contains the fields required by a declared simplified contract. Compare those records and the question separately, then require both matches.

same question = true
same context = false
reuse = true AND false = false

Production reuse also needs expiry and a fresh permission decision. A revoked permission must not remain valid because the cached context is old.

Do not concatenate ambiguous fields

A structured encoding retains the boundary between question and tenant.

Python

import json
print(json.dumps(["ab","c"],separators=(",", ":")))
print(json.dumps(["a","bc"],separators=(",", ":")))

Output

["ab","c"]
["a","bc"]

Plain concatenation would turn both inputs into abc. The structured representation keeps them distinct.

Check yourself

What makes these requests equivalent?

Question 1 of 2

May a question-only cache safely assume equivalence?

Use the information below.

Same question, tenant A versus tenant B.

3. Version changes should invalidate the result

A corrected manual changes the evidence. A revised prompt changes the procedure. A model replacement can change outputs. Keeping these identities in the key makes invalidation explicit.

For nondeterministic generation, reuse intentionally returns an earlier sample. Decide whether that behaviour is acceptable for the application.

Caching an embedding, a retrieved candidate set and a final answer are different choices with different dependencies.

4. Test across boundaries

Use the same wording across tenants, change only the evidence version, and then change only the model version. Each should produce a different key under this contract. Equivalent records with different key order should match.

Keep a bounded cache and remove expired entries. Correct identity does not itself limit storage growth.

The challenge hashes a declared tuple of fields. It performs no retrieval and is not an authorisation service.