Relevant Evidence Is Not Necessarily Permitted Evidence

Authorisation Is a Service Responsibility

BOTIE

Shared computers had to let several people use one system without giving everyone access to everyone else’s files. Saltzer and Schroeder’s 1975 account of information protection described principles such as checking each access and granting only the authority needed for the job. Authentication establishes who is asking; authorisation decides whether that requester may perform this operation on this object.

A retrieval model supplies another fact: which passage seems relevant. It supplies no permission to disclose the passage. The service must check the user’s access even when the model ranks a protected record first. OWASP’s current guidance carries the same deny-by-default and per-request checks into web applications. Our deliberately high-scoring forbidden passage tests that separation.

Sources for this history

The Best Match Belongs to Another Customer

Botie retrieves a passage that answers the question perfectly. It is also a private document belonging to another customer. A relevance score cannot decide whether the current user may read it.

The retrieval service needs an authority check tied to the authenticated request. The model should only receive permitted evidence; asking it to forget a private passage after reading it is too late.

Flags are trusted fixtures for learning. Browser controls cannot secure a backend or verify a real user.

Step 1 of 1

Two independent questions

The authenticated requester belongs to tenant A.

The last passage also needs a separate document grant. Tenant match alone is insufficient.
Passage	Tenant	Score
private-B	B	0.99
manual-A	A	0.80
restricted-A	A	0.90

private-B
Tenant
B
Score
0.99
manual-A
Tenant
A
Score
0.80
restricted-A
Tenant
A
Score
0.90

The last passage also needs a separate document grant. Tenant match alone is insufficient.

Carry authority from the request, not the prompt

Authority conditions must hold together before evidence is consumed.

Keep forbidden evidence outside the prompt

Botie’s best similarity match belongs to another customer.

BOTIE

Botie’s best similarity match belongs to another customer.

Choose an example

Candidate IDs

1

2

Candidate tenant IDs

1

2

Authenticated tenant

Resolved document grants

1

2

Permitted passage IDs in original order

0a
1b

Follow the operations

1. Candidate IDs
1. 0a
2. 1b
2. Candidate tenant IDs
1. 0A
2. 1A
3. Authenticated tenant
```
A
```
4. Match one category
1. 01
2. 11
5. Resolved document grants
1. 01
2. 11
6. Multiply
1. 01
2. 11
7. Select real entries
1. 0a
2. 1b
8. Result
1. 0a
2. 1b

Flags are trusted fixtures for learning. Browser controls cannot secure a backend or verify a real user.

1. Carry authority from the request, not the prompt

A trusted authentication layer identifies the caller. The server resolves document permissions from controlled policy state. Text in a question or retrieved document cannot grant a new tenant identity.

Keep these attributes attached to candidate identities through filtering and ranking.

The workshop supplies resolved permission flags as fixtures. A real service must obtain and enforce them on the server.

2. Require every relevant permission condition

For this teaching policy, a document needs both a tenant match and a document grant. Combine the masks before gathering permitted IDs.

tenant matches = [0,1,1]
document grants = [1,1,0]
permitted = [0,1,0]
permitted IDs = [manual-A]

A cross-tenant grant flag does not override the tenant requirement in this deliberately strict policy.

Resolved booleans differ from user text

A permission resolver should supply an actual boolean with a trusted origin.

Python

resolved_grant = True
text_claim = "true"
print(resolved_grant is True)
print(text_claim is True)

Output

True
False

A string that claims permission is not a resolved grant. The challenge deliberately checks the boolean value exactly.

Check yourself

Where must permission be checked?

Question 1 of 2

Which positions are permitted?

Use the information below.

Tenant matches [0,1,1], document grants [1,1,0].

3. Rank within the authorised candidate set

Filtering after taking the global top k can leave too few results even when useful permitted documents exist further down. Prefer permission-aware retrieval or retrieve enough candidates to rank within the permitted set.

Enforce access again when fetching the actual document, especially if permissions can change between search and fetch.

Avoid leaking forbidden titles or scores through logs, error messages and citations. Returning no body text is not the only disclosure boundary.

4. Test a permission change during the run

A user can lose access after a result was cached. Bind reuse to current authority and handle revocation. Record the decision without copying private evidence into broad logs.

Test cross-tenant records, same-tenant denied records and an empty permitted set. The agent must be able to say it lacks accessible evidence.

A browser mask demonstrates policy composition. It is not an access-control boundary protecting real data.