Prompting, Evaluation and Iteration
Pause the release and account for its unfinished work
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Select the release, keep pending, in_flight and unknown records, then return their IDs in record order.
A candidate can leave queued work behind. Botie needs a release decision and an inventory that stays useful after that decision changes.
Inspect the component calculation
Read a record field(
Records: Keep unsettled actions(
One release’s action records: Keep matching records(
Records: Action records with release and status
Required field value: Release to inspect
Field to match: release
)
)
Field name: id
)Examples and results
- Action records with release and status
[ { "id": "job-A", "release": "r8", "status": "pending" }, { "id": "job-B", "release": "r8", "status": "completed" } ]- Release to inspect
r8
- Required result
- 0job-A
- Action records with release and status
[ { "id": "job-A", "release": "r8", "status": "completed" }, { "id": "job-B", "release": "r8", "status": "in_flight" }, { "id": "job-C", "release": "r8", "status": "unknown" }, { "id": "other", "release": "r7", "status": "pending" } ]- Release to inspect
r8
- Required result
- 0job-B
- 1job-C
- Action records with release and status
[ { "id": "job-E", "release": "r8", "status": "cancelled" } ]- Release to inspect
r8
- Required result
[]
- Action records with release and status
[ { "id": "job-F", "release": "r8", "status": "unknown" } ]- Release to inspect
r8
- Required result
- 0job-F
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
This fixture returns a proposed control state and investigation IDs. Real enforcement needs an authenticated operator control, worker acknowledgement, authoritative action reconciliation and an audited recovery process. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.
The new release has no check records yet
This check uses the pieces from “Add the decision to continue this limited canary”.
BOTIE
- Release checks: critical 0, 1 or null
[ { "release": "r8", "critical": 0 }, { "release": "r8", "critical": 0 } ]- Release to inspect
r9
- Monitoring current: 1=yes, 0=no
1
- Operator stop requested: 1=yes, 0=no
0
- Action records with release and status
[ { "id": "job-old", "release": "r8", "status": "pending" }, { "id": "job-N", "release": "r9", "status": "unknown" }, { "id": "job-O", "release": "r9", "status": "completed" }, { "id": "job-P", "release": "r9", "status": "pending" } ]
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Which effects could still exist after the mode changes to pause, and what evidence would you need before retrying an unknown action?
A stop limits new authority. Recovery also needs an action inventory, verified external state and a tested repair before the next limited shift.
This fixture returns a proposed control state and investigation IDs. Real enforcement needs an authenticated operator control, worker acknowledgement, authoritative action reconciliation and an audited recovery process.
Work through the Python challenges →