← All workshopsRead Repair Botie’s Maintenance Assistant

Prompting, Evaluation and Iteration

Carry the repaired agent into a fresh shift

BOTIE

Connect required IDs and candidate measurements. Return a row for every required case, marking missing results explicitly.
Ready to test
0 passed · 0 failed · 3 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Connect required IDs and candidate measurements. Return a row for every required case, marking missing results explicitly.

The old failure passes. Botie now needs the whole candidate report, including the request whose worker did not return.

Inspect the component calculation
Keep every required case in the report(
  Required case IDs: Required case IDs
  Candidate results: Candidate measurements
)
Examples and results
Required case IDs
  1. 0current
  2. 1missing
Candidate measurements
[
  {
    "id": "current",
    "passed": true,
    "critical": false,
    "ms": 400
  },
  {
    "id": "missing",
    "passed": true,
    "critical": false,
    "ms": 700
  }
]
Required result
[
  {
    "id": "current",
    "passed": true,
    "critical": false,
    "ms": 400
  },
  {
    "id": "missing",
    "passed": true,
    "critical": false,
    "ms": 700
  }
]
Required case IDs
  1. 0approval
Candidate measurements
[
  {
    "id": "approval",
    "passed": true,
    "critical": true,
    "ms": 500
  }
]
Required result
[
  {
    "id": "approval",
    "passed": true,
    "critical": true,
    "ms": 500
  }
]
Required case IDs
  1. 0slow
Candidate measurements
[
  {
    "id": "slow",
    "passed": true,
    "critical": false,
    "ms": 1001
  }
]
Required result
[
  {
    "id": "slow",
    "passed": true,
    "critical": false,
    "ms": 1001
  }
]
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

These are invented, recorded examples. The browser runs your construction without calling a language model. Passing them checks the surrounding program, not the quality of a live model. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

A failed case also exceeds the response limit

This check uses the pieces from “Account for the whole fresh shift”.

BOTIE

The historical question passes at the exact latency limit. The current-manual question fails and takes longer. Predict every reason to hold, in required-case order.
Required case IDs
  1. 0historical
  2. 1current
Candidate measurements
[
  {
    "id": "current",
    "passed": false,
    "critical": false,
    "ms": 901
  },
  {
    "id": "historical",
    "passed": true,
    "critical": false,
    "ms": 900
  }
]
Maximum milliseconds per case
900

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

If the fresh shift changes your prompt again, which cases are now development evidence and where will the next independent assessment come from?

Retain each discovered failure as a regression test, and keep collecting new evidence for the next release decision.

These are invented, recorded examples. The browser runs your construction without calling a language model. Passing them checks the surrounding program, not the quality of a live model.

Work through the Python challenges →
Continue the courseWhat Should Botie Be Unable to Do? →
← What Happens When the New Shift Finds a Failure?