Evaluating and Improving Models
Route uncertain inspections without losing them
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Return a mask for confidence at or above the supplied acceptance threshold.
Botie can ask for help, but a handoff changes both the accepted sample and the review queue.
Inspect the component calculation
Accept scores at the threshold( Scores in [0, 1]: Confidence in predicted class Threshold: Acceptance threshold )
Examples and results
- Confidence in predicted class
- 00.95
- 10.8
- 20.6
- Acceptance threshold
0.75
- Required result
- 01
- 11
- 20
- Confidence in predicted class
- 00.4
- 10.5
- Acceptance threshold
0.9
- Required result
- 00
- 10
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
Observed confidence is not a correctness guarantee. The fixture does not simulate reviewer outcomes or queues. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.
Every inspection reaches the acceptance threshold
This check uses the pieces from “Keep the review queue in the report”.
BOTIE
- Confidence in predicted class
- 00.8
- 10.9
- 20.8
- Observed error flags
- 00
- 11
- 20
- Acceptance threshold
0.8
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
How would you know whether the review team makes the complete service more useful?
Measure review outcomes and workload as well as accepted model errors.
Observed confidence is not a correctness guarantee. The fixture does not simulate reviewer outcomes or queues.
Work through the Python challenges →