Model the System Before Building It
Budget the inspection device memory
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Combine parameter count times width, batch size times activation bytes, and runtime overhead.
Botie must leave room for the model, its intermediate values, and the runtime.
Inspect the component calculation
Add(
First contribution: Add(
First contribution: Multiply(
Value: Parameter count
Multiplier: Bytes per parameter
)
Second contribution: Multiply(
Value: Batch size
Multiplier: Activation bytes per example
)
)
Second contribution: Fixed runtime bytes
)Examples and results
- Parameter count
10
- Bytes per parameter
4
- Batch size
2
- Activation bytes per example
8
- Fixed runtime bytes
12
- Required result
68
- Parameter count
10
- Bytes per parameter
4
- Batch size
8
- Activation bytes per example
8
- Fixed runtime bytes
12
- Required result
116
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
The model assumes linear activation scaling and fixed runtime overhead. Actual peak memory and quantised-model quality need measurement. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.
A larger batch consumes the remaining memory
This check uses the pieces from “Subtract the complete allocation from the memory limit”.
BOTIE
- Parameter count
24000000
- Bytes per parameter
2
- Batch size
6
- Activation bytes per example
3000000
- Fixed runtime bytes
10000000
- Memory limit in bytes
80000000
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Which term explains the negative headroom for the large original-format batch?
Retain units and signed headroom, then test both peak resource use and task quality after a memory-saving change.
The model assumes linear activation scaling and fixed runtime overhead. Actual peak memory and quantised-model quality need measurement.
Work through the Python challenges →