Adapting and Evaluating Modern Models
Add a narrow correction
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Compress with the down matrix, expand with the up matrix, and return only that correction.
Adapting a large model by updating every weight is expensive. Let's leave the existing projection fixed and add a correction built from two narrow matrices. We can follow both paths separately to see which part belongs to the original model and which part an adapter would learn.
Inspect the component calculation
Matrix times vector(
Rows of coefficients: Up projection
Input vector: Matrix times vector(
Rows of coefficients: Narrow down projection
Input vector: Input vector
)
)Examples and results
- Up projection
2 1 - Narrow down projection
1 -1 - Input vector
- 01
- 12
- Required result
- 0-2
- 1-1
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.
A narrow adapter changes both outputs
This check uses the pieces from “Add an adapter beside the frozen projection”.
BOTIE
- Input vector
- 02
- 1-1
- Frozen projection
2 0 0 1 - Narrow down projection
1 1 - Up projection
1 -2
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Which matrices would adaptation train, and which matrix remains fixed?
A low-rank factorisation constrains the correction without replacing the base projection.
This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.
Work through the Python challenges →