← All workshopsRead Which Feature Mattered

Choosing and Checking a Model

Does the model depend on height?

BOTIE

Apply the supplied twice-height rule to the original heights and calculate its mean squared error against the known answers.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Apply the supplied twice-height rule to the original heights and calculate its mean squared error against the known answers.

The model uses height to make predictions, but how much does it depend on that measurement? Let's keep the model fixed and shuffle which height belongs to each observation. The change in error measures the damage from breaking that relationship in these examples.

Inspect the component calculation
Divide(
  Amount: Add a list(
    Numbers: Square(
      Value or list: Subtract(
        Starting value: Multiply(
          Value: Original heights
          Multiplier: 2
        )
        Value to remove: Known answers
      )
    )
  )
  Divisor: Count items(
    Items: Square(
      Value or list: Subtract(
        Starting value: Multiply(
          Value: Original heights
          Multiplier: 2
        )
        Value to remove: Known answers
      )
    )
  )
)
Examples and results
Original heights
  1. 01
  2. 12
  3. 23
Known answers
  1. 02
  2. 14
  3. 26
Required result
0
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

The original model already makes some mistakes

This check uses the pieces from “Measure the damage caused by shuffled heights”.

BOTIE

The fixed model still predicts twice the height, but the known answers are imperfectly matched. Predict the increase in mean squared error after shuffling heights.
Original heights
  1. 01
  2. 12
  3. 24
Shuffled heights
  1. 04
  2. 12
  3. 21
Known answers
  1. 02
  2. 15
  3. 27

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why must the model and targets stay fixed during the shuffle comparison, and why does this score describe dependence rather than causation?

Interpret permutation importance as a fitted-model measurement on a particular dataset and metric.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseWhen Should a Probability Become an Action? →
← Searching for a Setting