← All workshopsRead Would More Examples Help?

Choosing and Checking a Model

Compare the two greenhouse error curves

The gap is small while both errors stay high

Botie has training and validation errors at matching training sizes. A small gap can coexist with poor predictions on both sets.

BOTIE

The changed curves have training errors around 12 to 13 and validation errors around 13.5 to 14. Which next investigation would you prioritise before assuming more rows will solve the problem?
The inputs we are comparing

Illustrative training errors

Narrowing gap

  1. 01
  2. 12
  3. 23

High errors, small gap

  1. 012
  2. 112.5
  3. 213

Illustrative validation errors

Narrowing gap

  1. 09
  2. 16
  3. 24

High errors, small gap

  1. 014
  2. 113.8
  3. 213.5
Result for narrowing gap
  1. 08
  2. 14
  3. 21
Choose an explanation to test
The later gap widens

This check uses the pieces from “Pair training and validation errors at each size”.

BOTIE

The supplied error measurements no longer show a steadily shrinking gap. Predict validation error minus training error at each recorded training size.
Illustrative training errors
  1. 01
  2. 12
  3. 23
Illustrative validation errors
  1. 04
  2. 15
  3. 28

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What different evidence would high errors with a small gap provide compared with low training error and much higher validation error?

Read error levels and train-validation gaps together, then test an explanation rather than treating a curve pattern as a guarantee.

These errors are supplied teaching illustrations. No model is fitted by this construction.

Work through the Python challenges →
Continue the courseHave We Really Held Out a New Example? →
← What Must a Model Improve On?