← All workshopsRead Have We Really Held Out a New Example?

Choosing and Checking a Model

Audit the held-out greenhouses

Repeated copies of one overlapping greenhouse

Several rows can belong to one greenhouse. Botie needs to know whether validation really holds out new greenhouse identities.

BOTIE

Training IDs are [A, A] and validation IDs are [A, A, A]. How many overlapping greenhouse identities should the audit report?
The inputs we are comparing

Training greenhouse IDs

A proper group holdout

  1. 0A
  2. 1A
  3. 2B

Repeated overlap

  1. 0A
  2. 1A

Validation greenhouse IDs

A proper group holdout

  1. 0C
  2. 1C

Repeated overlap

  1. 0A
  2. 1A
  3. 2A
Result for a proper group holdout
0
Choose an explanation to test
Repeated groups within separate splits

This check uses the pieces from “Count shared greenhouse identities across the split”.

BOTIE

Each split contains repeated readings from its own greenhouses. Predict the number of greenhouse identities shared across the two splits.
Training greenhouse IDs
  1. 0west
  2. 1west
  3. 2east
Validation greenhouse IDs
  1. 0south
  2. 1north
  3. 2north

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why does a row-level holdout fail to establish performance on new greenhouses, and which unrecorded relationships could this ID audit still miss?

Choose the held-out unit to match the intended deployment claim, then audit overlap at that unit.

This checks the supplied group identity only. It cannot discover hidden relationships or incorrectly assigned IDs.

Work through the Python challenges →
Continue the coursePipelines →
← Would More Examples Help?