What Could We Know at Prediction Time?

Botie wants to predict at noon whether a greenhouse sensor will need repair later that day. The data export includes a morning reading, a noon reading, and a completed repair report entered that evening.

The evening report could make the prediction look easy. It may even state the answer. But it was not available at noon, so using it would test a different task from the one we intended to solve.

The export contains information that arrived at different times. Move the decision cutoff and check which measurements Botie could actually have used at that moment.

State what is predicted, for whom, and at what time. Without that definition, we cannot judge whether a feature was available legitimately.

Our task is a noon prediction of later repair need. A completed evening report fails the timing requirement even if it is stored in the same row as the morning measurements.

A column's name is not enough. Investigate how the value was created, updated, and joined into the table.

Each field has an availability time. Keep it only when that time is at or before the prediction cutoff. The workshop uses whole hours to make this boundary visible.

Prediction cutoff: 12
Availability times: [9, 12, 18]
Available by cutoff: [1, 1, 0]

The timestamp of the underlying event can differ from the time the system learned about it. A report describing a morning event but uploaded in the evening still cannot be used at noon.

Leakage does not require an obviously future-looking column. A mean, vocabulary, selected feature, or imputation reference learned using held-out observations has also crossed the evaluation boundary.

Split first according to the intended prediction setting. Fit preprocessing and the model on training data. Apply those fitted choices to validation or test data without relearning from the held-out group.

Pipelines help with the order of fitting operations. They cannot determine whether a raw field was available at the intended decision time. That requires knowledge of the data collection process.

A realistic evaluation replays the information constraints of deployment. Time order, delayed labels, related entities, and record updates may all matter. A random row split is not automatically wrong, but it answers the wrong question when it ignores the dependency we care about.

The workshop returns the eligible measurements in their original order. It does not decide whether those features are useful or ethically appropriate; it checks the stated timing boundary.

Removing leakage may lower the score. That is a better measurement of the intended task, not a reason to put the unavailable information back.

A repair may concern a noon failure but only be entered that evening. If Botie predicts at noon, the repair report is unavailable even when a retrospective export assigns it the failure's date. An event timestamp alone does not establish that a feature was usable.

For each input, record how it is obtained and when it becomes available. Reconstruct training rows from that point of view. A lag must account for reporting delay, not merely move a value to an earlier-looking column.

A useful test is to alter future observations while leaving the available history fixed. A correctly constructed historical feature should stay unchanged. This does not prove the whole pipeline is clean, but it catches a direct violation of the information boundary.

Apply the same reasoning to preprocessing, category vocabularies and feature selection. If future rows influence a fitted reference, the leak can remain even when the final model sees only historical rows.

import numpy as np
def trailing_mean(y, origin):
    return float(y[origin-2:origin].mean())
y = np.array([10., 12., 14., 16., 18.])
changed = y.copy()
changed[3:] = 999
print(trailing_mean(y, 3))
print(trailing_mean(changed, 3))

13.0
13.0