Compare two forecasts on the same days

Let the earlier winner lose on another window

Botie has predictions recorded before their targets arrived. The methods must be compared on the same dates.

BOTIE

What should happen when the first method is exact on the changed targets?

The inputs we are comparing

Shared target observations

Two origins

A reversed winner

One row per forecasting method

Two origins

6	9
6.5	7.5

A reversed winner

5	5
4	7

Result for two origins

02
11.5

Choose an explanation to test

Report that method's lower error on this windowKeep the earlier ranking because it was already measured

Which rule coped with the surge?

This check uses the pieces from “Keep a separate error mean for each forecast method”.

BOTIE

Botie recorded three successive held-out targets and the forecasts two rules issued before those targets arrived. Return each rule's mean absolute error. Each prediction row is one rule, and columns follow the same dates as the actual values.

Shared target observations: 010
120
210
One row per forecasting method: 10 10 20
12 16 12

What result should your machine produce for this case?

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Can this error reducer establish that each forecast was made without seeing its target?

Your notes for later. These stay with this journey.

Check forecast provenance before comparing errors; the final arithmetic cannot detect earlier leakage.

The provided forecasts must already respect their historical origins. This reducer cannot detect leakage in how they were created.

Work through the Python challenges →

Continue the courseReinforcement Learning →

← Learning From a Series' Own Past