When the Camera Misses a Frame, Keep Predicting Honestly

The Estimate Has to Continue Between Observations

BOTIE

Separating prediction from measurement correction made recursive filtering useful for more than one stream of perfectly timed readings. Between observations, a system still moves. The earlier estimate and the motion model let us predict its next state, including the uncertainty added by that movement. A new sensor reading supplies a separate correction when it is available.

The Kalman formulation introduced in the previous lesson keeps these operations distinct. We use that separation when a camera frame supplies no position: perform the prediction and leave out the measurement correction. Barfoot’s treatment of state estimation develops the broader motion and observation models behind this reasoning. Our chronological example handles an absent reading; using a late reading from an earlier time requires another decision.

Sources for this history

There Is No Detection. That Does Not Put the Part at Zero.

A part passes behind an obstruction. The next camera frame gives Botie no usable position, although the belt continues moving. If we fill that blank with zero, the filter treats it as a real observation at the origin and pulls the estimate toward somewhere the camera never measured.

We still have the previous estimate and expected movement. Use those to predict where the part may be. Keeping the missing marker visible lets us distinguish that prediction from an estimate corrected by new camera evidence.

Two chronological scalar intervals with supplied uncertainty; no out-of-sequence update, real camera or physical actuation.

Step 1 of 1

The object keeps moving while a frame is absent

Each interval has a motion prediction; only actual measurements can supply a correction.

A missing entry is null in the workshop and None in Python. A numeric zero remains a real observation.
Boundary	Expected displacement	Camera observation
First interval	1m	Missing
Second interval	1m	4 m

First interval
Expected displacement
1m
Camera observation
Missing
Second interval
Expected displacement
1m
Camera observation
4 m

A missing entry is null in the workshop and None in Python. A numeric zero remains a real observation.

Record whether an observation exists

Each motion interval remains present even when its observation does not.

Keep the timeline while a measurement is absent

The camera misses a position while the conveyor keeps moving. Botie must carry the estimate through that interval without using zero as an invented reading.

BOTIE

The camera misses a position while the conveyor keeps moving. Botie must carry the estimate through that interval without using zero as an invented reading.

Choose an example

Initial mean/variance record

mean

variance

First expected displacement in m

First added motion variance in m²

First measurement in m, or null

First measurement variance in m²

Second expected displacement in m

Second added motion variance in m²

Second measurement in m, or null

Second measurement variance in m²

Position mean and variance after the available updates

{
  "mean": 3.75,
  "variance": 0.625
}

Follow the operations

1. Initial mean/variance record
```
{
  "mean": 0,
  "variance": 1
}
```
2. First expected displacement in m
```
1
```
3. First added motion variance in m²
```
1
```
4. Predict through one motion interval
```
{
  "mean": 1,
  "variance": 2
}
```
5. First measurement in m, or null
```
3
```
6. First measurement variance in m²
```
1
```

7. Correct when a measurement arrived

{
  "mean": 2.3333,
  "variance": 0.6667
}

8. Second expected displacement in m
```
1
```
9. Second added motion variance in m²
```
1
```

10. Predict through one motion interval

{
  "mean": 3.3333,
  "variance": 1.6667
}

11. Second measurement in m, or null
```
4
```
12. Second measurement variance in m²
```
1
```
13. Correct when a measurement arrived
```
{
  "mean": 3.75,
  "variance": 0.625
}
```
14. Result
```
{
  "mean": 3.75,
  "variance": 0.625
}
```

Two chronological scalar intervals with supplied uncertainty; no out-of-sequence update, real camera or physical actuation.

1. Record whether an observation exists

Keep one entry for each observation time so a missed frame does not erase an interval. In the Python example, None means that no position was supplied. A numerical zero means that the camera measured the origin. They must follow different paths through the update. Repeating an old image also does not make it a new independent reading.

A detector abstaining can be useful information for an application, but it is not automatically a position measurement with the usual observation model.

Repeatedly applying the same old reading as if it were independent new evidence can make the variance shrink without new information.

2. A prediction still happens during the gap

The belt moves through the first interval whether the camera sees it or not. Advance the mean and add that interval’s stated motion variance. If its frame is absent, carry this predicted state onward. The next interval then starts from that result, rather than from the original estimate.

Under our coefficient-one motion model with nonnegative independent process variance, variance grows or stays unchanged during unobserved intervals. This statement is specific to these dynamics; general state-transition matrices can contract some uncertainties.

initial mean = 0 m; variance = 1 m²
first displacement = 1 m; Q = 1 m²
first prediction = (mean1, variance2)
first measurement = absent: no correction

second displacement = 1 m; Q = 1 m²
second prediction = (mean2, variance3)

Carry the resulting mean and variance from each interval into the next prediction.

Check yourself

Questions on Parts 1 to 2

Question 1 of 2

Which action follows this lesson’s contract?

Use the information below.

A position observation is missing at an interval boundary. The motion model is still available.

3. The returning camera can carry more relative weight

When the camera returns, use the uncertainty accumulated through the gap. The larger prior variance changes the correction gain. Compare with a run that received both observations rather than reusing a gain from an earlier frame.

returning measurement = 4 m; R = 1 m²
K = 3 / (3 + 1) = 0.75
corrected mean = 2 + 0.75 × (4 - 2) = 3.5 m
corrected variance = 0.75 m²

The higher gain is a response to the supplied uncertainty model, not proof that a detector becomes more accurate after missing a frame.

4. An interval must carry its own duration and motion uncertainty

A longer interval gives the motion model more time to accumulate error. We need a model for how its uncertainty depends on duration. For a simple random walk, variance can grow at a specified rate over time. A model that carries position and velocity and allows acceleration has a different calculation. Reusing one unchanged Q for every possible interval would hide that choice.

The coding exercise accepts one displacement and added variance per interval so these choices remain explicit.

A stale image that eventually arrives is a measurement of an earlier time. Applying it directly as a current-time reading is outside this simple missing-observation procedure.

5. Keep the reference time beside the estimate

A filter result belongs to a particular time and frame. If a controller needs a later time, it needs another prediction and its additional uncertainty. A longer gap can also trigger a separately justified abstention or reacquisition policy.

Do not silently turn an estimate into an action recommendation merely because the arithmetic returned a finite number.

A downstream policy must decide when the age and uncertainty make another observation necessary.

6. Implement a sequence without hiding the missing entries

Store the state as mean and variance, and append the state after each boundary. The output has one row per interval, even when that interval had no measurement. That lets us inspect where uncertainty grew and where actual evidence reduced it.

An empty interval sequence returns an empty two-column array. It does not reset the initial estimate or invent a new first observation.

This is filtering in chronological order. Smoothing past states using later evidence is a separate procedure.