A Plausible Position Might Belong to the Other Part

Which Observation Belongs to the Object We Were Following?

BOTIE

Tracking adds a question that a detector alone does not answer. A detector reports objects in the current image; the tracker needs to know which report continues an earlier object’s history. Assigning observations to those continuing objects is called data association. An assignment can look plausible by position and still attach the history to the wrong object.

In robotic mapping, Neira and Tardós examined a related problem in 2001: matches that each look plausible alone can be incompatible when considered together. Their joint-compatibility method considered that combined evidence. We will use a much smaller one-track example. First reject readings that disagree too much with the predicted position and uncertainty. If several still fit, our rule leaves the identity unresolved instead of claiming to have solved the full assignment problem.

Sources for this history

Two Parts Pass the Camera. Which One Did Botie Track?

Botie has been following one part along the conveyor. The next image contains two detections. Feeding the first row into the filter can attach the tracked identity to a different part while still producing a smooth-looking position.

How much should the positions differ before we reject a candidate? That depends partly on how uncertain the prediction and camera are. Even after this check, two nearby parts may both fit. We will keep that unresolved choice visible before allowing a reading into the correction.

The example follows one track and gives every camera candidate the same error variance. It does not solve assignment across several tracks, use image appearance or move a physical robot.

Step 1 of 1

The detector reports candidates, not a known identity

All positions refer to the same frame and capture time.

A distance test can reject a large disagreement. Nearby candidates can still be ambiguous.
Quantity	Value
Predicted position	10 m
Predicted variance	1 m²
Observation variance for each candidate	1 m²
Candidate positions	[10.5,20]m

Predicted position
Value
10 m
Predicted variance
Value
1 m²
Observation variance for each candidate
Value
1 m²
Candidate positions
Value
[10.5,20]m

A distance test can reject a large disagreement. Nearby candidates can still be ambiguous.

A detector row does not automatically preserve object identity

A gate feeds an association decision before any new measurement can reduce uncertainty.

Gate candidates before correcting the tracked position

A nearby second part also fits Botie’s position prediction. Before correcting the track, decide whether the camera supplies one usable match or leaves an unresolved choice.

BOTIE

A nearby second part also fits Botie’s position prediction. Before correcting the track, decide whether the camera supplies one usable match or leaves an unresolved choice.

Choose an example

Predicted mean/variance record

mean

variance

Candidate positions in m at this time

1

2

Common candidate measurement variance in m²

Maximum normalised squared innovation

An updated estimate, or the unchanged prediction when association is unresolved

{
  "mean": 10.25,
  "variance": 0.5
}

Follow the operations

1. Predicted mean/variance record
```
{
  "mean": 10,
  "variance": 1
}
```
2. Candidate positions in m at this time
1. 010.5
2. 120
3. Maximum normalised squared innovation
```
4
```
4. Common candidate measurement variance in m²
```
1
```
5. Scale candidate disagreements
1. 00.125
2. 150
6. Threshold gate
1. 01
2. 10
7. Use only an unambiguous candidate
```
10.5
```
8. Correct when a measurement arrived
```
{
  "mean": 10.25,
  "variance": 0.5
}
```
9. Result
```
{
  "mean": 10.25,
  "variance": 0.5
}
```

The example follows one track and gives every camera candidate the same error variance. It does not solve assignment across several tracks, use image appearance or move a physical robot.

1. A detector row does not automatically preserve object identity

A learned detector may return several positions without knowing which one belongs to an earlier track. Data association is the assignment of such observations to the objects or states we have been estimating.

An appearance embedding or a label can provide more evidence, but neither should silently replace the time, frame and motion checks. A row’s place in a detector output is not a persistent object identifier.

We follow one existing track here. Keeping identities across several interacting tracks requires additional assignment reasoning.

2. Compare disagreement with how much disagreement was expected

Suppose the camera reports a position away from the prediction. The distance means more when both sources are precise than when their errors are large. Use the squared disagreement, called the squared innovation, and divide by the expected variance of that disagreement. Under this model, that variance includes the prediction variance and the camera-error variance.

This produces a dimensionless squared distance. A fixed metre cutoff would treat a precise prediction and a very uncertain prediction as if they were equally informative.

predicted mean = 10 m; P⁻ = 1 m²; R = 1 m²
S = P⁻ + R = 2 m²
candidate 10.5 m: d² = (10.5 - 10)² / 2 = 0.125
candidate 20 m:   d² = (20 - 10)² / 2 = 50

This scalar expression is the one-dimensional Mahalanobis squared distance for the innovation.

Check yourself

Questions on Parts 1 to 2

Question 1 of 2

What does row0 establish about an earlier track?

Use the information below.

A detector returns two position rows in one image but no persistent object IDs.

3. A gate marks candidates worth considering

Use a threshold to mark which scaled disagreements are small enough to consider. This threshold test is called a gate. With an appropriate Gaussian model, a threshold can be chosen from a probability region. Here the threshold is simply supplied as part of the teaching policy; these few cases do not establish a real false-match rate.

Our threshold is a supplied teaching policy. We do not tune it from these few examples or claim a false-association rate.

squared-distance threshold = 4
d² = [0.125,50]
pass mask = [1,0]

A rejected observation might be an outlier, another object, or evidence that the prediction model is wrong. The gate alone cannot tell which.

4. Several passing candidates leave a decision unresolved

Count the candidates that pass. If there is exactly one, this small rule selects it. If there are none, no available reading passed the check. If there are several, we still do not know which belongs to the part. In both cases the update receives no measurement and keeps the prediction.

For candidates on both sides of the same prediction, even the nearest candidate can change from frame to frame. A complete tracking system may compare appearance, geometry, multiple tracks and longer histories.

candidates = [10.5,9.5] m
d² = [0.125,0.125]
passing candidates = 2
association = unresolved
measurement correction = skipped

Skipping is not proof that continuing without an observation is acceptable for an application. A downstream policy may need to abstain.

5. Only an associated observation enters the correction

Use the selected numeric observation with the same measurement-error model used by the gate. If association returned no observation, preserve the predicted mean and variance. Do not shrink variance just because several nearby detections were available.

The workshop assumes the candidates share one R. If candidates have different uncertainty estimates, the selected candidate must carry its own variance into the correction.

Matching coordinate frames and timestamps is a prerequisite to this calculation, not an optional improvement after gating.

6. The vector form keeps coordinate relationships

A camera position may have several coordinates whose errors are related. The innovation covariance S describes those relationships as well as their individual spreads. The vector calculation below solves a linear system using S, then uses that result to scale the disagreement. Treating every coordinate as an unrelated scalar would discard those relationships.

innovation ν = z - H x⁻
S = H P⁻ Hᵀ + R
d² = νᵀ S⁻¹ ν

NumPy pattern:
scaled = np.linalg.solve(S, innovation)
d2 = float(innovation @ scaled)

The covariance must be appropriate and nonsingular for this solve. Ill-conditioned models and nonlinear observations need further treatment.

7. Inspect failures instead of widening the gate automatically

If the predicted model is biased, widening a gate can admit more wrong-object detections without fixing the bias. If the scene is crowded, a tight gate may still contain several candidates. Track ambiguity, rejected observations and identity switches separately from position error.

Use annotated sequences and held-out conditions when evaluating a learned detector plus tracker. This fixture supplies no evidence about a real system’s association accuracy.

A smooth track can still follow the wrong part. Keep identity switches beside position errors.