The Right Numbers in the Wrong Columns

A Model Can Depend on More Than Its Code

BOTIE

A model project can look like a small program wrapped around a learning algorithm. In practice, the learned coefficients also depend on the records that supplied their inputs. Sculley and colleagues’ 2015 technical debt paper examined how those surrounding dependencies make ML systems difficult to maintain. An upstream change can alter the meaning of a prediction even when the model’s source code stays untouched.

Feature order is one particularly small dependency we can inspect. A coefficient learned for temperature must keep receiving temperature when another program sends the record. A schema describes the named fields; an adapter arranges accepted fields into the representation the model expects. This agreement is a software contract. Keeping names attached to values gives us a way to check it before a matrix calculation conceals the mix-up.

Sources for this history

Sculley and colleagues: Hidden Technical Debt in Machine Learning Systems (2015)

Why Did the Same Pump Get a Different Score?

Botie receives the same temperature and vibration readings as yesterday. The model has not changed, yet its score has. Someone changed the order of the fields in the response. Both entries are numbers, so the matrix multiplication happily carries on.

We need a boundary between the outside record and the model. That boundary names the measurements, checks their meaning, and arranges them in the order the model was trained to use.

The workshop assumes numerical values in agreed units; the Python adapter covers structural errors.

Step 1 of 1

Keep a name beside each measurement

The coefficient belongs to the feature name, not whichever number arrives first.

Invented readings and coefficients; these are not physical fault probabilities.
Feature	Incoming position	Coefficient
temperature	1	2
vibration	0	10

temperature
Incoming position
1
Coefficient
2
vibration
Incoming position
0
Coefficient
10

Invented readings and coefficients; these are not physical fault probabilities.

A list keeps order; a record keeps names

Names reconnect measurements to their learned coefficients.

Reconnect the feature adapter

A reordered sensor response changes Botie’s prediction.

BOTIE

A reordered sensor response changes Botie’s prediction.

Choose an example

Incoming feature names

1

2

Aligned values

1

2

Saved model order

1

2

Saved coefficients

1

2

Score after arranging the features

Follow the operations

1. Aligned values
1. 020
2. 13
2. Incoming feature names
1. 0t
2. 1v
3. Saved model order
1. 0t
2. 1v
4. Locate named columns
1. 00
2. 11
5. Gather selected entries
1. 020
2. 13
6. Saved coefficients
1. 02
2. 110
7. Dot product
```
70
```
8. Result
```
70
```

The workshop assumes numerical values in agreed units; the Python adapter covers structural errors.

1. A list keeps order; a record keeps names

A Python list stores values in a sequence. A dictionary lets us address a value by a key. Neither structure tells us the unit automatically. A temperature field still needs a stated unit and a policy for missing measurements.

Our adapter receives column names and aligned values. The saved model supplies its required names. Looking up each required name gives us the positions to gather.

Reject repeated names. If temperature appears twice, choosing the first one conceals an ambiguous input.

2. Arrange the features before multiplying

The incoming record puts vibration first. The model expects temperature first. We gather by name, then perform the weighted sum. The adapter belongs before scaling as well, because the saved scaling values also belong to specific features.

incoming names = [vibration, temperature]
incoming values = [3, 20]
model order = [temperature, vibration]
arranged values = [20, 3]
score = 20 × 2 + 3 × 10 = 70

Multiplying the original sequence would produce 206. Both answers are finite, so a numerical check alone cannot find the mistake.

Read by name

A dictionary can make the intended order explicit.

Python

row = {"vibration": 3, "temperature": 20}
print([row[name] for name in ["temperature", "vibration"]])

Output

[20, 3]

The dictionary order is irrelevant because the requested names determine the output order.

Check yourself

Which field reaches each weight?

Question 1 of 2

Which vector reaches the model?

Use the information below.

Incoming names [vibration, temperature], values [3,20]. Model order [temperature,vibration].

3. Give the boundary a useful failure

A missing required field should identify the field. An unexpected optional field can be ignored if the contract says so. Reject unequal name and value lengths before pairing them; zip otherwise truncates silently.

Version a unit change or implement an explicit conversion. Accepting the same name with a different meaning creates the original bug again.

The challenge checks structural correctness. A production adapter also validates types, finite values, ranges and units before serving a prediction.

4. Test a change that should preserve the score

Reorder the incoming names and values together. The arranged model vector should stay unchanged. Then remove a required field and expect an error. These tests describe observable behaviour and survive a rewrite of the adapter.

Store the required feature order with the fitted artifact. Keeping it in a second handwritten list gives the next change two places to drift.

A passing contract test does not establish predictive accuracy. It establishes that the model received the intended representation.

5. Version the boundary with the artifact

Suppose the temperature name remains unchanged while its unit changes. A name-based gather will pass every shape test and still send the wrong quantity to the model. Store feature units and accepted schema version with the artifact, and make the adapter either convert a documented version or reject it.

A migration test should send the same physical reading through both supported representations and compare the resulting model vector. Keep that test separate from predictive evaluation so a unit bug cannot hide behind a favourable score.

The feature contract is part of the model procedure. Review it whenever preprocessing or the producer changes.