Did the Caller Leave It Out, or Send No Reading?
A Named Field Still Needs an Agreement
BOTIE
A schema records rules that a program can check, including which field names must be present and which values they may contain. JSON Schema treats presence and null as separate questions. That distinction addresses a practical problem in exchanging records: a sender can produce a readable message that the receiver misunderstands. Sculley and colleagues’ 2015 paper describes how such upstream dependencies become a maintenance problem in ML systems. We will make this small part of the agreement explicit before applying the model’s saved transformations.
The Default Replaced a Real Zero
Botie receives a temperature reading of zero. The helper checks whether the value is true or false in Python, sees that zero counts as false, and substitutes yesterday’s default. Nothing crashes. The model now receives a temperature the sensor never sent.
Before arranging values into the model’s feature order, we need to decide which values the service accepts. A field can be absent, present with no measurement, or present with a number. Those are different situations. Let’s keep them separate long enough to choose a sensible policy.
Rejected null entries are diagnostic placeholders and must stay out of the model input. A deployed service also needs request-size limits and enforcement on the server.
Step 1 of 1
Four requests that a truthiness check confuses
For this optional field, only absence permits the declared default.
| Incoming record | What is known | Policy |
|---|---|---|
| {} | No field supplied | Use the declared default |
| {"temperature": null} | Field supplied without a measurement | Reject |
| {"temperature":0} | A real numerical zero | Keep zero |
| {"temperature":"0"} | Text, not a numerical measurement | Reject |
{}
- What is known
- No field supplied
- Policy
- Use the declared default
{"temperature": null}
- What is known
- Field supplied without a measurement
- Policy
- Reject
{"temperature":0}
- What is known
- A real numerical zero
- Policy
- Keep zero
{"temperature":"0"}
- What is known
- Text, not a numerical measurement
- Policy
- Reject
Look for the key before looking at its value
The state of the field remains visible beside the value we may use.
Keep the zero; expose the missing measurement
Botie must distinguish what each caller supplied before preparing model inputs.
BOTIE
Decision and usable numerical value for each request
[
[
"keep",
"default"
],
[
8,
5
]
]Follow the operations
1. Sensor request records
[ { "temperature": 8 }, {} ]2. Optional numerical field name
temperature
3. Distinguish absent, null and numerical fields
[ { "kind": "number", "value": 8 }, { "kind": "missing", "value": null } ]4. Choose the optional-number policy
- 0keep
- 1default
5. Declared default
5
6. Fill only absent optional numbers
- 08
- 15
7. Collect two results
[ [ "keep", "default" ], [ 8, 5 ] ]8. Result
[ [ "keep", "default" ], [ 8, 5 ] ]
Rejected null entries are diagnostic placeholders and must stay out of the model input. A deployed service also needs request-size limits and enforcement on the server.
1. Look for the key before looking at its value
A Python dictionary connects a name, such as temperature, to a value. First ask whether that name is in the dictionary. Then, if it is, inspect what was supplied. A present None says something different from an omitted name; JSON decoding represents a supplied null as Python None.
The expression record.get(name) returns None in both of those situations. One option is to test name in record first. Another is to give get a unique object to return when the key is absent. That object is called a sentinel: it marks absence without being confused with a real supplied value.
The field-order adapter from the earlier lesson still matters. We are now checking whether each named measurement is usable before arranging it.
2. Choose the missing-value policy deliberately
Our contract makes temperature optional and gives absence a documented default. Explicit null is rejected because this particular model path needs a measurement. Zero remains a measurement.
A default is part of model behaviour. If a service uses a training-derived replacement, save it with the fitted model. Recomputing a replacement from each request would reintroduce the mutable fitted-state problem.
default temperature = 5
request {} → use 5
request {temperature:0} → use 0
request {temperature:null} → reject
request {temperature:"0"} → rejectThese are policy decisions, not estimates that the sensor is working correctly.
3. A number-shaped string is still another type
Converting every incoming value with float can accept text that a caller sent accidentally. It also hides which producer changed. Decide whether conversion belongs in a separately named compatibility adapter or whether the public boundary rejects it.
Python bool is a subclass of int. A numerical type check that accepts all int instances therefore also accepts True. Our small validator excludes booleans explicitly and rejects infinite or not-a-number values.
JSON does not define nonfinite numerical literals. The Python procedure checks them too because internal callers can bypass JSON decoding.
Inspect before replacing
This small program distinguishes presence and shows why a zero must survive. It does not implement the later reusable validator.
records = [{}, {"temperature": None}, {"temperature": 0}]
for row in records:
print("temperature" in row, row.get("temperature"))False None
True None
True 0Presence distinguishes the first two rows. The third row contains a valid number even though bool(0) is false.
4. Keep validation separate from preprocessing
First decide whether the record is acceptable. That is validation. For an accepted record, put the measurements into the model’s feature order and apply its saved transformations. Those steps prepare the input for prediction. Keeping them separate lets us say which part failed.
If temperature contains text that this service rejects, return an input error identifying that field. Passing the text onward and waiting for NumPy to fail would make the caller diagnose a later calculation instead. Keep sensitive raw readings out of general-purpose logs.
The workshop keeps rejected entries visible as null beside a reject decision. Those entries are diagnostic output and must not be treated as model inputs.
5. Required and nullable answer different questions
A required property must be present. A nullable property permits the explicit null value. It is possible to require a property while allowing null, or make a property optional while rejecting null when it is supplied.
A complete service schema also specifies extra fields, collection sizes and accepted ranges. In JSON Schema, properties alone does not require a field, and additional fields are allowed unless the schema constrains them. These defaults deserve a deliberate review.
These rules let a caller distinguish an omitted optional input from an invalid supplied measurement.
Check yourself
Questions on Parts 1 to 5
Question 1 of 2
Which value should this contract pass onward?
Use the information below.
The optional temperature default is 5. The request is {"temperature":0}.6. Do not let two producers invent two defaults
The Python client library and the server can each supply a default. If they choose different values, two callers can get different predictions while sending messages that both look valid. Decide whether omission asks the server to choose, and keep that decision with the model release.
Track rejected fields so a change becomes visible. A sudden rise in nulls could come from a failed sensor or a change in the code that writes messages. The validation result tells us where to investigate; it does not identify the cause by itself.
Preserve the original request identity when reporting failure so the caller can correct that request rather than create an unrelated prediction.
Check yourself
Questions on Parts 1 to 6
Question 1 of 4
What should happen?
Use the information below.
The request is {"temperature":"20"}. The contract accepts finite numerical values without coercion.Try it with Botie
Build with Botie
Write and run Python