What Would This Prediction Actually Change?

A model can flag a pump that may fail, but someone still has to decide what to do. Begin with that decision and work backwards to the information it needs.

13 sections · 4 questions · 1 coding challenge

1 / 13

The arrow keys move between sections too.

A Requirement Has to Describe Something People Need

BOTIE

Engineers working on large systems have to translate a broad request into parts that can be built and tested. NASA’s systems engineering handbook describes a progression from stakeholder expectations to technical requirements and a design. Otherwise, separate teams can each deliver a working component while the assembled system fails the people using it.

We will use that reasoning for an invented pump inspection service. The arithmetic can tell us whether a proposed inspection policy meets stated limits. Whether the predictions are good enough still needs evidence from representative pumps. First principles give us a way to question the design; they do not supply missing measurements.

Sources for this history

NASA Systems Engineering Handbook: System Design Processes

Botie Has an Alarm. Who Is Going to Answer It?

Suppose we want to help a maintenance team find pumps that need inspection. We could start by choosing a model. But imagine that it sends the team hundreds of warnings each morning. They can inspect only a few pumps, so most of those predictions never lead to anything.

Before building that model, let's work out what the team can actually do. Who receives a warning? How much time do they have? What happens if a faulty pump is missed? Those answers give our design something to aim for.

These are invented maintenance records, with one row per pump.

Step 1 of 2

Follow one warning

A warning needs a recipient and an action before it can help a pump.

The model is one part of the decision procedure.
Step	Information	What happens
Sensor	Vibration reading	Send a measurement
Model	Estimated failure risk	Send a score
Policy	Score and inspection capacity	Schedule or defer
Technician	Inspection findings	Repair if needed

Sensor
Information
Vibration reading
What happens
Send a measurement
Model
Information
Estimated failure risk
What happens
Send a score
Policy
Information
Score and inspection capacity
What happens
Schedule or defer
Technician
Information
Inspection findings
What happens
Repair if needed

The model is one part of the decision procedure.

Step 2 of 2

Count the work that reaches the team

These decisions schedule two inspections. Their durations differ, so counting warnings alone would miss the time requirement.

1 means schedule an inspection; 0 means defer.
Pump	Inspect?	Minutes if inspected
A	1	20
B	0	40
C	1	30

A
Inspect?
1
Minutes if inspected
20
B
Inspect?
0
Minutes if inspected
40
C
Inspect?
1
Minutes if inspected
30

1 means schedule an inspection; 0 means defer.

From a prediction to an appointment

Each connected pair refers to the same pump. Multiplication counts its time only when an inspection is scheduled.

How much inspection work does this policy create?

Change the inspection decisions or the time each inspection takes. Botie totals the work the team would receive.

BOTIE

Botie has proposed inspections, but the team needs to know how many minutes they require.

Choose an example

Inspection decisions, 1 = inspect

Inspection duration per pump (minutes)

Inspection minutes required

Follow the operations

1. Inspection decisions, 1 = inspect
1. 01
2. 10
3. 21
2. Inspection duration per pump (minutes)
1. 020
2. 140
3. 230
3. Multiply
1. 020
2. 10
3. 230
4. Add a list
```
50
```
5. Result
```
50
```

This checks workload, not whether an inspection prevents a fault. Durations are invented fixed estimates.

1. Name the action someone can take

“Predict failures” leaves several questions unanswered. A failure next year and a failure before tomorrow’s shift require different responses. For this service, the decision is whether to schedule an inspection during the next shift. A technician receives the assignment and records what they find.

A software example follows the same reasoning. If we predict that a support message is urgent, who will move it ahead in the queue? The prediction needs an agreed action and enough time for that action to help.

Write the action in ordinary language before choosing the model output. If nobody can act on the answer, revisit the problem.

2. Separate the outcome from the signal

The outcome we care about is avoidable pump downtime. The model sees measurements such as vibration and temperature. An inspection finding is another observation, while a risk score is an estimate produced from observations. These are different things.

We might label a training row by whether a fault was found within a day. That label is convenient, but only inspected pumps get findings. A model trained on those records may learn the old inspection policy as well as the faults. Ask how each label becomes observable.

A proxy is a measurable substitute for something we care about. State what it misses. More warnings is a poor substitute for less downtime.

3. Turn “manageable” into a limit

The maintenance team has 60 inspection minutes available in this small example. Each proposed assignment consumes part of that time. Represent a decision with 1 for inspect and 0 for defer. Multiplying a decision by its duration gives the minutes assigned to that pump.

The inputs are aligned lists: the first decision and first duration both belong to pump A. A dot product multiplies matching entries and adds the contributions.

Decisions = [1, 0, 1]
Durations in minutes = [20, 40, 30]
Assigned minutes = 1 × 20 + 0 × 40 + 1 × 30 = 50
Remaining minutes = 60 − 50 = 10

The policy fits the time budget. We have not yet shown that it chooses the pumps that most need help.

Count a different pair of appointments

NumPy stores the decisions and durations in one-dimensional arrays. Their positions must describe the same pumps.

Python

import numpy as np
decisions = np.array([1, 1, 0])
minutes = np.array([15, 25, 35])
print(int(decisions @ minutes))

Output

The scalar result is total scheduled time. The zero prevents the third duration from contributing.

4. Keep facts separate from assumptions

A shift length may be a contractual limit. Inspection duration is usually an estimate with variation. The chance that an inspection prevents a breakdown is a further assumption that needs evidence. Writing all of these as bare numbers makes them look equally certain.

Keep a small design record: what each quantity means, its units, where it came from, and what range seems plausible. For an unknown duration, a timed trial tells us more than another decimal place in the estimate.

We will deliberately vary these assumptions in later workshops. The design should make it clear which measurement could change our decision.

5. Give the model an information boundary

A prediction made before a shift can use earlier sensor readings. It cannot use the technician’s later findings. Define a prediction time and an outcome window before assembling training rows.

Also decide what happens when readings are missing. A missing temperature should not silently become a normal temperature. The design may defer to a technician, use a documented fallback, or decline to make a prediction.

The next lessons develop this boundary into a testable data contract.

6. Write a requirement we can check

“The service should be useful” gives us no acceptance test. “The scheduled inspections must fit the available minutes” gives us an immediate calculation. Predictive usefulness needs a separate requirement, such as a limit on missed faults in a representative evaluation, with its sample size reported.

A requirement also needs an owner. Someone must decide whether a miss is tolerable and what evidence would justify changing that limit. An engineer should not quietly bury those choices inside a default threshold.

Botie’s workshop handles the scheduling calculation. Our Python challenge also reports remaining capacity, including a negative result when the proposed schedule is too long.