What Does an Alarm Tell Us?

Botie has a warning light that usually turns on when a greenhouse sensor is faulty. That sounds useful. But when he checks the sensors that triggered it, many are working normally. Has the warning light stopped working?

Before answering, we need to ask two different questions. How often does a faulty sensor trigger the light? And, among sensors that trigger the light, how many are faulty? Those questions describe different groups. Let's follow the sensors through both groups before reaching for a formula.

All alarms = 9 + 99 = 108
Faulty among alarms = 9 / 108 ≈ 0.0833
Alarms among faulty sensors = 9 / 10 = 0.9

An alarm can come from a faulty sensor or a working one. Change how common faults are while keeping the alarm rates fixed and see how the explanation of an alarm changes.

A probability needs a reference group. In this report, the fault rate describes all sensors. The alarm rate among faulty sensors describes only the faulty ones. Once we are told that a sensor raised an alarm, our reference group becomes the sensors that raised alarms.

Conditioning means restricting our attention using information we know. The vertical bar in a conditional probability reads as 'given'. Its position matters.

P(alarm | faulty) = 9 / 10 = 0.9
P(faulty | alarm) = 9 / 108 ≈ 0.0833

Both statements can be true at the same time. Catching most faults does not establish that most alarms are faults.

An alarm can come from either a faulty sensor or a working one. We need both contributions. If we count only the faulty sensors that triggered, we have a numerator but no complete denominator.

The fault rate before reading the alarm is the prior probability. The alarm rate within each group tells us how compatible the alarm is with that group.

P(faulty and alarm) = 0.01 × 0.9 = 0.009
P(working and alarm) = 0.99 × 0.1 = 0.099
P(alarm) = 0.009 + 0.099 = 0.108

These are shares of all sensors. They become expected counts if we multiply by a proposed population size; expected counts need not be whole numbers.

To find the faulty share of the alarm group, divide its contribution by the total alarm contribution. This is the posterior probability, meaning the probability after incorporating this observation.

Bayes' rule writes the same bookkeeping for named events. The denominator accounts for all the ways the observed evidence can occur.

P(faulty | alarm)
  = P(alarm | faulty) × P(faulty) / P(alarm)
  = 0.009 / 0.108
  ≈ 0.0833

This updates a probability. Choosing whether to send a technician is another question, involving the cost of a missed fault and an unnecessary visit.

Now move the same warning system to sensors with a much higher fault rate. Even if its behaviour within the faulty and working groups stays the same, an alarm can become more convincing.

That is why we should not carry a positive predictive value from one population to another without checking the assumptions. The new population might also change the alarm's behaviour within each group.

New prior: P(faulty) = 0.2
Faulty alarm contribution = 0.2 × 0.9 = 0.18
Working alarm contribution = 0.8 × 0.1 = 0.08
Updated faulty probability = 0.18 / 0.26 ≈ 0.6923

The hardware did not improve in this calculation. We changed how common each possible cause was.

Our two causes cover every sensor and do not overlap. Their prior probabilities must therefore sum to one. The alarm rates are conditional on those causes; they do not have to sum to one.

An event assigned zero probability by every cause cannot be used in this calculation because there is no alarm group to condition on. Repeating the update with another alarm also needs a model of dependence: two copies of the same report are not two independent pieces of evidence.

Botie's workshop asks you to form the contributions and normalise them. It leaves the final inspection decision to a later lesson.

import numpy as np
prior = np.array([0.01, 0.99])
alarm_rate = np.array([0.9, 0.1])
joint = prior * alarm_rate
posterior = joint / joint.sum()
print(np.round(posterior, 4).tolist())

[0.0833, 0.9167]

A warning system can catch most faults while producing many false alarms. To see why, start with a population rather than a percentage. Separate faulty sensors from working sensors, then ask how many in each group trigger the light.

Only after counting those two routes should we restrict attention to the sensors whose lights are on. The denominator has changed from all sensors, or all faulty sensors, to all alarms. That change of group is the reason the two conditional probabilities differ.

1,000 sensors: 10 faulty, 990 working
80% of faulty sensors trigger: 8 alarms
10% of working sensors trigger: 99 alarms
Faulty among alarms = 8 / (8 + 99) ≈ 0.0748

Now suppose Botie checks a batch that may have been damaged in transit. Faults are more common in this batch. Even with the same sensitivity and false-alarm rate, the warning light now means something different: a larger share of the alarms can come from faulty sensors.

So, before using a probability attached to a test result, we need to know which population it describes. A result from ordinary deliveries may not apply to this damaged batch.

import numpy as np
prior = np.array([0.01, 0.20])
sensitivity, false_alarm = 0.8, 0.1
alarm_fault = prior * sensitivity
alarm_working = (1 - prior) * false_alarm
print(np.round(alarm_fault / (alarm_fault + alarm_working), 4).tolist())

[0.0748, 0.6667]