Updating an Unknown Failure Rate

One of Botie’s first two sensors fails. The observed fraction is easy to calculate, but two trials give little information about the supplier’s long-run rate.

Several failure rates could plausibly have produced these few results. We will keep track of that uncertainty instead of immediately choosing one rate. First we describe which rates seem plausible before the trials; that is the prior. Then we calculate how likely the observed results would be at each rate. Combining that likelihood with the prior gives the posterior, our updated distribution over rates.

Observed trials: failure, working, working
New alpha = 1 + 1 = 2
New beta = 1 + 2 = 3

Next failure probability = alpha / (alpha + beta)
Next failure probability = 2 / (2 + 3) = 0.4

Record the new failures and working sensors separately. Compare the update from a weak prior with one based on much more prior evidence.

0.4

Let the rate be the probability that a sensor fails in a specified test. Each outcome is binary, and we assume trials are conditionally independent given one fixed rate. If sensors share a faulty batch, this assumption needs reconsideration.

A Beta distribution uses positive parameters alpha and beta to describe rates between zero and one. Equal parameters give a symmetric prior; increasing both while holding their ratio fixed concentrates it.

Failure has value one in all the examples. Reversing that convention reverses which count updates alpha.

A failure contributes one factor of the failure rate to the likelihood. A success contributes one factor of its complement. Multiplying these factors into the Beta density adds their counts to its two parameters.

Prior: alpha = 1, beta = 1
One failure, two working sensors
Posterior: alpha = 2, beta = 3
Posterior mean = 2 / (2 + 3) = 0.4

The prior parameters act like weights in this update, but need not represent literal historical observations.

The posterior describes uncertainty about the fixed rate. A future sensor still has a binary outcome, even if we somehow knew that rate exactly. Averaging the failure chance over the posterior gives its predictive probability.

Compare different reasonable priors when data are scarce. A strong prior can dominate a short trial. That is an assumption to inspect rather than a benefit to assume automatically.

More trials usually concentrate the rate posterior under this model; they do not eliminate random variation in individual outcomes.

import numpy as np
prior = np.array([1., 3.])
new_failures = 2
new_working = 0
posterior = prior + np.array([new_failures, new_working])
print(posterior.tolist())
print(float(posterior[0] / posterior.sum()))

[3.0, 3.0]
0.5

Keep the test procedure and observation window consistent. A sensor observed for one hour is not directly comparable with one observed for a year. Selection can also matter: testing only returned devices would not estimate the failure rate of all devices.

A posterior interval is conditional on the likelihood, prior, and data collection assumptions. It does not account automatically for a changed supplier or unrecorded batch effects.

The workshop updates the predictive mean. Inspecting the full distribution is the next step when interval estimates matter.

One observed failure in two sensors does not identify the supplier's long-run failure rate. Instead of choosing one candidate immediately, give each candidate a prior weight and multiply it by the likelihood of what happened.

Normalising those products gives posterior weights. A discrete candidate grid makes this process visible; a continuous beta distribution provides an analytic version for the Bernoulli model used earlier. A coarse grid is an approximation, not an alternative physical description of failure rates.

The next sensor can fail under any of the candidate rates. Average that failure probability using the posterior weights. This integrates uncertainty about the parameter rather than selecting the largest posterior entry and discarding the rest.

The example uses three equally weighted candidate rates. With a denser grid, the prior weights must still represent the intended prior distribution.

import numpy as np
rates = np.array([.1,.5,.9])
prior = np.array([1/3,1/3,1/3])
weight = prior*rates*(1-rates)
posterior = weight/weight.sum()
print(np.round(posterior, 4).tolist())
print(round(float(posterior@rates), 4))

[0.2093, 0.5814, 0.2093]
0.5

For a simulated future batch, first draw one rate from the posterior and then draw that batch's outcomes conditional on the rate. Drawing a new rate independently for every sensor would describe a different model.

The shared rate introduces dependence after averaging over parameter uncertainty. A batch with many failures may be more plausible than a plug-in calculation at the posterior mean suggests. This distinction matters when preparing for unusually poor shipments.

Generate replicated batches under the posterior and compare features such as total failures, runs of failures and differences between suppliers. A Bernoulli model with one shared rate may reproduce the total while missing clustering by shipment.

A posterior can be internally consistent and still answer the wrong model. If sensors share a manufacturing defect or conditions change over time, revise the data-generating assumptions rather than interpreting a narrow posterior as universal certainty.