Which Rule Could Have Produced These Observations?

Botie tests a batch of seeds and records whether each one germinates. A proposed germination probability tells us how plausible each outcome would be. But the outcomes have already happened. What Botie wants to choose is the probability used by his model.

Let's try several proposed germination probabilities against the seeds we actually observed. Which setting makes that sequence most likely? Once we can calculate the comparison, we can ask how much it tells us about seeds we haven't tested.

Success probability = 2 / 3
Failure probability = 1 / 3
Sequence likelihood = (2 / 3) × (2 / 3) × (1 / 3) ≈ 0.1481

Keep the observed seed outcomes fixed and change a candidate success probability. Each candidate must explain the same successes and failures, not a different batch of its own.

0.75

A Bernoulli trial has two outcomes, encoded here as one and zero. Its parameter is the probability of one. Once we choose that parameter, the probability of zero is its complement.

Our model assumes each seed has the same germination probability and that outcomes are independent given that probability. Shared damage or different seed varieties could make those assumptions unsuitable.

If p = 0.8:
P(germination) = 0.8
P(no germination) = 1 - 0.8 = 0.2

The parameter describes the proposed process. A short batch does not have to reproduce that fraction exactly.

Under the independence assumption, the sequence probability is the product of its outcome probabilities. A germinated seed contributes the proposed probability; a seed that did not germinate contributes its complement.

The sequence contains two successes and one failure. Neither can be ignored just because one candidate fits the successes well.

L(0.5) = 0.5 × 0.5 × 0.5 = 0.125
L(0.8) = 0.8 × 0.8 × 0.2 = 0.128
L(p) = p² × (1 - p)

These numbers compare candidate rules on the same observations. They do not say there is a 12.8 percent probability that the parameter is 0.8.

The particular sequence success, success, failure is only one ordering. If we instead ask for exactly two successes in any order, there are three possible arrangements. A binomial count probability includes that extra factor.

For this fixed batch and fixed success count, the factor is the same for every proposed probability. It changes the reported probability but does not change which parameter maximises it.

P(exactly two successes | p = 0.8)
  = 3 × (0.8² × 0.2)
  = 0.384

Say which observation your likelihood describes. Do not compare a sequence probability from one model with an aggregated count probability from another.

For this model, the maximum-likelihood estimate is the successful share. A longer list of probability factors can become extremely small, so we often add their logarithms instead. Logarithms preserve which positive likelihood is largest.

Negating that sum turns maximisation into minimisation. Averaging by the number of trials gives the familiar binary negative log-likelihood used in classification. Predicting a separate probability for each example is a later extension.

p̂ = successes / trials = 2 / 3
log L(p) = 2 log(p) + log(1 - p)
mean negative log-likelihood = -log L(p) / 3

A logarithm is not a probability conversion. The log-likelihood can be negative and need not sum to one across settings.

An all-successful batch has its maximum at a probability of one in this simple model. That does not establish that failure is impossible in the wider population. Small samples can put the fitted setting at an extreme.

A prior distribution supports a different question about parameter uncertainty after seeing the batch. Held-out outcomes test whether the fitted rule predicts new observations. Neither uncertainty nor generalisation is settled by the training likelihood alone.

The workshop compares candidate settings rather than claiming to have run a general optimiser.

import numpy as np
p = np.array([0.25, 0.5, 0.75])
y = np.array([1, 1, 0])
factors = np.where(y[None, :] == 1, p[:, None], 1 - p[:, None])
print(np.prod(factors, axis=1).tolist())

[0.046875, 0.125, 0.140625]

Botie observes three germinated seeds and one that did not germinate. Both a moderate rate and a high rate can produce that outcome. Likelihood compares how well candidate rates account for the observations; it does not assign those candidates posterior probabilities.

Working with log likelihood turns products into sums and avoids tiny products for long records. When a count is zero, its log term contributes zero. Do not calculate an undefined logarithm and hope multiplying it by zero will repair it.

import numpy as np
rates = np.array([0.25, 0.50, 0.75])
log_likelihood = 3 * np.log(rates) + np.log1p(-rates)
relative = np.exp(log_likelihood - log_likelihood.max())
print(np.round(relative, 4).tolist())

[0.1111, 0.5926, 1.0]

If every seed in a small batch germinates, the maximum-likelihood rate is one. That is where this batch's likelihood is greatest. It does not establish that failure is impossible in future batches.

Keep the number of trials beside the estimate. A posterior distribution or an interval procedure addresses uncertainty that the single optimum omits. Also examine the trial model: seeds from one shared tray can be dependent, and combining them as independent trials may exaggerate the evidence.