Comparing Treatments in the Same Population

Botie cannot rerun an old season as a randomised experiment. He does have starting soil measurements and outcomes under both schedules in each soil group.

We can calculate an outcome mean for each schedule within each group, then average both using the same target-population proportions. Whether that adjusted number is causal depends on what the measurements and causal assumptions leave out.

Adjusted treatment mean = (0.6 × 4) + (0.4 × 9) = 6
Adjusted control mean = (0.6 × 3) + (0.4 × 8) = 5
Adjusted difference = 6 − 5 = 1

Use the same target proportions to combine both schedules' outcomes. Change that target mix and see how the average effect depends on the population we are asking about.

1

Assume measured starting soil condition blocks the relevant common causes of schedule and outcome. Also require well-defined treatments, consistency between observed and specified treatment, no relevant interference, and positive chances of both treatments in the target strata.

These conditions are not established merely because a model fits outcomes well. Unmeasured plant health could still confound the comparison.

The example states identification assumptions so readers can see exactly what the arithmetic depends on.

Suppose we want the effect for a particular mix of dry and moist beds. The target weights record the proportions of those starting conditions. They must be nonnegative and sum to one.

Use that same mix to average the outcomes under both schedules. If we instead use the treated group's proportions for one average and the control group's for the other, differences in starting conditions enter the comparison again.

Target weights = [0.6,0.4]
Standardised treatment = 0.6 × 4 + 0.4 × 9 = 6
Standardised control = 0.6 × 3 + 0.4 × 8 = 5
Difference = 1

Changing the target population can change an average effect when stratum effects differ.

A mediator is affected by treatment and lies on a pathway to the outcome. Adjusting for it can remove part of the total effect. A collider is a common effect; conditioning on it can open an otherwise blocked association path.

Use the causal story and measurement timing to decide roles. A feature-selection algorithm designed for prediction cannot decide this from predictive usefulness alone.

Our adjustment uses a pretreatment common cause, not a later moisture measurement caused by the schedule.

import numpy as np
treatment = np.array([3., 8.])
control = np.array([1., 7.])
target_mix = np.array([0.5, 0.5])
effects = treatment - control
print(effects.tolist())
print(float(target_mix @ effects))

[2.0, 1.0]
1.5

If one schedule never occurs in a soil group, its outcome mean there cannot be estimated directly from those observations. A regression may still return a value, but it is extrapolating.

Report overlap, sample counts, model dependence, and sensitivity to unmeasured confounding. More elaborate fitting does not create missing treatment comparisons.

The workshop assumes all supplied stratum means are estimable; the limitations remain part of the result.

Within each pre-treatment group, estimate the outcome mean under each treatment. Then average both sets of means using the same group weights. Subtracting those standardised outcomes answers what the difference would be for that target mixture under the identification assumptions.

The target could be the study population or another specified population. Changing the weights changes the estimand. Keep their source explicit rather than letting whichever group has more records decide the question.

Treatment effects can differ between soil groups. In that case, two target populations can have different average effects even when every group-specific estimate is unchanged.

This is a reason to report the group estimates and support, not only one overall adjusted number.

import numpy as np
untreated = np.array([2.,7.])
treated = np.array([6.,8.])
effect = treated-untreated
for weights in [np.array([.5,.5]), np.array([.8,.2])]:
    print(round(float(weights@effect), 3))

2.5
3.4

If every dry bed received treatment, we have no untreated dry-bed mean to compare. An outcome model can produce a numerical prediction there, but it extrapolates beyond observed treatment overlap.

Inspect the counts behind every estimated group mean. Combining sparse groups or restricting the target population changes assumptions or the question. State the change. More flexible modelling does not create observations of a treatment that was never used in that group.