Image Augmentation

Botie's training photographs show seeds facing in only a few directions. A new photograph places the same kind of seed the other way around. We still recognise it, but why should a model know that orientation was irrelevant?

We can include transformed copies in training. But we need to follow what happens to the answer too. Flipping a seed photograph can leave its class unchanged. For segmentation, the target is a picture: we must flip its mask by the same operation, or we will tell the model to label the wrong pixels.

The image and its mask describe the same scene. Follow one marked pixel as the columns reverse and check that the mask moves with the intensity values.

A transformation is useful when it represents variation we expect at prediction time without corrupting the target. Position, orientation, illumination, and scale may matter differently for different tasks.

Do not start with a list of effects to apply. Start with the measurement process. Can the camera rotate? Can brightness vary? Would the class remain the same? A crop might remove the only object carrying an image-level label.

Augmentation encodes assumptions about the task. More aggressive changes are not automatically better examples.

For classification, the label may be a single class ID. For segmentation, the label is a grid aligned with the image. A horizontal flip reverses columns in both arrays. A crop must use the same selected rectangle in both.

Sampling a random flip separately for the image and mask can misalign them. Sample the decision once and apply it to the pair. If resizing categorical masks, use a method such as nearest-neighbour interpolation that does not invent fractional class IDs.

flipped_image = image[:, ::-1].copy()
flipped_mask = mask[:, ::-1].copy()

The copy gives each result its own storage. It also avoids passing a negative-stride view to an interface that cannot accept one.

import numpy as np
image = np.array([[200,120,10], [180,20,10]])
mask = np.array([[1,1,0], [1,0,0]])
print(image[:, ::-1].copy().tolist())
print(mask[:, ::-1].copy().tolist())

[[10, 120, 200], [10, 20, 180]]
[[0, 1, 1], [0, 0, 1]]

A flipped photograph is related to its source. If the original is in training and its reflected copy is in testing, the evaluation no longer asks whether the model can handle an independent photograph.

Split by the appropriate original unit first, such as tray, plant, person, or acquisition session. Then augment only the training examples as part of fitting. Use a defined, repeatable evaluation preprocessing procedure.

Original trays → training / validation / test
Training photographs → sampled training transformations

Adding ten versions of a tray does not give us ten independently collected trays. Augmentation changes appearances, not the amount of independent evidence.

A random generator can choose whether to apply a transformation on each training presentation. Record its seed while debugging so a surprising image-label pair can be reproduced.

Reproducibility also depends on the order of examples and the sequence of random calls. A seed does not excuse inspecting only the original data. Display transformed pairs, including border cases where a crop removes an object.

rng = np.random.default_rng(7)
flip = rng.random() < 0.5
# Apply this same decision to image and spatial labels.

A random horizontal flip can be useful training variation; it does not simulate every possible orientation or camera effect.

Compare a model trained with the proposed augmentation against the same fitting procedure without it, using held-out original observations. Inspect which errors changed, not just the average score.

Botie's workshop concentrates on the first correctness check: transform an image and its mask together. A model trained on misaligned labels would be learning from errors we introduced ourselves.

This lesson changes the training inputs and labels. It does not train a network or claim that these transformations improve every dataset.

A horizontal flip reverses the x direction across the image width. The old right boundary becomes the new left boundary, measured from the opposite edge. Swapping array columns without updating rectangles leaves the labels on the wrong object.

Apply one sampled transformation to every spatial label associated with the photograph: masks, boxes and keypoints. Each representation needs its own coordinate conversion, even though the underlying geometric change is shared.

import numpy as np
width = 10
box = np.array([1., 2., 4., 6.])
flipped = box.copy()
flipped[[0,2]] = width - box[[2,0]]
print(flipped.tolist())
print(float(box[2]-box[0]) == float(flipped[2]-flipped[0]))

[6.0, 2.0, 9.0, 6.0]
True

A crop can remove the diseased part of a leaf while retaining the image-level damaged label. A reflection can reverse readable text. Brightness changes can erase a colour cue that defines the class. Visual plausibility does not guarantee label preservation.

Inspect transformed examples with the target task in mind. Compare training with and without the proposed augmentation on the same held-out acquisition groups. Keep the evaluation preprocessing fixed so an apparent improvement is not merely a change in what was measured.