Images as Arrays

Botie has a photograph of seeds on a dark tray. We can point to a seed without thinking about the information that makes it visible. But the computer receives no instruction saying 'this part is a seed'. It receives measurements arranged in a grid.

So, let's begin with a tiny image we can read one entry at a time. Before trying to recognise anything, we need to know what each number describes, where it belongs, and what happens when we calculate with it.

Each entry belongs to a pixel position. Change a brightness and follow that same position into the floating-point image, including values too small for integer division to preserve.

Let's begin with grayscale. Each position holds one intensity measurement. The array's first dimension is its height and the second is its width, so we index it by row followed by column. In the displayed image, moving down increases the row number; moving right increases the column number.

This is easy to mix up with a geometric coordinate written as x followed by y. That convention names the horizontal position first. Before selecting a pixel, check whether the function wants an array index or a geometric coordinate.

image.shape = (height, width)
image[row, column] = image[y, x]

A value of 220 describes an intensity in our example. It does not identify object number 220.

An RGB image keeps three measurements at each position: red, green, and blue. In a height-by-width-by-channel array, selecting a row and column returns a three-entry vector. Selecting a channel as well returns one number.

Channel order is part of the representation. OpenCV commonly loads colour images in BGR order, while many plotting tools expect RGB. The shape alone cannot tell you which convention the data uses.

RGB pixel = [240, 30, 10]
image.shape = (height, width, 3)
red = image[row, column, 0]  # Only when the order is RGB

Some libraries use channel-first arrays instead. Read the interface rather than assuming every three-dimensional array has the same layout.

An eight-bit unsigned integer can store whole numbers from zero through 255. It cannot represent a negative intensity or a value above that range. Arithmetic performed in that type can wrap around instead of doing what we intended.

Convert to a floating-point array before an operation that needs fractions or a wider range. Dividing by 255 puts these particular eight-bit values on a zero-to-one scale. It does not automatically correct lighting, undo gamma encoding, or produce a physical measurement of light.

pixels = image.astype(np.float64)
scaled = pixels / 255.0

Scaling makes the input units explicit. Other image formats can use different ranges, so 255 is not a universal denominator.

We can inspect just the part of the tray containing a seed. Two slices select a range of rows and a range of columns. The starting indices are included and the stopping indices are excluded.

A basic NumPy slice usually shares storage with its original array. Make a copy if you intend to edit the crop without changing the source image.

crop = image[1:3, 1:3].copy()
# Rows 1 and 2, columns 1 and 2

Cropping removes positions outside the selected rectangle. Resizing, which changes the sampling grid, is a different operation.

import numpy as np
image = np.array([[10,20,10], [20,220,30], [10,200,20]], dtype=np.uint8)
crop = image[1:3, 1:3].copy()
print(image.shape)
print(int(image[1, 1]))
print(crop.tolist())

(3, 3)
220
[[220, 30], [200, 20]]

Flattening turns the grid into a list. That can be useful for calculating an overall mean, but a list does not retain neighbourhoods through its shape. A filter needs to know which pixels were above, below, and beside each other.

In the next lesson we will compare pixel intensities with a cutoff. Keep the result in the original grid so that the later steps can use its spatial arrangement.

In Botie's workshop, convert the tray's intensities to the requested scale without changing its rows or columns.

Botie crops a seed before measuring its location. Inside that crop, row zero and column zero refer to the crop's upper-left pixel, not the original photograph. To place a result back into the full image, restore the crop offset.

Keep this coordinate conversion alongside any resizing or rotation. An accurate detector can produce an incorrect displayed box if its coordinate system is forgotten between preprocessing and presentation.

import numpy as np
crop_origin = np.array([4, 7])
local_pixel = np.array([2, 3])
global_pixel = crop_origin + local_pixel
print(global_pixel.tolist())
print(global_pixel[::-1].tolist())

[6, 10]
[10, 6]

Combining red, green and blue channels at one pixel changes how we describe that location. Combining neighbouring pixels changes spatial detail. Both use averages, but their axes and information loss are different.

A plain channel mean is a teaching simplification, not a universal perceptual grayscale conversion. Preserve the original image when trying representations, and state channel order, numerical range and any colour conversion used by a trained model.