Section 9 of 24 · 11 lessons

Computer Vision with Neural Networks

We can write rules to find an edge or match a known pattern in a picture, but deciding which patterns matter becomes difficult as the task grows. Neural networks let us learn useful image features from examples. We will use them to recognise objects, label individual pixels, and find similar pictures.

0 of 11 finished

Spatial Layers

An edge is still an edge when it moves across a picture. We need layers that can apply the same detector in different places and summarise what it finds.

  1. 01ConvolutionA useful image pattern can appear anywhere in a picture. Convolution applies the same small set of weights at different positions, producing a map of where that pattern receives a response.
    Build with Botie →Read the lesson
  2. 02PoolingAfter detecting local patterns, we may not need every response at its original resolution. Pooling summarises nearby values, reducing the amount of data while giving up some information about exact positions.
    Build with Botie →Read the lesson

Convolutional Networks

Knowing which object is in a picture is one task; knowing exactly which pixels belong to it is another. We will build toward both from the image layers we have already covered.

  1. 03Convolutional NetworksRecognising an object usually takes more than finding one edge. A convolutional network combines learned local patterns through layers, using the resulting features to make a prediction about the image.
    Build with Botie →Read the lesson
  2. 04U-NetWhere does an object begin and end in an image? U-Net learns from images paired with labelled masks, using spatial detail from earlier layers to help predict the region belonging to each object.
    Build with Botie →Read the lesson

Embedding Pictures

Two pictures of the same object can have very different pixel values. To search by similarity, we need a representation that preserves the differences relevant to the task.

  1. 05A Vector for a PictureTwo photographs of the same object may look very different pixel by pixel. A trained network can provide a compact vector of features that gives us a more useful way to compare them.
    Build with Botie →Read the lesson
  2. 06Learning the Metric ItselfWhat makes two pictures similar depends on the task. By training on examples of useful matches, we can learn a representation in which the differences we care about affect the distance.
    Build with Botie →Read the lesson
  3. 07Searching a Collection of PicturesOnce pictures have numerical representations, we can search for nearby ones. Comparing every picture works for a small collection, but a larger collection gives us a reason to consider search indexes and their compromises.
    Build with Botie →Read the lesson

Learning From Limited Images

We may have too few labelled photographs to learn every visual pattern from scratch. We can create appropriate variations of our training images or test features learned elsewhere, while keeping the labels and evaluation honest.

  1. 08Image AugmentationThe same seed can appear in several positions or orientations. We can teach a model which changes should preserve an answer, but the transformation must keep the image and its labels consistent.
    Build with Botie →Read the lesson
  2. 09Transfer LearningA model trained on other photographs may already measure useful visual patterns. We can test those features with a new prediction head before deciding whether the feature extractor also needs to change.
    Build with Botie →Read the lesson

Finding and Counting Objects

Knowing an object is present does not tell us where it is or how many we found. We will describe locations with rectangles, measure their overlap, and work through what to do when several predictions point at the same object.

  1. 10Bounding Boxes and OverlapKnowing that a seed appears in a photograph does not tell us where it is. Rectangles give us a location to predict, and overlap gives us a way to compare that prediction with a labelled target.
    Build with Botie →Read the lesson
  2. 11Non-Maximum SuppressionA detector can report several rectangles around the same seed. We will rank candidates, keep a strong one, and remove sufficiently overlapping alternatives while preserving candidates for other objects.
    Build with Botie →Read the lesson