Section 3 of 24 · 29 lessons

Classical Machine Learning

Different problems give us different information to work with. We might have examples with known answers, observations we want to group, or a pattern we need to find in a picture. We will work through the methods developed for these problems and why their different approaches can be useful.

0 of 29 finished

Distance

Before we ask a model to find similar examples, we need to decide what similar means. Different ways of measuring distance can give us different answers.

  1. 01What Near MeansWhich of two examples is nearer depends on how we measure distance. We will compare those choices before using nearness to find neighbours or form groups.
    Build with Botie →Read the lesson

Regression

A single straight line will not explain every relationship. We can include more measurements or allow a more flexible rule, but we still need to work out how to fit it and when that flexibility stops helping.

  1. 02Multiple & Polynomial RegressionOne measurement may not be enough to explain the outcome. We can give a regression model several inputs, including transformed inputs, and work out how much each contributes to the prediction.
    Build with Botie →Read the lesson
  2. 03Fitting by WalkingRather than solving for a fitted line all at once, we can improve an initial guess through small adjustments. The gradient tells us which direction increases the error, so we can try moving the other way.
    Build with Botie →Read the lesson
  3. 04Ridge & LassoA flexible model can fit accidental details in the training data. Penalising large coefficients gives us a way to discourage that behaviour, with ridge and lasso making different kinds of adjustment.
    Build with Botie →Read the lesson

Classification

Sometimes we want a category rather than a number. We will look at how nearby examples, a sequence of questions, and scores for different classes can each give us a way to decide.

  1. 05k-Nearest NeighboursIf similar examples tend to have similar answers, nearby labelled examples may help us predict a new one. We still need to decide how to measure distance and how many neighbours to ask.
    Build with Botie →Read the lesson
  2. 06Decision TreesA series of simple questions can narrow down a decision. A decision tree learns which questions to ask from the examples and places a prediction at the end of each route.
    Build with Botie →Read the lesson
  3. 07More Than Two ClassesSome problems have more than two possible answers. We need a way to compare the evidence for each category and decide how the resulting scores become a prediction.
    Build with Botie →Read the lesson
  4. 08Naive BayesA warm casing and an unstable reading may each suggest a faulty sensor. We need a way to combine those clues, while being explicit about what we assume about their relationship.
    Build with Botie →Read the lesson
  5. 09Support Vector Machines and MarginsSeveral lines may separate the same labelled examples. We need a reason to prefer one, and a way to handle examples that cannot all be separated cleanly.
    Build with Botie →Read the lesson

Ensembles

One model may depend too heavily on a few examples or miss a pattern another model captures. Combining models can help, but the way we train and combine them matters.

  1. 10BaggingSmall changes in the training data can produce very different fitted models. Fitting several models to resampled data and combining their answers can make the result less dependent on one particular fit.
    Build with Botie →Read the lesson
  2. 11Random ForestsA collection of trees is less useful if they all make the same mistakes. Giving them different samples and different feature choices encourages the variety that can make their combined prediction more reliable.
    Build with Botie →Read the lesson
  3. 12Gradient BoostingA model's mistakes tell us what it has not yet explained. Boosting adds models in stages, with each new model learning a correction to the predictions already made.
    Build with Botie →Read the lesson

Clustering

We do not always have labels telling us which examples belong together. Clustering tries to find groups from the measurements themselves, using a chosen definition of similarity.

  1. 13k-Means ClusteringWe may want to group similar observations without having labels for those groups. K-means uses nearby centres to assign the observations, then moves the centres to better represent their assigned points.
    Build with Botie →Read the lesson
  2. 14Hierarchical ClusteringA collection can have small groups inside larger groups. Joining the closest groups one step at a time lets us inspect that hierarchy without choosing one group count at the outset.
    Build with Botie →Read the lesson
  3. 15Density-Based ClusteringGroups need not form round clouds, and isolated observations need not belong to any group. We can instead ask where enough neighbours support extending a cluster.
    Build with Botie →Read the lesson

Kernels

A pattern that is difficult to separate in its original coordinates may become simple after a transformation. Kernels let us use comparisons in that transformed space without explicitly calculating every new coordinate.

  1. 16The Kernel TrickA pattern may be difficult to separate with a straight boundary until we change how it is represented. A kernel lets us compare examples in a transformed space without explicitly building every new feature.
    Build with Botie →Read the lesson
  2. 17Kernel Ridge RegressionRidge regression gives us a way to control a fit, but its original features may not describe the relationship well. Kernel comparisons let us apply the same idea in a transformed feature space.
    Build with Botie →Read the lesson
  3. 18Kernel Principal ComponentsThe useful structure in a dataset may not lie along a straight direction in its original coordinates. Kernel PCA looks for variation after a transformation, using comparisons between examples to do the work.
    Build with Botie →Read the lesson

Sequences

The order of observations can matter. We will start with a model that uses the current state to predict the next one, so we can see what this limited form of memory allows.

  1. 19Markov ChainsThe next event may depend on what is happening now. A Markov chain describes that dependence using probabilities of moving from one state to another, while leaving earlier history out of the prediction.
    Build with Botie →Read the lesson

Classical Computer Vision

A computer receives a picture as an array of pixel values. We need ways to turn those values into useful observations, such as where an edge lies or whether a familiar shape is present.

  1. 20Images as ArraysBefore a computer can find a seed in a photograph, it needs measurements to work with. We will look at the rows, columns, channels, and number types that make up an image.
    Build with Botie →Read the lesson
  2. 21Image ThresholdingA bright seed on a dark tray gives us a first rule to try. Compare each pixel with a cutoff, keep the result as a mask, and inspect where that rule fails.
    Build with Botie →Read the lesson
  3. 22Cleaning Masks With MorphologyA threshold can leave speckles and gaps. We can test a small shape against the mask to remove or connect particular structures, provided we understand what else that operation will change.
    Build with Botie →Read the lesson
  4. 23Connected ComponentsA mask tells us which pixels were selected. To count regions, we need a definition of connected and a way to give each reachable group its own label.
    Build with Botie →Read the lesson
  5. 24Filters and EdgesThe boundary of an object often appears as a change in brightness. Small image filters compare nearby pixels so we can measure those changes instead of treating each pixel in isolation.
    Build with Botie →Read the lesson
  6. 25Template MatchingIf we already know what a pattern looks like, we can search for it in a larger picture. Template matching compares that known pattern with each possible location in the image.
    Build with Botie →Read the lesson
  7. 26Histogram of Oriented GradientsAn object's outline can remain recognisable when the lighting changes. Histograms of oriented gradients describe local edge directions, giving a detector information about shape without relying on exact pixel values.
    Build with Botie →Read the lesson
  8. 27Keypoints and DescriptorsTwo photographs may show the same scene from different positions. We need distinctive points we can recognise in both pictures and a numerical description that lets us compare their surroundings.
    Build with Botie →Read the lesson
  9. 28Image AlignmentA camera can move or turn between photographs. Matching points gives us evidence of that movement, which we can use to work out how to bring the images into alignment.
    Build with Botie →Read the lesson
  10. 29Haar CascadesSearching every part of a picture can be expensive, especially when most regions contain nothing relevant. A cascade uses inexpensive checks to reject unlikely regions before applying more demanding tests.
    Build with Botie →Read the lesson