← All workshopsRead Normalisation Layers

Neural Networks

Normalise within each example

A shifted and stretched version of the same row

Botie's activation row can shift and change scale as earlier layers learn. We want to inspect the pattern relative to that row's own mean and spread.

BOTIE

The second row has a different offset and a larger positive scale. How should its normalised pattern compare with the first row?
The inputs we are comparing

One example's activations

Example 1

  1. 01
  2. 13
  3. 25

Example 2

  1. 010
  2. 114
  3. 218
Result for example 1
  1. 0-1.2247
  2. 10
  3. 21.2247
Choose an explanation to test
Two low responses and two high responses

This check uses the pieces from “Express activations in their row's standard deviations”.

BOTIE

Normalise this example's four activations using their own mean and population standard deviation. Return the four standardised coordinates without changing their order.
One example's activations
  1. 02
  2. 12
  3. 26
  4. 36

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why would a constant row need additional handling that these examples do not show?

Real normalisation layers add stabilisation and often learned scale and shift; keep the axis of normalisation explicit.

These rows have nonzero variance. The challenge omits learned scale, shift and epsilon to focus on the normalisation itself.

Work through the Python challenges →
Continue the courseHopfield Networks →
← Dropout