← All workshopsRead Residual Connections

Neural Networks

Let the layer make a correction

A correction is not a replacement

Botie's next layer should be able to preserve a useful representation and contribute only the correction it needs.

BOTIE

For the nonzero correction, what should the block return?
The inputs we are comparing

Incoming representation

Example 1

  1. 02
  2. 13

Example 2

  1. 01
  2. 1-2

Layer correction

Example 1

  1. 00
  2. 10

Example 2

  1. 00.5
  2. 11
Result for example 1
  1. 02
  2. 13
Choose an explanation to test
A correction cancels one feature and preserves another

This check uses the pieces from “Keep an identity route beside the correction”.

BOTIE

Add the layer's correction to its incoming representation. Return every coordinate, including one where the correction cancels the old value and another where the correction is zero.
Incoming representation
  1. 03
  2. 1-1
  3. 22
Layer correction
  1. 0-3
  2. 10
  3. 20.5

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What does the zero-correction case establish about the block's ability to leave a representation alone?

An identity path gives a deep block a straightforward way to preserve information and pass gradients.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseLearning From a Few Records at a Time →
← Radial Basis Networks