Generative Models

Pack only what the decoder needs

Use a different input with the same average

Botie's two pixel intensities must pass through a one-coordinate code. Equal averages can conceal different inputs.

BOTIE

Why can [1, 3] produce a reconstruction error when [2, 2] did not?

The inputs we are comparing

Input vector

Example 1

Example 2

Encoder matrix

Same in both cases

0.5

Decoder matrix

Same in both cases

Result for example 1

Choose an explanation to test

The decoder learns a new weight for the second caseBoth encode to the same scalar, so the decoder cannot distinguish them

The average loses a large disagreement

This check uses the pieces from “Measure what the bottleneck loses”.

BOTIE

Two sensor channels report 0 and 6. Botie's encoder compresses them to their average, and the decoder copies that average back into both channels. Measure the mean squared reconstruction error.

Input vector: 00
16
Encoder matrix: 0.5 0.5
Decoder matrix: 1
1

What result should your machine produce for this case?

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Which distinction between the two pixels is absent from the latent code?

Your notes for later. These stay with this journey.

Inspect the representation's information loss before interpreting reconstruction quality.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →

Continue the courseVariational Autoencoders →

← Sampling and Temperature