Give the unlikely token a chance

Lower temperature with vocabulary order fixed

Botie has fixed next-token scores and needs to control how concentrated their probability distribution becomes.

BOTIE

How does changing temperature from 2 to 0.5 affect these logits [2, 1, 0]?

The inputs we are comparing

Model logits

Same in both cases

Temperature

Example 1

Example 2

0.5

Result for example 1

00.5065
10.3072
20.1863

Choose an explanation to test

Concentrate probability on the first tokenChange which token has the largest probabilityMake all three probabilities equal

Two possible words tie for first

This check uses the pieces from “Scale logit gaps before softmax”.

BOTIE

Botie has two equally scored candidate tokens and a third with a lower score. Convert the supplied scores to probabilities at temperature 1. Return a probability for every candidate rather than selecting a token.

Model logits: 02
12
20
Temperature: 1

What result should your machine produce for this case?

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why can a non-leading token still have a chance even when its probability becomes small?

Your notes for later. These stay with this journey.

Distinguish a probability distribution from the particular token sampled from it.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →

Continue the courseAutoencoders →

← Autoregressive Generation