Skip to content
AI LAB
2/5
LAB 02 / PREDICTION & SAMPLING

The model doesn't know the next token. It guesses.

Pick a prompt below, then adjust temperature, top-K and top-P to see how the next-token distribution changes.

The capital of France is

Probability distribution
At each step, the model doesn't just pick a token (a token can be a whole word, part of a word, or punctuation — see the Tokens lab). It calculates how likely every possible next token is (the bars below), and a token is chosen based on those odds. This is called sampling.
Temperature
Controls how sharply the model favours high-probability tokens, not which token it considers 'best'. Lower values make the highest-probability tokens dominate the choice. Higher values flatten the distribution, giving less-likely tokens a more competitive chance of being picked.
Top-K
Instead of considering every possible token, only look at the K most likely ones, then choose among just those. A small K (e.g. 3) keeps output focused; turning it off considers everything.
Top-P (nucleus sampling)
Instead of a fixed count, keep adding the next most likely tokens until their combined probability reaches P (e.g. 90%), then choose among that group. It adapts automatically: a small group when the model is confident, a bigger one when it's unsure.
1.00

Balanced

off
off

The probabilities shown in this lab are illustrative. Real language models calculate distributions across vocabularies containing tens of thousands of possible tokens.

Next03 EmbeddingsHow can meaning become geometry?