The model doesn't know the next token. It guesses.
Pick a prompt below, then adjust temperature, top-K and top-P to see how the next-token distribution changes.
The capital of France is
- Probability distribution
- At each step, the model doesn't just pick a token (a token can be a whole word, part of a word, or punctuation — see the Tokens lab). It calculates how likely every possible next token is (the bars below), and a token is chosen based on those odds. This is called sampling.
- Temperature
- Controls how sharply the model favours high-probability tokens, not which token it considers 'best'. Lower values make the highest-probability tokens dominate the choice. Higher values flatten the distribution, giving less-likely tokens a more competitive chance of being picked.
- Top-K
- Instead of considering every possible token, only look at the K most likely ones, then choose among just those. A small K (e.g. 3) keeps output focused; turning it off considers everything.
- Top-P (nucleus sampling)
- Instead of a fixed count, keep adding the next most likely tokens until their combined probability reaches P (e.g. 90%), then choose among that group. It adapts automatically: a small group when the model is confident, a bigger one when it's unsure.
1.00
Balanced
off
off
The probabilities shown in this lab are illustrative. Real language models calculate distributions across vocabularies containing tens of thousands of possible tokens.