← guides.brandon-behring.dev

Sampling the next token

A language model doesn't emit words — it emits a probability distribution over the next token. Everything we call "sampling" — temperature, top-k, top-p, min-p — is just reshaping that distribution before drawing from it. Drag the controls and watch the shape change; then press Sample and watch draws pile up where the probability is.

1.00
off
off
off

Thank you very

50 tokens in play
  1. ␣much
    99%
  2. ␣very
    0.3%
  3. ,
    0.3%
  4. much
    0.1%
  5. ␣Much
    0.0%
  6. ␣well
    0.0%
  7. ␣highly
    0.0%
  8. ␣greatly
    0.0%
  9. ␣sincerely
    0.0%
  10. ␣MUCH
    0.0%
  11. ␣deeply
    0.0%
  12. ␣good
    0.0%
  13. ␣carefully
    0.0%
  14. ␣best
    0.0%
  15. ␣many
    0.0%
  16. ␣hard
    0.0%

Showing top 16 of 50 candidates · 50 kept after filtering. Probabilities are over the dumped candidates (a softmax of the top-50 logits), not the full vocabulary. Draw repeatedly and the gold ticks converge to the bars.

How this demo is built

No model runs here. A one-time offline script (scripts/build_traces.py) ran gpt2 once, dumped the top-50 (token, logit) pairs for each prompt to src/data/sampling_traces.json, and committed that tiny file. Everything you see is pure TypeScript (src/components/sampling-core.ts) reshaping that vector in your browser — so the page stays a static asset with zero inference cost. It's the reusable recipe for any interactive demo on this site: dump → JSON → typed island.