Temperature scales the sharpness of the sampling distribution. T=0 collapses to greedy; T=1 is unmodified; T>1 flattens. It is the single most misunderstood knob in the LLM stack — most operators reach for it when top-p is the right tool.
- 01State the temperature formula.
- 02Explain what T=0, T=1, T=2 do behaviorally.
- 03Distinguish temperature from top-p and top-k.
The formula
softmax(logits / T). Low T amplifies the highest-probability tokens; high T flattens the distribution toward uniform. T=0 is equivalent to argmax (greedy).
The misunderstanding
'Higher temperature = more creative' is true but crude. Temperature affects the whole distribution uniformly. Top-p is usually the better creativity knob because it truncates the tail — high-temperature outputs often veer incoherent because low-probability nonsense tokens get amplified.
Practical defaults
Structured extraction, code, classification: T=0 or T≤0.3. Chat and general text: T=0.7 with top-p=0.9. Creative writing: T=0.8-1.0 with top-p=0.95. Almost never set T>1.2 in production.
- Temperature = sharpness of the distribution.
- For creativity, prefer top-p; for determinism, prefer T=0.