Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Decoding Temperature › Visualizing Temperature Effects

Michael BrenndoerferUpdated July 27, 20259 min read

Language AI HandbookAll chapters

Part 28 of 67GPT Architecture

Chapter 202Decoding Temperature › Visualizing Temperature Effects

Reading section 3 of 11 · Visualizing Temperature Effects · 9 minSection 3 of 11 · 9 min

Section 3 of 11 · 9 min read

Decoding Temperature › Visualizing Temperature EffectsOne section · Sections in this chapter3 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

To develop intuition for temperature, let's visualize how it reshapes a realistic probability distribution. We will simulate logits from a language model predicting the next token after "The weather today is" and examine distributions across a range of temperatures. This section gives you a feel for the continuous nature of temperature's effect, rather than just discrete snapshots.

A single number like $T = 0.7$ can feel abstract. What makes temperature intuition concrete is seeing the full distribution shift as $T$ changes. You will notice that some tokens barely change their relative rank while others experience dramatic swings in their probability. Tokens in the middle of the logit range are most affected: the very top tokens at low temperature already capture nearly all probability, so there is little room to grow, and the very bottom tokens are so far down the distribution that even high temperature cannot lift them to meaningful probabilities.

The redistribution of probability under temperature also has an important implication for generation diversity. When you double the temperature, you do not simply add a fixed amount of probability to each low-ranked token. Instead, you proportionally adjust the ratios between all token pairs. Tokens that were already close together in logit value (say, logit difference of 0.1) end up nearly equal in probability at any non-extreme temperature. Tokens far apart in logit value (say, logit difference of 3.0) remain unequal across a wide range of temperatures. This means the "effective vocabulary" from which sampling draws grows nonlinearly with temperature.

In[11]:

Code

# Simulate logits for a vocabulary of plausible next tokens
vocab = [
    "sunny",
    "cloudy",
    "rainy",
    "nice",
    "terrible",
    "cold",
    "warm",
    "hot",
    "perfect",
    "unpredictable",
]

# Logits which reflects different likelihoods
logits_weather = torch.tensor(
    [
        3.2,  # sunny - most likely
        2.8,  # cloudy
        2.1,  # rainy
        1.9,  # nice
        0.5,  # terrible
        1.5,  # cold
        1.8,  # warm
        1.2,  # hot
        2.4,  # perfect
        0.3,  # unpredictable
    ]
)

Out[12]:

Visualization

Bar chart showing peaked distribution at T=0.3 with sunny dominating. — At T=0.3, 'sunny' dominates with over 60% probability. This peaked distribution produces consistent, predictable completions at the cost of lexical variety.

Bar chart showing moderate spread at T=0.7. — At T=0.7, the distribution spreads across alternatives. 'Cloudy' and 'perfect' receive meaningful probabilities while 'sunny' retains its lead.

Bar chart showing standard distribution at T=1.0. — At T=1.0 (standard softmax), the model's learned preferences are preserved exactly. This is the baseline from which other temperatures diverge.

Bar chart showing flattened distribution at T=2.0. — At T=2.0, multiple completions carry meaningful probability. The top five tokens each have 10-18% probability, letting genuinely varied outputs.

At $T = 0.3$ , "sunny" captures over 60% of the probability mass, leaving little room for alternatives. This produces predictable text but misses the natural variation in language. Most weather descriptions do not use "sunny" as frequently as this distribution implies. At $T = 2.0$ , the top five tokens each have between 10% and 18% probability. Sampling here introduces meaningful diversity while still favoring contextually appropriate completions. Notice that even "terrible" and "unpredictable," which had the lowest logits, receive non-trivial probabilities at $T = 2.0$ , which could produce interesting creative writing but problematic factual generation.

A heatmap reveals the continuous transformation more clearly. Each row shows one token's probability as temperature varies from left (low $T$ , sharp distribution) to right (high $T$ , flat distribution). Watching a single row across the heatmap shows you exactly how that token's fate changes with temperature:

Out[13]:

Visualization

Heatmap with tokens on y-axis and temperature on x-axis, showing probability as color intensity. Bright band at top-left for sunny fades to uniform coloring on right. — Probability heatmap showing how each token's likelihood changes continuously across temperatures from 0.2 to 3.0. At low temperatures (left), probability concentrates on high-logit tokens like 'sunny' and 'cloudy'. As temperature increases (right), probability redistributes toward uniformity across all tokens, indicated by the even color saturation.

The heatmap shows the redistribution of probability mass. At low temperature, the top token ("sunny") absorbs most probability (dark band at top-left). As temperature increases, probability spreads more evenly across all tokens (lighter, more uniform coloring toward the right). The transition is not abrupt but smooth and continuous. Tokens that start with low probabilities (bottom rows) gain probability slowly at first, then more rapidly as temperature increases.

Entropy and TemperatureLink Copied

Entropy quantifies the uncertainty or "spread" of a probability distribution. A distribution that assigns all probability to a single token has zero uncertainty: we know exactly what will be sampled. A distribution that assigns equal probability to all tokens has maximum uncertainty: every outcome is equally surprising. Shannon entropy gives this intuition a precise mathematical form.

For a discrete distribution over $n$ tokens, Shannon entropy measures the average number of bits of information in a random draw. It is computed as:

H(P) = -\sum_{i=1}^{n} P(i) \log_2 P(i)

where:

$H(P)$ : the entropy of distribution $P$ , measured in bits
$P(i)$ : the probability of token $i$
$\log_2$ : logarithm base 2, so entropy is measured in bits
The negative sign ensures entropy is positive, since $\log$ of a probability between 0 and 1 is negative

Why does this formula make sense? Consider the term $-\log_2 P(i)$ : this is the surprise of observing token $i$ . A very likely token ( $P(i) \approx 1$ ) has near-zero surprise, while a very unlikely token ( $P(i) \approx 0$ ) has enormous surprise. Entropy is the probability-weighted average surprise across all tokens, measuring how surprised we expect to be, on average, when we sample from the distribution. A high-entropy distribution contains more "information" in each sample because samples are harder to predict.

Entropy has intuitive extremes. When one token has probability 1.0 and all others have 0, entropy equals 0 bits: no uncertainty at all. When all $n$ tokens are equally likely with probability $1/n$ , entropy reaches its maximum of $\log_2(n)$ bits, which for a vocabulary of 10 tokens is $\log_2(10) \approx 3.32$ bits.

Temperature directly controls entropy. Low temperature concentrates probability mass on high-logit tokens, reducing entropy and predictability. High temperature spreads probability more evenly, increasing entropy toward the maximum. The relationship between temperature and entropy is monotonically increasing: any increase in temperature (for the same set of logits) increases entropy. This means you can think of temperature as a direct entropy control: set $T$ to the level of uncertainty you want the model to exhibit at each generation step.

The entropy connection also explains why extremely high temperatures cause qualitative degradation in output. When entropy approaches $\log_2(V)$ where $V$ is the full vocabulary size of 50,000+ tokens, you are essentially sampling from a uniform distribution over the entire vocabulary. A 50,000-sided die produces tokens with no semantic relationship to the context. Real coherent text has low entropy at most positions: given a sentence like "The dog chased the," the next word is very constrained (nouns or pronouns, probably animate, probably syntactically appropriate), and the true distribution has entropy of perhaps 2 to 4 bits out of a possible 15+ bits for the full vocabulary.

In[14]:

Code

def compute_entropy(probs: torch.Tensor) -> float:
    """Compute Shannon entropy of a probability distribution."""
    # Filter out zero probabilities to avoid log(0)
    probs = probs[probs > 0]
    return -(probs * torch.log2(probs)).sum().item()

Out[15]:

Console

Temperature vs Entropy:

Temperature  Entropy (bits) 
---------------------------
0.3          0.693          
0.5          1.733          
1.0          2.877          
1.5          3.127          
2.0          3.214          
3.0          3.262          

Maximum possible entropy (uniform): 3.322 bits

With 10 tokens in our vocabulary, maximum entropy (achieved when the distribution is uniform) is $\log_2(10) \approx 3.32$ bits. At $T = 0.3$ , entropy drops to around 1.5 bits, showing the distribution is highly concentrated on just a few tokens. At $T = 3.0$ , entropy approaches 3 bits, nearly matching the uniform distribution's maximum uncertainty. Each additional bit of entropy roughly doubles the effective number of tokens that receive meaningful probability mass.

Out[16]:

Visualization

Line plot showing entropy in bits increasing from about 1.5 at T=0.2 to near 3.3 at T=3.0 with horizontal dashed line at maximum entropy. — Shannon entropy of the weather token distribution as a function of temperature. Entropy increases monotonically from roughly 1.5 bits at low temperature to near the maximum of 3.32 bits (dashed line) at high temperature. The shaded area highlights the range of achieved entropy values.