Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Causal Language Modeling › The CLM Objective

Michael BrenndoerferUpdated July 7, 20257 min read

Language AI HandbookAll chapters

Part 22 of 67Pre-training Objectives

Chapter 155Causal Language Modeling › The CLM Objective

Reading section 3 of 13 · The CLM Objective · 7 minSection 3 of 13 · 7 min

Section 3 of 13 · 7 min read

Causal Language Modeling › The CLM ObjectiveOne section · Sections in this chapter3 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Now that we understand how to decompose sequence probability, we need a way to train a model to produce good probability estimates. This requires two things: a loss function that measures prediction quality, and a mechanism to improve the model based on that measurement. The CLM objective is the standard answer to both. It is derived from maximum likelihood estimation, a classical statistical principle, applied to the autoregressive factorization we just derived. Understanding the derivation step by step reveals why the final formula looks the way it does and why the choices made along the way are the right ones.

From Maximum Likelihood to Minimum LossLink Copied

Our goal is to find model parameters $\theta$ that make the training data as probable as possible. Given a training sequence $x_1, \ldots, x_n$ , we want to maximize:

P_\theta(x_1, \ldots, x_n) = \prod_{t=1}^{n} P_\theta(x_t | x_{<t})

Maximizing a product of many small probabilities is numerically unstable. As sequences grow longer, the product shrinks toward zero, causing underflow. The standard solution is to work with logarithms. Since $\log$ is monotonically increasing, maximizing a probability is equivalent to maximizing its log:

\log P_\theta(x_1, \ldots, x_n) = \sum_{t=1}^{n} \log P_\theta(x_t | x_{<t})

The product becomes a sum, which is numerically stable and computationally convenient. Now, optimization algorithms typically minimize rather than maximize, so we flip the sign to get our loss function:

\mathcal{L}(\theta) = -\sum_{t=1}^{n} \log P_\theta(x_t | x_{<t})

where:

$\mathcal{L}(\theta)$ : the loss function we minimize during training
$\theta$ : the model parameters (weights and biases of the neural network)
$n$ : the length of the training sequence
$P_\theta(x_t | x_{<t})$ : the probability the model assigns to the correct token $x_t$ given the context $x_{<t}$
$\log P_\theta(x_t | x_{<t})$ : the log-probability, which is negative since probabilities lie in $(0, 1)$

The Connection to Cross-EntropyLink Copied

This loss function has a beautiful interpretation: it's the cross-entropy between the model's predicted distribution and the true distribution (which puts all probability mass on the actual next token).

To see why, recall that cross-entropy measures how well a predicted distribution $Q$ matches a true distribution $P$ :

H(P, Q) = -\sum_x P(x) \log Q(x)

At each position $t$ , the "true distribution" is a one-hot vector: probability 1 for the actual token $x_t$ , and 0 for everything else. The model predicts a distribution $P_\theta(\cdot | x_{<t})$ over all vocabulary tokens. The cross-entropy simplifies to:

H = -1 \cdot \log P_\theta(x_t | x_{<t}) - \sum_{x \neq x_t} 0 \cdot \log P_\theta(x | x_{<t}) = -\log P_\theta(x_t | x_{<t})

Only the true token's probability matters. This is why cross-entropy loss is also called "negative log-likelihood" in language modeling contexts.

Understanding the Loss SignalLink Copied

The loss function creates an intuitive learning signal. When the model assigns high probability to the correct token, the loss contribution is small. When the model is surprised, the loss contribution is large.

Out[4]:

Visualization

Line plot showing negative log of probability decreasing from infinity near zero to zero near one, with steep descent for low probabilities. — The negative log-likelihood loss function. As the predicted probability for the correct token increases (x-axis), the loss decreases rapidly (y-axis). The curve is steep near zero, creating strong gradients when the model is confident but wrong. Near probability 1.0, the curve flattens. This gives gentle gradients when the model is already performing well.

Consider these scenarios:

Confident and correct ( $P = 0.9$ ): $-\log(0.9) \approx 0.1$ . Small loss, weak gradient. The model is doing well here.
Uncertain ( $P = 0.1$ ): $-\log(0.1) \approx 2.3$ . Moderate loss. Room for improvement.
Confident and wrong ( $P = 0.01$ ): $-\log(0.01) \approx 4.6$ . Large loss, strong gradient. The model needs to update significantly.

This asymmetry is powerful: the model receives the strongest teaching signal precisely where it's making the biggest mistakes.

Efficient Learning from Every TokenLink Copied

A remarkable property of CLM loss is that it decomposes across positions. A single sequence of length $n$ provides $n$ independent gradient signals, one for each prediction. This is dramatically more efficient than classification tasks where a single input yields a single label.

Consider training on a document with 1,000 tokens. Each forward pass produces 1,000 predictions and 1,000 loss terms. The model learns from every token simultaneously, extracting maximum information from the training data. This efficiency is one reason language models can learn so much from their training corpora.

ImplementationLink Copied

Let's implement this loss function to see how it works in practice:

In[5]:

Code

import torch.nn.functional as F


def compute_clm_loss(logits, targets, ignore_index=-100):
    """
    Compute causal language modeling loss.

    Args:
        logits: Model outputs of shape (batch, seq_len, vocab_size)
        targets: Target token IDs of shape (batch, seq_len)
        ignore_index: Token ID to ignore in loss computation (e.g., padding)

    Returns:
        Scalar loss value (mean over all valid positions)
    """
    # Reshape for cross-entropy: (batch * seq_len, vocab_size)
    batch_size, seq_len, vocab_size = logits.shape
    logits_flat = logits.view(-1, vocab_size)
    targets_flat = targets.view(-1)

    # Cross-entropy computes -log(softmax(logits)[target])
    loss = F.cross_entropy(
        logits_flat, targets_flat, ignore_index=ignore_index, reduction="mean"
    )
    return loss

Out[6]:

Console

Example CLM loss: 4.7462
Perplexity: 115.15
Random baseline (vocab=100): loss=4.6052, ppl=100

With random logits, the loss is close to $\log(V)$ where $V$ is the vocabulary size, because the model assigns roughly uniform probability $1/V$ to all tokens. As training progresses, the model learns to concentrate probability mass on likely continuations, reducing the loss.

Out[7]:

Visualization

Bar chart showing flat uniform distribution before training. — Before training, the model assigns nearly uniform probability across all tokens.

Bar chart showing peaked distribution after training. — After training, probability concentrates on likely continuations given the context.