CRF Training › Gradient Computation
Language AI HandbookAll chapters CRF Training › Gradient ComputationOne section · Sections in this chapter4 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To optimize the log-likelihood, we need its gradient with respect to each weight . The gradient has an elegant form that emerges from the structure of the CRF, and understanding this form gives deep insight into what training accomplishes.
Taking the derivative of the log-likelihood for a single example:
where:
- The first term is the observed feature count: how often feature fires on the correct label sequence
- The second term is the expected feature count: the expected number of times feature would fire under the model's current probability distribution
This gradient has an intuitive interpretation:
- If a feature fires more often in the training data than the model expects, increase its weight (positive gradient)
- If a feature fires less often than expected, decrease its weight (negative gradient)
- When observed counts match expected counts, the gradient is zero (we've reached a stationary point)
The key insight is that the gradient is a simple difference: observed minus expected. This mirrors the gradient of logistic regression and maximum entropy models. In fact, you can view CRF training as a natural generalization of logistic regression to structured outputs: instead of matching observed class frequencies with expected class frequencies, you match observed feature counts along label sequences with expected feature counts under the model's current distribution.
The elegance of this formulation makes the convergence criterion transparent. You know the model has converged when, for every feature, the number of times it fires in the training data equals the number of times the model expects it to fire. This is a moment-matching condition, and it has a beautiful statistical interpretation: the model has learned to reproduce the sufficient statistics of the training data under its own probability distribution.
This has a practical consequence. Features that are rare in the training data will have noisy observed counts, which means their gradients are noisy too. A feature that fires exactly once in a large dataset has an observed count of 1 and a very uncertain expected count. Regularization, which we cover later, is partly a response to this noise: by penalizing large weights, it prevents the optimizer from placing too much confidence in observations from very rare features.
Computing Expected Feature Counts
The expected feature count requires summing over all positions and all label pairs, weighted by their posterior probabilities. This is where the pairwise marginals come in:
where:
- : the posterior probability that the label pair occurs at positions and , marginalized over all other positions
- : the value of feature when the previous label is , the current label is , and we are at position
The forward-backward algorithm gives us exactly what we need: the pairwise marginals . Each call to forward-backward for a single training example produces all the information needed to compute the gradient contribution from that example. This is computationally efficient: rather than running a separate algorithm for inference and a separate algorithm for gradient computation, the forward-backward algorithm serves both purposes simultaneously.
def compute_gradient(
log_potentials, transition_scores, true_labels, log_alpha, log_beta, log_Z
):
"""
Compute gradient of log-likelihood for one training example.
Returns gradients for emission potentials and transition scores.
"""
T, num_labels = log_potentials.shape
# Gradient for emission scores
emission_grad = np.zeros_like(log_potentials)
# Observed counts: feature fires at correct positions
for t, label in enumerate(true_labels):
emission_grad[t, label] += 1.0
# Expected counts: subtract marginal probabilities
marginals = np.exp(log_alpha + log_beta - log_Z)
emission_grad -= marginals
# Gradient for transition scores
transition_grad = np.zeros_like(transition_scores)
# Observed transition counts
for t in range(1, len(true_labels)):
prev_label, curr_label = true_labels[t - 1], true_labels[t]
transition_grad[prev_label, curr_label] += 1.0
# Expected transition counts
for t in range(1, T):
for i in range(num_labels):
for j in range(num_labels):
log_prob = (
log_alpha[t - 1, i]
+ transition_scores[i, j]
+ log_potentials[t, j]
+ log_beta[t, j]
- log_Z
)
transition_grad[i, j] -= np.exp(log_prob)
return emission_grad, transition_grad# Example: gradient computation
true_labels = [0, 1, 2, 2, 0] # O, B-PER, I-PER, I-PER, O
emission_grad, transition_grad = compute_gradient(
log_potentials, transition_scores, true_labels, log_alpha, log_beta, log_Z
)Gradient Analysis ================================================== True labels: ['O', 'B-PER', 'I-PER', 'I-PER', 'O'] Emission Gradient (observed - expected): -------------------------------------------------- Position O B-PER I-PER -------------------------------------------------- 1 +0.7892 -0.3561 -0.4332 ← true: O 2 -0.5345 +0.8472 -0.3127 ← true: B-PER 3 -0.3839 -0.5196 +0.9035 ← true: I-PER 4 -0.3917 -0.1740 +0.5657 ← true: I-PER 5 +0.2055 -0.0880 -0.1175 ← true: O Transition Gradient (sample): -------------------------------------------------- O → O: -1.0741 O → B-PER: +0.5573 B-PER → O: -0.2358 B-PER → B-PER: -0.0744 B-PER → I-PER: +0.1078 I-PER → O: +0.2053 I-PER → B-PER: -0.4172 I-PER → I-PER: +0.9352
The gradient reveals which adjustments would improve the model. Positive gradients at the true labels mean the model should increase those scores. Negative gradients at incorrect labels mean those should be suppressed. The transition gradient shows similar patterns: transitions that occurred in the true sequence but have low model probability will have positive gradients.
In practice, you will often visualize gradient magnitudes during training to diagnose convergence problems. A gradient norm that decreases steadily toward zero indicates smooth convergence. A gradient norm that oscillates or increases may indicate a learning rate that is too large (for gradient descent) or a numerical issue in the forward-backward computation. A gradient norm that plateaus above zero for many iterations suggests the model has hit a flat region and may benefit from better initialization or regularization.
One subtlety worth understanding: the gradient computation above is for a single training example. In practice, we sum gradients across all training examples and optionally normalize by the number of examples. When computing the gradient over the full training set, you are essentially comparing the model's expected feature frequencies with the empirical feature frequencies in the corpus. The model is trying to learn a distribution that matches the corpus statistics exactly, subject to the constraints imposed by the feature set and the regularization penalty.
