Bradley-Terry Model › The Log-Linear Form

While the ratio formulation is intuitive, a more convenient parameterization uses log-strengths. This transformation opens up powerful connections to logistic regression and makes optimization considerably easier. It also reveals the deep mathematical structure that links Bradley-Terry to the neural network training objectives we will use for reward models. Let $\beta_i = \log(\pi_i)$. Since strengths must be positive, the log-strength can be any real number, which is mathematically more convenient for optimization algorithms that operate over unconstrained real-valued spaces. We can now derive the equivalent expression in terms of these log-strengths:

where:

• $\beta_i, \beta_j$: the log-strength parameters of items $i$ and $j$
• $\pi_i, \pi_j$: the latent strength parameters ($\beta = \log \pi$)
• $\sigma(x)$: the sigmoid function $\frac{1}{1 + e^{-x}}$

The function $\sigma(x)$ is the standard sigmoid familiar from logistic regression and neural network activations (as covered in our chapter on activation functions). The key insight here is that the log transformation converts a ratio of two quantities into a difference of their logs. The win probability depends not on the absolute strengths but on how much larger one log-strength is than the other. A difference of $\beta_i - \beta_j = 1$ means item $i$ is $e \approx 2.72$ times stronger than item $j$, and this corresponds to a win probability of $\sigma(1) \approx 0.73$. A difference of 2 means item $i$ is $e^2 \approx 7.39$ times stronger, corresponding to a win probability of $\sigma(2) \approx 0.88$.

The sigmoid function has several properties that make it ideal for representing probabilities. It maps any real number to the interval $(0, 1)$. This keeps valid probabilities. It is symmetric around zero, meaning $\sigma(-x) = 1 - \sigma(x)$, which captures the natural symmetry of pairwise comparisons: the probability that $j$ beats $i$ equals one minus the probability that $i$ beats $j$. The sigmoid is also smooth and differentiable everywhere, which enables gradient-based optimization. This differentiability is important when the parameters $\beta$ are outputs of a neural network: we can backpropagate gradients through the sigmoid to update the network weights.

Think of the log-strength parameterization as converting the strength problem into a distance problem. Each item is placed on a one-dimensional number line by its log-strength value. The probability that item $i$ beats item $j$ is a function of the signed distance $\beta_i - \beta_j$: positive means $i$ is to the right of $j$ on the line and thus more likely to win, negative means $j$ is more likely to win. The sigmoid converts this signed distance into a probability. The further apart two items are on this line, the more predictable their comparison outcomes become. Items clustered near the same point on the line will have comparison outcomes close to 50/50.

This log-linear form reveals that the Bradley-Terry model is equivalent to logistic regression where the probability of $i$ beating $j$ depends on the difference in their log-strengths. The sigmoid function ensures probabilities stay between 0 and 1, with the midpoint at 0.5 when $\beta_i = \beta_j$. This equivalence to logistic regression lets us use decades of research on efficient optimization algorithms, regularization techniques, and statistical theory about confidence intervals and hypothesis tests. Maximum likelihood estimation for Bradley-Terry is a well-studied convex optimization problem with guaranteed convergence to a global optimum.