Bradley-Terry Model › Worked Example

Let's work through a concrete numerical example to solidify these concepts and see how the mathematics plays out with real numbers from start to finish. This will make the abstract framework concrete before we move to the code implementation. Suppose we have three model responses (A, B, C) and the following comparison outcomes from human annotators:

We have 30 total comparisons across three matchups. Looking at the raw data, A beats B 70% of the time, B beats C 80% of the time, and A beats C 90% of the time. These observed frequencies give a rough sense of the ranking (A is best, C is worst), but they are not the Bradley-Terry parameters. The parameters must be inferred by maximum likelihood, and they will differ from the simple observed frequencies because they must be globally consistent.

Step 1: Set the anchor. We set $\beta_C = 0$ as our anchor. This choice is arbitrary, but it resolves the identifiability issue and allows us to interpret the other parameters as "how much better than C" each item is. With this convention, $\beta_A$ and $\beta_B$ become the only free parameters.

Step 2: Write the log-likelihood. The log-likelihood function aggregates all our comparison data:

$$\begin{aligned} \log L(\beta_A, \beta_B) &= 7\log\sigma(\beta_A - \beta_B) + 3\log\sigma(\beta_B - \beta_A) \\ &\quad + 8\log\sigma(\beta_B - 0) + 2\log\sigma(0 - \beta_B) \\ &\quad + 9\log\sigma(\beta_A - 0) + 1\log\sigma(0 - \beta_A) \end{aligned}$$

where:

• $\log L$: the log-likelihood function aggregating all comparisons
• $7, 3, \dots$: the observed counts (e.g., A beats B 7 times)
• $\sigma(\cdot)$: the sigmoid function mapping strength differences to probabilities
• $\beta_A, \beta_B$: the unknown log-strength parameters
• $0$: the fixed anchor value for item C ($\beta_C = 0$)

Each term in this sum corresponds to a set of comparisons. The first term, $7\log\sigma(\beta_A - \beta_B)$, contributes to the likelihood every time A beats B, which happened 7 times. The second term accounts for the 3 times B beat A. Notice how the symmetry of preferences emerges: when A beats B, we use $\sigma(\beta_A - \beta_B)$, and when B beats A, we use $\sigma(\beta_B - \beta_A) = 1 - \sigma(\beta_A - \beta_B)$.

Step 3: Take derivatives. To find the maximum, we take partial derivatives and set them to zero. We rely on the derivative property of the log-sigmoid function: $\frac{d}{dx} \log \sigma(x) = 1 - \sigma(x)$. This elegant result comes from the chain rule and the fact that $\sigma'(x) = \sigma(x)(1-\sigma(x))$, giving $\frac{d}{dx}\log\sigma(x) = \frac{\sigma'(x)}{\sigma(x)} = 1 - \sigma(x)$.

Applying the chain rule to the terms involving $\beta_A$:

1. When A wins (e.g., against B): The term is $\log \sigma(\beta_A - \beta_B)$. The derivative with respect to $\beta_A$ is $1 - \sigma(\beta_A - \beta_B)$.
2. When A loses (e.g., against B): The term is $\log \sigma(\beta_B - \beta_A)$. The derivative with respect to $\beta_A$ is $(1 - \sigma(\beta_B - \beta_A)) \cdot (-1)$, which simplifies to $-\sigma(\beta_A - \beta_B)$.

Substituting the counts into the gradient equation for $\beta_A$:

$$\begin{aligned} \frac{\partial \log L}{\partial \beta_A} &= 7(1 - \sigma(\beta_A - \beta_B)) - 3\sigma(\beta_A - \beta_B) \\ &\quad + 9(1 - \sigma(\beta_A)) - 1\cdot\sigma(\beta_A) \end{aligned}$$

where:

• $\frac{\partial \log L}{\partial \beta_A}$: the gradient of the log-likelihood with respect to $\beta_A$
• $\beta_A, \beta_B$: the log-strength parameters
• $\sigma(\cdot)$: the sigmoid function
• $1 - \sigma(\cdot)$: the derivative of the $\log \sigma(\cdot)$ term for wins
• $-\sigma(\cdot)$: the derivative of the $\log \sigma(\cdot)$ term for losses

Using the property that $1 - \sigma(x) = \sigma(-x)$, we can write this more compactly as:

$$\frac{\partial \log L}{\partial \beta_A} = 7\sigma(\beta_B - \beta_A) - 3\sigma(\beta_A - \beta_B) + 9\sigma(-\beta_A) - \sigma(\beta_A)$$

where:

• $\sigma(\beta_B - \beta_A)$: the probability that B beats A
• $\sigma(-\beta_A)$: the probability that C beats A (equivalent to $1 - \sigma(\beta_A)$)
• $\beta_A, \beta_B$: the log-strength parameters
• $\sigma(\cdot)$: the sigmoid function

There is a beautiful interpretation of this gradient. The first term, $7\sigma(\beta_B - \beta_A)$, is 7 times the probability that B beats A under the current model. The second term, $3\sigma(\beta_A - \beta_B)$, is 3 times the probability that A beats B. At the maximum likelihood solution, the gradient equals zero, which means: the expected number of times B should beat A under the model equals the observed number (3), and the expected number of times A should beat B equals the observed number (7). This is the method of moments interpretation of maximum likelihood for Bradley-Terry: at the optimum, observed win counts equal expected win counts for each item.

Step 4: Solve numerically. The gradient equations don't have a closed-form solution because the sigmoid function is nonlinear. Unlike linear regression, where we can solve for parameters directly, the nonlinearity of the sigmoid means we must use numerical methods. We run gradient ascent (or equivalently, minimize the negative log-likelihood) using an efficient optimizer like L-BFGS. The convexity of the objective guarantees we will find the global maximum.

The true maximum likelihood estimates for our three-item example work out to approximately $\beta_A \approx 2.05$, $\beta_B \approx 0.82$, and $\beta_C = 0$ (fixed). Let's verify these make sense. The gap $\beta_A - \beta_B \approx 1.23$ gives $\sigma(1.23) \approx 0.77$, close to the observed 70% but adjusted upward by the model's use of A-vs-C information. The gap $\beta_B - \beta_C = 0.82$ gives $\sigma(0.82) \approx 0.69$, close to the observed 80% but adjusted downward by the indirect information flowing through A. The Bradley-Terry model reconciles all three matchups simultaneously to find the best global explanation.