Bradley-Terry Model › Connection to Elo Ratings

If the Bradley-Terry model sounds familiar, it should. The Elo rating system, used to rank chess players and now common in competitive gaming and language model evaluation, is a Bradley-Terry model in disguise. Understanding this connection illuminates both systems and explains why Elo ratings have become standard for comparing language models on public leaderboards.

The Elo system was designed for practical use in tournament settings where ratings need to be updated incrementally after each game rather than recomputed from scratch with every new observation. Rather than running a global optimization over all historical comparisons, Elo uses a simple online update rule: after each game, the winner gains points and the loser loses points, with the amount proportional to how surprising the outcome was given the current ratings. If the expected winner wins, little rating change occurs. If the underdog wins, a large rating transfer happens. This online update rule is an approximation to the maximum likelihood gradient, making Elo an efficient algorithm for the same underlying Bradley-Terry model.

Elo ratings are typically scaled so that a 400-point difference corresponds to a 10:1 expected score ratio. This scaling was chosen for historical and practical reasons: it produces ratings that are easy to interpret and compare. A rating difference of 400 points means the stronger player is expected to score about 10 points for every 1 point the weaker player scores over many games.

In standard Elo, the expected score of player $A$ against player $B$ is:

$$E_A = \frac{1}{1 + 10^{(R_B - R_A)/400}}$$

where:

• $E_A$: the expected score (win probability) of player $A$
• $R_A, R_B$: the Elo ratings of players $A$ and $B$
• $400$: the scaling factor where a difference of 400 points implies a 10:1 odds ratio

The use of base-10 exponentials rather than natural exponentials is purely conventional: Arpad Elo chose this scaling when he developed the system in the 1960s because it produced convenient numbers for chess ratings. The mathematical structure is identical to Bradley-Terry; only the scaling differs.

We can rewrite this in terms of the sigmoid function by converting the base-10 exponential to base-$e$:

$$\begin{aligned} E_A &= \frac{1}{1 + 10^{(R_B - R_A)/400}} \\ &= \frac{1}{1 + e^{\ln(10) \cdot (R_B - R_A)/400}} && \text{(change of base: } 10^x = e^{x \ln 10} \text{)} \\ &= \frac{1}{1 + e^{-\frac{\ln 10}{400} (R_A - R_B)}} && \text{(rearrange exponent)} \\ &= \sigma\left( \frac{\ln 10}{400} (R_A - R_B) \right) && \text{(definition of sigmoid)} \end{aligned}$$

where:

• $E_A$: the expected score (win probability) for player $A$
• $\ln 10 \approx 2.30$: the natural logarithm of 10
• $\sigma(\cdot)$: the standard sigmoid function

This derivation demonstrates that the Elo rating is just a scaled version of the Bradley-Terry log-strength parameter. Specifically, if $\beta$ is the Bradley-Terry parameter, then the corresponding Elo rating is $R = \frac{400}{\ln 10} \beta \approx 173.7 \beta$. The mathematical structure is identical; only the units differ. Every Elo system is a Bradley-Terry model, and every Bradley-Terry model can be expressed as an Elo system with a particular scaling convention.

The connection matters because Elo-style ratings are now used to evaluate LLMs. Platforms like LMSYS Chatbot Arena use Elo ratings computed from human preference comparisons to rank models. When you vote for which model response you prefer, these votes become pairwise comparisons that update the models' Elo ratings. The Bradley-Terry framework provides the statistical foundation for these rankings. This keeps the ratings are consistent and interpretable. A model with a 200-point Elo advantage over another is predicted to win head-to-head comparisons about 76% of the time: a concrete, actionable interpretation that pure benchmark scores on fixed datasets cannot provide.

The Elo system also demonstrates an important practical consideration: rating systems must handle the problem of new entrants. When a new language model joins the arena, we have no comparison data for it, so we initialize it at some default rating. As it accumulates comparisons against known models, its rating converges toward the true value. How quickly this convergence happens depends on the K-factor parameter in the Elo update rule, which controls the magnitude of rating changes per comparison. The Bradley-Terry maximum likelihood framework does not have this initialization problem: it estimates all parameters jointly from all available comparisons, so new models with few comparisons get wide uncertainty intervals rather than noisy point estimates.