Preference Evaluation › Preference Aggregation and Ranking

When you have collected thousands of pairwise comparisons across dozens of models, you need reliable methods to aggregate these into a final ranking. The challenge is that online Elo updates, while computationally convenient, are sensitive to the order in which matches are processed. A model that faces its strongest competitors early in the sequence will receive lower ratings than one that faces them later, even if both eventually participate in the same number of matches. Batch methods that process all comparisons simultaneously avoid this path dependence.

Rather than iteratively updating Elo ratings, you can fit the Bradley-Terry model directly using maximum likelihood estimation. Given a comparison matrix $W$ where $W_{ij}$ counts wins of model $i$ over model $j$, the log-likelihood is:

$$\mathcal{L}(\beta) = \sum_{i < j} \left[ W_{ij} \ln \frac{e^{\beta_i}}{e^{\beta_i} + e^{\beta_j}} + W_{ji} \ln \frac{e^{\beta_j}}{e^{\beta_i} + e^{\beta_j}} \right]$$

where:

• $\mathcal{L}(\beta)$: the log-likelihood function of the Bradley-Terry model parameters $\beta$
• $\beta$: the vector of strength parameters $[\beta_1, \beta_2, ..., \beta_K]$ for all $K$ models
• $W_{ij}$: the number of times model $i$ defeated model $j$ (element of comparison matrix $W$)
• $W_{ji}$: the number of times model $j$ defeated model $i$
• $\ln$: the natural logarithm
• The summation $\sum_{i < j}$ iterates over all unique pairs of models where $i$ is less than $j$

You maximize this subject to the constraint $\sum_i \beta_i = 0$ to fix the scale. This constraint is necessary because adding a constant to all $\beta$ values leaves the probabilities unchanged; we need an anchor to make the parameters identifiable. This approach is more statistically efficient than online Elo updates, especially when comparison graphs are sparse (some models never play each other directly). Maximum likelihood estimation uses all available information simultaneously rather than processing matches sequentially, yielding estimates with lower variance when the model assumptions hold.

The MLE approach also produces natural measures of uncertainty through the Fisher information matrix. The curvature of the log-likelihood at its maximum tells us how precisely the parameters have been estimated: sharp curvature implies high precision, while flat curvature indicates that the data are consistent with a wide range of parameter values. This uncertainty quantification is directly interpretable as confidence intervals around each model's rating. This gives the statistical infrastructure needed to determine which differences in rankings are reliable and which are within the margin of error.

A practical challenge arises when the comparison graph is disconnected. If Group A models only ever compared against Group A, and Group B models only against Group B, you cannot determine whether Group A or Group B is stronger. Mathematically, the likelihood becomes flat in directions that shift mass between disconnected components, making the parameters unidentifiable. This requires anchor models: reference systems that bridge different communities of models by playing against all of them.

In practice, strong baseline models like GPT-4 or Claude act as universal anchors. This gives the connective tissue that allows Elo ratings to be comparable across the entire ecosystem. Without such anchors, you might find that Model X has a higher rating than Model Y, but this could merely reflect that X faced weaker competition within its isolated subgroup. The inclusion of widely-evaluated anchor models ensures that the comparison graph remains connected. This allows the ranking algorithm to establish a global scale rather than merely local orderings.

Graph connectivity affects both the technical analysis and measurement validity. Imagine comparing two national chess leagues without any players who competed in both. Any ranking that spans both leagues is purely speculative. The same logic applies to LLM evaluation. If your comparison graph has disconnected subgraphs, any ranking that places models from different subgraphs relative to each other is statistically unjustified. This is why evaluation systems like LMSYS Chatbot Arena are careful to include widely-compared models that serve as reference points across the entire comparison graph.

Regularization gives an alternative approach to the disconnected graph problem. Instead of requiring connectivity, you can add a prior over the $\beta$ parameters that shrinks them toward zero. This prevents any model's rating from diverging to infinity when it has a perfect record (which would happen with the unregularized MLE when a model wins every comparison), and it implicitly connects all models through the shared prior. The regularized objective is:

$$\mathcal{L}_\text{reg}(\beta) = \mathcal{L}(\beta) - \lambda \sum_i \beta_i^2$$

where $\lambda$ controls the strength of the regularization. Larger $\lambda$ produces stronger shrinkage toward equal ratings, while smaller $\lambda$ allows ratings to spread more freely. The choice of $\lambda$ can be made via cross-validation or by treating it as a hyperparameter that shows prior beliefs about how much the models differ in quality.

Random pairing is statistically inefficient. If Model A has a 1500 rating and Model B has 1000, the outcome is nearly deterministic. This gives little information. The variance of the rating update is proportional to $E_A(1-E_A)$, which approaches zero when one model is heavily favored. Active sampling strategies pair models with similar ratings or high uncertainty, maximizing the information gained per comparison.

Thompson sampling or Bayesian optimization approaches can select matchups that most reduce uncertainty in the global ranking, cutting the number of required comparisons by 50% or more compared to random pairing. These methods maintain a posterior distribution over possible rating configurations and select the matchup with the highest expected information gain. For example, if Model C and Model D have similar ratings but high uncertainty, a match between them gives more information about their relative ordering than a match between a certain champion and a certain loser. This adaptive approach is particularly useful when evaluation budgets are limited, letting precise ranking with fewer expensive human judgments.

The information-theoretic framing of active sampling reveals a deep connection to the design of scientific experiments more broadly. In both settings, the goal is to select observations that maximally reduce uncertainty about quantities of interest. For Elo-based evaluation, the quantity of interest is the vector of ratings $\beta$, and the optimal sampling strategy is the one that minimizes the posterior entropy after observing the new comparison result. Greedy approaches that approximate this criterion by selecting the matchup with the highest expected prediction error are easy to implement and perform well in practice.