Bradley-Terry Model › Preference Strength and Transitivity

A important property of the Bradley-Terry model is that it produces transitive preferences. Transitivity means that if A is better than B, and B is better than C, then A must be better than C. If we know how often A beats B, and how often B beats C, the model automatically determines how often A beats C. This follows directly from the parameterization, and understanding why reveals something deep about the model's structure. Think of transitivity as the model insisting that quality lives on a one-dimensional line: if A is to the right of B, and B is to the right of C, then A must be to the right of C. There is no room for circular orderings in a linear arrangement.

The transitivity arises because each item has a single strength parameter that governs all of its comparisons. When we estimate $\beta_A$, $\beta_B$, and $\beta_C$, these values must simultaneously explain the A-versus-B comparisons, the B-versus-C comparisons, and the A-versus-C comparisons. The model cannot assign different "strengths" to A depending on which item it is facing. This constraint forces global consistency. Every comparison provides information that constrains all the parameters jointly, and the maximum likelihood estimates balance all this information to find the single assignment of positions on the number line that best explains everything we observed.

This global consistency is what makes the model so powerful for synthesizing large amounts of comparison data. With $n$ items, there are $\binom{n}{2}$ possible pairwise matchups, but only $n-1$ free parameters (since one parameter is the anchor). The model over-constrains the problem: we use many more observations than parameters, and the maximum likelihood estimate finds the parameter values that best reconcile all these constraints simultaneously. This over-determination is a feature, not a bug. It means that comparisons between pairs we have observed well can inform predictions about pairs we have observed rarely, through the chain of transitivity.

Consider three items with strengths $\beta_A = 2$, $\beta_B = 1$, and $\beta_C = 0$. The pairwise probabilities are:

where:

• $\beta_A, \beta_B, \beta_C$: the example strengths ($2, 1, 0$)
• $P(A \succ B)$: the probability A beats B
• $\sigma(\cdot)$: the sigmoid function applied to strength differences

Notice that even though A and B have the same preference probability over their respective "next" item (both beat their weaker neighbor about 73% of the time), A has a much higher probability of beating C than B does of beating C. The strengths compose properly: the gap between A and C is the sum of the gaps between A and B and between B and C. Specifically, $\beta_A - \beta_C = (\beta_A - \beta_B) + (\beta_B - \beta_C) = 1 + 1 = 2$. This additive property of log-strengths is what makes the model so elegant. Strength differences accumulate predictably across chains of comparisons. The model implicitly answers the question "If A usually beats B and B usually beats C, how often should A beat C?" without needing us to collect A-versus-C comparisons directly.

This transitivity is a feature, not a bug. It forces the model to find a consistent ranking that best explains all observed comparisons, even when individual comparisons contain noise or contradictions. If humans sometimes prefer B over A, the model can still conclude A is better overall if A wins more often than it loses. The model essentially averages over the noise to find the underlying signal. When we observe an intransitivity in the data (say, A beats B 70% of the time, B beats C 70% of the time, but C beats A 55% of the time), the model will not try to represent this cycle explicitly. Instead, it will find the one-dimensional ordering that minimizes the total surprise across all observed outcomes. This averaging behavior is highly desirable for LLM evaluation: no single human comparison should be treated as ground truth, but the aggregate pattern across thousands of comparisons should be trusted.

However, as we will discuss in the Limitations section, this built-in transitivity can also be a constraint when human preferences violate the one-dimensional ordering assumption. Different annotators may prefer different attributes of responses, and these multi-dimensional preferences can produce systematic intransitivities that a Bradley-Terry model cannot capture. For now, it is enough to note that transitivity is both what makes the model tractable and what limits its expressiveness.