Load Balancing › Worked Example: Tracking Metrics Through Collapse

Let's walk through a concrete numerical example to solidify your understanding of how these metrics evolve together. We'll trace the values at three snapshots through a collapse simulation: the beginning, the middle, and the end.

Suppose we start with an 8-expert system at step 0, where all experts receive exactly uniform token fractions of 0.125. At this point, the metrics are:

• Token fractions: each $f_i = 0.125$
• LIF: $8 \times 0.125 = 1.0$ (perfect balance)
• CV: $\sigma(f) = 0$, so $\text{CV} = 0$ (no variation)
• Normalized entropy: $-8 \times (0.125 \times \log 0.125) / \log 8 = 1.0$ (maximum)

Now suppose we reach step 50, where the feedback loop has been running with moderate strength. Expert 0 now receives 0.25 of tokens, experts 1 and 2 receive 0.15 each, experts 3-6 receive 0.10 each, and expert 7 receives 0.05. Let's compute:

• LIF: $8 \times 0.25 = 2.0$ (most loaded expert has twice its fair share)
• Mean: $1/8 = 0.125$, Standard deviation $\approx 0.063$, so $\text{CV} \approx 0.063 / 0.125 = 0.504$
• Entropy requires summing $-f_i \log f_i$ for each expert; this gives a normalized entropy around 0.91, showing the distribution is noticeably but not critically skewed

Finally at step 200, after strong reinforcement, expert 0 receives 0.70 of all tokens, expert 1 receives 0.25, and the remaining 6 experts share 0.05 total (approximately 0.008 each). The metrics now read:

• LIF: $8 \times 0.70 = 5.6$ (expert 0 receives 5.6 times its fair share)
• CV: very high, around 2.0 or above, which reflects massive spread
• Normalized entropy: close to 0.3, far below the maximum, showing highly concentrated routing

This progression illustrates the value of monitoring all three metrics simultaneously. The LIF gives you the severity of the worst-case situation. The CV reveals whether the problem is concentrated (one dominant expert) or diffuse (many experts off their ideal load). The entropy provides the earliest warning, declining first and most sensitively. In practice, you would set alerting thresholds: for example, flag a potential problem when LIF exceeds 1.5 or normalized entropy falls below 0.85, and intervene when LIF exceeds 2.0 or entropy falls below 0.7.