Evaluation Prompt Engineering: Designing Reliable LLM Judges › Calibration and Score Distribution Analysis

Beyond the design of individual prompts, a well-maintained evaluation pipeline requires ongoing monitoring of score distributions. Even a prompt that was well-calibrated at launch can drift over time as the distribution of model responses shifts. Tracking score distributions is the primary tool for detecting this drift before it silently degrades your evaluation quality.

A score distribution tells you how often your judge assigns each possible score. An ideal distribution for a 1-5 scale should be roughly bell-shaped if the responses you are evaluating vary in quality, or right-skewed if your model is generally performing well. What you do not want to see is a distribution that is heavily concentrated at one end of the scale, because that concentration destroys your ability to use the scale to discriminate between responses.

The most common pathological distribution in LLM evaluation is leniency bias: the judge consistently assigns scores in the range of 4-5 on a 1-5 scale, regardless of response quality. Leniency bias has several causes. The most common is that the high end of the scale is underspecified: if "5 = excellent" is the only anchor at the top, the judge fills in the gap with its prior about what "excellent" means, and its prior often includes fluent, confident-sounding responses, which are overrepresented in its training data. Adding a concrete description of what a 5 looks like, including what it does not include, usually corrects this.

Severity bias is less common but occurs with prompts that emphasize flaw detection. If your evaluation prompt asks the judge to identify weaknesses in the response, it may apply a higher implicit standard than intended, clustering scores in the 1-3 range. This is why task description framing matters: "assess the quality" and "find flaws" produce systematically different score distributions from the same judge on the same responses.

A practical monitoring approach is to maintain a calibration set: a fixed collection of (response, human verdict) pairs that you evaluate with your judge at regular intervals. By tracking the judge's score distribution on this fixed set over time, you can detect drift independently of changes in the model outputs you are evaluating. If the judge's scores on the calibration set shift without any change to the evaluation prompt, something has changed in the judge model itself (due to an update, a change in serving infrastructure, or context window effects), and you need to investigate.

For pipelines that evaluate models at multiple stages of development, calibration set management is especially important. The responses from an early-stage model and a late-stage model may be so different in quality that a judge calibrated on early-stage responses is effectively measuring a different dimension when applied to late-stage responses. Re-calibrating the judge (updating the few-shot examples and scale anchors) when the model population shifts significantly is good practice.