Text Generation Applications › Evaluation Patterns for Production Systems

Production text generation requires automated evaluation to catch quality regressions and constraint violations at scale. Evaluation is not a one-time activity at deployment; it is a continuous monitoring function. Models are updated, prompts drift, and the distribution of user inputs changes over time. A quality regression that would have been caught at launch may develop gradually and go unnoticed without ongoing measurement.

The most effective production evaluation systems layer multiple complementary signals. No single signal is sufficient. Hard constraint checks catch violations but say nothing about quality within the constraints. Reference-based metrics measure similarity to known-good outputs but cannot detect valid outputs that happen to be phrased differently. LLM judges evaluate overall quality but introduce their own biases. Human evaluation is the gold standard but is expensive and slow. The art of production evaluation design is combining these signals in a cost-effective way.

Reference-based metrics compare generated text against one or more human-written references. BLEU and ROUGE, covered in the Evaluation Fundamentals chapters, are the most common. They measure n-gram overlap and work well when there is an accepted reference (e.g., translation, summarization). They work poorly for open-ended generation where many valid outputs exist.

BLEU was designed for machine translation, where the task has a clearly defined correct output and deviations from it are meaningful. For content creation tasks where creativity and novelty are valued, a BLEU score penalizes outputs that use different but equally valid phrasing. This makes BLEU a poor choice as a primary metric for most content generation evaluation. However, BLEU remains useful as a regression-detection metric: if the BLEU score for a generation pipeline drops significantly from baseline, something has changed in the model or prompt, and investigation is warranted.

BERTScore is a more semantically sensitive alternative to BLEU that uses contextual embeddings to compute similarity. It can recognize paraphrases that BLEU would penalize, making it better-suited to content generation evaluation. The trade-off is computational cost: BERTScore requires running a BERT-sized model on every output, which is significantly more expensive than n-gram overlap computation.

A strong language model can evaluate generated text by reading the output and answering specific questions about quality dimensions. This is now the standard approach for evaluating instruction-following quality. The pattern is: generate outputs, send each to an evaluation model with a scoring rubric, aggregate scores. The primary caveat is that judge models have their own biases, particularly toward outputs that are verbose and confident.

The design of the scoring rubric is the most important engineering decision in an LLM-as-judge setup. A rubric that asks "Rate the quality of this text from 1 to 10" is too vague to produce reliable scores. A rubric that asks specific questions ("Does the text include the product's battery life? Yes or No. Is the text under 120 words? Yes or No. Does the text avoid making claims not supported by the provided attributes? Rate 1 to 5.") produces more reliable and interpretable results.

Judge model bias is a real concern. Models are known to prefer outputs from models similar to themselves, to prefer longer outputs to shorter ones of equivalent quality, and to give inflated scores to outputs that use confident language. The standard mitigation is to validate judge scores against human judgments on a calibration set, and to use multiple judge models and aggregate their scores rather than trusting a single judge.

Hard constraints (word count, required inclusions, forbidden content) should always be checked programmatically. These checks run quickly and reliably while catching a significant fraction of output failures. The constraint checker we built above is a simplified version of what production systems use.

The programmatic constraint-checking layer should be treated as a gate, not a score. Outputs that fail hard constraints are rejected, not downgraded. This reflects the business reality that constraint violations often have qualitative implications that scoring cannot capture: an email draft that mentions a competitor's product when forbidden is a brand safety violation regardless of how well-written the rest of the email is.

For high-stakes applications (legal documents, medical content, financial disclosures), human review remains essential. The statistical properties of human evaluation, including inter-annotator agreement and preference evaluation design, are covered in the Human Evaluation Design and Preference Evaluation chapters (not yet published).

Human evaluation for text generation typically uses either Likert scales (rate from 1 to 5 on each quality dimension) or pairwise preference judgments (which of these two outputs is better?). Pairwise preferences produce more consistent inter-annotator agreement because they are a simpler cognitive task than absolute scale judgment. The Bradley-Terry model and similar pairwise comparison models can convert pairwise preferences into a global quality ranking, which is how most modern preference-based evaluation systems work.

The cost of high-quality human evaluation should not be underestimated. Evaluating 1,000 generated outputs on five quality dimensions at two annotators per output, at a professional annotation rate, costs thousands of dollars. This is why automated evaluation is not optional for production systems: it is the cost-effective primary signal, with human evaluation reserved for calibration, auditing, and high-stakes decisions.