Synthetic Data › Quality Verification

Generating synthetic data is cheap. Generating good synthetic data requires systematic quality verification. Models hallucinate, produce vague responses, copy instructions verbatim into answers, generate structurally malformed outputs, and occasionally produce fluent but completely wrong answers. Without quality filters, these examples enter the training set and degrade the student model. The quality verification layer is where a synthetic data pipeline earns its value.

Quality verification is a multi-stage funnel. Each stage catches a different class of failures. The stages are ordered by cost: fast rule-based filters first, slower model-based scoring second, and expensive consistency verification last. This ordering ensures that only examples that pass cheap filters proceed to expensive evaluation, keeping the overall pipeline cost manageable.

The first line of defense is a set of fast, cheap rule-based filters. These catch obvious quality failures without requiring model calls:

• Length filters: Remove responses that are too short (a one-word answer to a detailed question) or too long (runaway generation exceeding a reasonable token budget for the task type)
• Repetition filters: Detect n-gram repetition within a response, since degenerate generation often produces repeated phrases or entire paragraphs as the model loops
• Format filters: Verify that structured outputs (JSON, code, markdown tables) parse correctly and conform to the required schema
• Keyword filters: Flag responses containing phrases that indicate refusal ("As an AI language model, I cannot...") when the task does not require refusal
• Language ID filters: Verify the response is in the target language, using a lightweight language identification library
• Instruction-response length ratio filters: Catch cases where the response is much shorter than the instruction would imply necessary, which often indicates a non-answer

These filters are not perfect but they are fast. Running them on every generated example before any model-based scoring saves significant compute. In practice, a well-tuned rule-based filter removes 15-30% of generated examples at essentially no cost per example, compared to the per-token cost of a model-based scoring call.

One important implementation detail is to make rule-based filters configurable per task type. For a creative writing task, you want to allow longer responses. For a code completion task, the length ratio between instruction and response may be inverted. A general pipeline that applies the same filters uniformly to all task types will either be too restrictive for some tasks or too permissive for others.

After rule-based filtering, model-based scoring evaluates the semantic quality of responses. The most common approach is to prompt a strong judge model (often the same teacher that generated the data, or a more capable model) to score each example on a numerical scale.

A typical scoring prompt asks the judge to evaluate the response along several dimensions:

• Accuracy: Does the answer correctly address the question?
• Completeness: Does the response cover all aspects of the request?
• Relevance: Is the content directly relevant to the instruction, without unnecessary tangents?
• Coherence: Is the response logically consistent and well-structured?
• Helpfulness: Would a person asking this question be satisfied with this response?

The judge returns a score (often on a 1-5 or 1-10 scale) along with a brief explanation. Examples below a threshold are discarded. The explanation helps with debugging and threshold calibration: if the judge consistently explains that 4/5 scores correspond to minor omissions while 3/5 scores correspond to factual errors, you can use that information to set a threshold that matches your tolerance for different failure types.

A practical consideration when using model-based scoring is position bias: judge models tend to rate responses that use more confident, structured language more highly, independent of factual accuracy. If you generate ten responses to the same question and the one rated highest is the most verbose and well-formatted but not the most accurate, you are training the student to imitate a style rather than to be correct. Cross-referencing model-based scores with rejection sampling results, where available, helps detect this bias.

The choice of judge model also matters significantly. Using the same model as the teacher introduces correlated errors: the teacher's systematic biases will be rated favorably by itself. Using a stronger or independently trained model as judge reduces this correlation, at higher per-call cost. A practical middle ground is to use the teacher model for initial filtering and bring in a stronger judge for borderline examples near the threshold.

When generating instruction-following data with explicit format constraints ("respond in JSON", "use bullet points", "keep your answer under 100 words"), you can verify format compliance programmatically. The IFEval benchmark evaluates models on instructions with verifiable format constraints, and the same verification functions can be used to filter training data.

For example, if a generated example has the instruction "Respond only with a numbered list", you can check:

• Does the response contain numbered items?
• Does the response start with a number, not a sentence?
• Does it avoid prose paragraphs entirely?

Format verification provides a hard filter that complements the soft scores from model-based judging. It is particularly valuable when you want to train a model to follow explicit format instructions reliably, since format compliance is fully verifiable without any model call.

The same principle extends to many other structural constraints: word count limits (count words), response structure (check for required sections), code syntax (parse with AST), output in a specific language (language ID), presence of citations (pattern match for URLs or reference formats), and so on. Every constraint in your instruction set that is mechanically verifiable should have a corresponding filter function in your pipeline.

A trained reward model provides a learned quality signal rather than a prompted one. Reward models are trained on human preference data: pairs of responses where a human has labeled which is better. The reward model learns to predict human preference, and this learned function can score individual examples without pairwise comparison.

Using a reward model for synthetic data filtering has two advantages. First, it is faster than calling a large judge model, since the reward model is typically much smaller and requires no elaborate prompting. Second, it captures quality dimensions that are hard to articulate in a prompt, because they were learned implicitly from thousands of human judgments rather than specified by a single prompt author.

The limitation is that a reward model can overfit to surface features that humans associate with quality but that do not reflect correctness. Responses that are verbose, politely formatted, and carefully structured may score highly even when they are factually wrong. This is the reward model version of the same position bias problem described above.

For factual questions, you can filter for consistency by generating multiple responses to the same question and checking agreement. If most responses agree on the answer, that consensus provides evidence for correctness. Responses that contradict the consensus are discarded.

This approach is analogous to self-consistency decoding (majority voting at inference time) applied to data generation. If you generate eight responses and six agree that the answer to a historical question is a particular year while two give different years, the majority response has higher evidence for correctness than any single response would. The consistency signal is particularly useful for math and factual knowledge, where there is a deterministic correct answer, but it works surprisingly well for open-ended questions too: consistent responses tend to reflect the model's well-grounded beliefs rather than hallucinated one-offs.

Consistency filtering requires multiple generation passes per example, which multiplies the API cost. A practical approach is to reserve consistency filtering for high-stakes subsets: factual knowledge about specific entities, dates, quantities, and causal claims, where a hallucinated answer is most damaging if it enters the training data.