Code Generation › A Worked Example: The Median Problem
Language AI HandbookAll chapters Code Generation › A Worked Example: The Median ProblemOne section · Sections in this chapter7 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let us trace through the complete generation process for a concrete example, examining what can go wrong at each stage and how different strategies address those failures. This grounded walkthrough connects the abstract concepts above to the practical reality of code generation.
The problem is to generate a function find_median(numbers) that returns the median of a list of numbers. The median is the middle value when the list is sorted; for even-length lists, it is the average of the two middle values.
Stage 1: Docstring-Driven Generation
A minimal docstring like "Return the median of a list" might produce this output:
def find_median(numbers):
return sorted(numbers)[len(numbers) // 2]This implementation is wrong for even-length lists: find_median([1,2,3,4]) returns 3 instead of 2.5. The model followed the most common pattern for "find the middle element of a sorted list" without accounting for the even-length case, because the docstring did not mention it.
A more complete docstring produces better results:
def find_median(numbers):
"""
Return the median of a list of numbers.
For odd-length lists, return the middle value.
For even-length lists, return the average of the two middle values.
"""
sorted_nums = sorted(numbers)
n = len(sorted_nums)
mid = n // 2
if n % 2 == 0:
return (sorted_nums[mid - 1] + sorted_nums[mid]) / 2
return sorted_nums[mid]The complete docstring forced the model to address the even-length case explicitly.
Stage 2: Test-Driven Repair
Even the improved version may not handle the empty list case. Suppose the caller adds:
assert find_median([]) == 0 # or raises ValueErrorAnd the current implementation raises IndexError: list index out of range. Running this test and feeding the error to the model triggers a repair:
def find_median(numbers):
if not numbers:
raise ValueError("Cannot compute median of empty list")
sorted_nums = sorted(numbers)
n = len(sorted_nums)
mid = n // 2
if n % 2 == 0:
return (sorted_nums[mid - 1] + sorted_nums[mid]) / 2
return sorted_nums[mid]The repair added explicit empty-list handling. The error message was precise enough to tell the model exactly what failed, and the model correctly chose to raise ValueError rather than return a default value, following Python convention.
Stage 3: Best-of-N Selection
For a problem this simple, best-of-N is not strictly necessary. But consider what happens if the model generates multiple candidates at temperature 0.5:
- Candidate 1: Correct implementation using
sorted() - Candidate 2: Incorrect implementation using
list.sort()in-place that breaks the original list - Candidate 3: Correct implementation using
statistics.median() - Candidate 4: Correct implementation with explicit type conversion to
float - Candidate 5: Incorrect implementation that returns the wrong middle index
Running tests on all five candidates immediately identifies candidates 2 and 5 as incorrect and selects from the correct alternatives. The selection criterion is test execution, which takes milliseconds and requires no human judgment. This example illustrates the core value of best-of-N: even when individual generation quality is imperfect, generating multiple candidates and selecting with a reliable oracle produces high-quality output efficiently.