Code Understanding › Code Review

Michael BrenndoerferFebruary 2, 20265 min read
Language AI HandbookAll chapters
Part 47 of 67Code Generation
Chapter 358Code Understanding › Code Review
Reading section 7 of 13 · Code Review · 5 minSection 7 of 13 · 5 min

Book contents

Language AI Handbook67 parts · 513 chaptersView the whole book →
Active chapter 358 of 513

Section 7 of 13 · 5 min read

Code Understanding › Code ReviewOne section · Sections in this chapter7 / 13 ↓
Sections in this chapter13
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Code review combines explanation and bug detection into a higher-level task: given a diff or a change set, produce a structured assessment of code quality, correctness, style, and maintainability.

What Code Review Covers

Professional code review addresses multiple concerns simultaneously. The distinction from bug detection is important: bug detection asks a binary question (is there a defect?) while code review asks a qualitative question (is this code good, and if not, what should be improved?). This makes code review more open-ended and harder to evaluate, because quality is not binary.

  • Correctness: Does the code implement the intended behavior? Does it handle edge cases?
  • Readability: Is the code understandable? Are names descriptive? Is the structure clear?
  • Maintainability: Is the code organized in a way that will be easy to modify? Is complexity justified?
  • Style: Does the code follow project conventions and language idioms?
  • Security: Does the change introduce vulnerabilities?
  • Performance: Does the change introduce unnecessary computational cost?
  • Test coverage: Does the change include tests? Do the tests cover the important cases?

Human reviewers balance these concerns together and interact with the author in a dialogue that spans multiple rounds. Models currently approximate the first pass of this process: flagging specific issues and suggesting improvements, without the iterative dialogue.

The History of Automated Code Review

The idea of automated code quality feedback predates language models by decades. lint, the original C code checker released in 1978, analyzed source code for suspicious constructs that were legal but likely bugs. Tools like Checkstyle, PMD, and FindBugs followed in the Java ecosystem. These tools operate on hard-coded rules: "methods should not exceed 50 lines," "catch blocks should not be empty," "assignments inside conditionals are suspicious."

Rule-based tools scale well and have predictable behavior, but they address a fixed set of quality dimensions and cannot adapt to project-specific conventions. Language models offer a different capability: they have been trained on human-written code reviews, so they have learned what human reviewers consider important, including the context-specific judgments that are hard to encode as rules.

The CodeReviewer model from Microsoft (Li et al., 2022) was fine-tuned on 80,000 real code review comments from GitHub pull requests. In human evaluations, reviewers judged CodeReviewer-generated comments as useful in approximately 60% of cases, compared to 35% for a strong baseline. This is a meaningful improvement, though the 60% rate also shows how much room for improvement remains.

Review Quality: Actionable vs. Generic

The most important dimension of review quality is actionability. A comment that says "this function is complex" is not useful; a comment that says "this function has four nested conditional branches and could be simplified by extracting the inner branch into a helper function check_validity(x)" is useful.

Language models tend to produce generic comments when the prompt is underspecified. Training on human-written review comments from GitHub pull requests substantially improves specificity, because the model learns the style and precision of expert human reviewers. Research on the CodeReviewer model (Li et al., 2022) showed that fine-tuning on code review data produces reviews that human evaluators judge as significantly more useful than zero-shot prompted reviews.

The challenge is that review quality depends on context that is often absent from a single diff: project conventions, the purpose of the change, the history of the codebase, and the skill level of the author. A function that would be over-engineered in one context might be exactly right in another.

Worked Example: Reviewing a Python Change

Consider a pull request that adds a new utility function:

In[14]:
Code
# Before (empty)

# After:
def process(items, threshold=0.5):
    res = []
    for i in items:
        if i > threshold:
            res.append(i * 2)
        else:
            res.append(i)
    return res

A surface-level review might note: "The function uses a loop that could be replaced with a list comprehension." That is correct but minor.

A deeper review would note:

  1. The function name process and parameter items are too generic: what is being processed, and what are the items?
  2. The docstring is missing: the caller cannot know what the function does without reading the implementation.
  3. The function does two different things to items above and below the threshold, and naming these paths explicitly would improve clarity.
  4. If the input contains negative values, i * 2 on a negative will produce a more negative value, which may or may not be intended.
  5. Type annotations are absent. For a utility called from many places, type hints like def process(items: list[float], threshold: float = 0.5) -> list[float]: would improve IDE support and error detection.

This review is more useful because it addresses both syntax and semantics, and it asks questions that help the author think through the design rather than just telling them to fix formatting. A language model producing this kind of review can accelerate the engineering process.