Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Code Understanding › Code Implementation: Automated Code Review

Michael BrenndoerferFebruary 2, 20263 min read

Language AI HandbookAll chapters

Part 47 of 67Code Generation

Chapter 358Code Understanding › Code Implementation: Automated Code Review

Reading section 8 of 13 · Code Implementation: Automated Code Review · 3 minSection 8 of 13 · 3 min

Section 8 of 13 · 3 min read

Code Understanding › Code Implementation: Automated Code ReviewOne section · Sections in this chapter8 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's implement automated code review by prompting a code model with a structured review template.

In[15]:

Code

def review_code(code: str, max_new_tokens: int = 200) -> str:
    """Generate a structured code review for the given code snippet."""
    prompt = (
        "# Code Review\n"
        f"# Code to review:\n{code}\n"
        "# Review (issues and suggestions):\n"
        "# 1."
    )
    inputs = tokenizer(prompt, return_tensors="pt")
    with torch.no_grad():
        outputs = model_explain.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.3,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id,
        )
    generated = tokenizer.decode(outputs[0], skip_special_tokens=True)
    review_start = generated.find("# Review (issues and suggestions):\n") + len(
        "# Review (issues and suggestions):\n"
    )
    review_text = generated[review_start:].strip()
    lines = [l for l in review_text.split("\n") if l.strip().startswith("#")][
        :5
    ]
    return "\n".join(lines)

In[16]:

Code

code_to_review = """\
def process(items, threshold=0.5):
    res = []
    for i in items:
        if i > threshold:
            res.append(i * 2)
        else:
            res.append(i)
    return res"""

review_output = review_code(code_to_review)

Out[17]:

Console

Generated Code Review:
# 1. The code should be able to handle the case when the number of items is less than the threshold
# 2. The code should be able to handle the case when the number of items is greater than the threshold
# 3. The code should be able to handle the case when the number of items is less than the threshold
# 4. The code should be able to handle the case when the number of items is greater than the threshold
# 5. The code should be able to handle the case when the number of items is less than the threshold

The quality of zero-shot review varies substantially with model size and instruction-following capability. For production review tooling, instruction-tuned models prompted with explicit criteria (naming, documentation, type hints, complexity) and fine-tuned on curated review datasets produce more consistent, actionable output.

Key Parameters for Review QualityLink Copied

The parameters most important for code review quality are:

temperature: Lower values (0.1-0.3) produce more consistent, less creative reviews. Higher values (0.7-1.0) can surface more diverse issues but at the cost of reliability.
max_new_tokens: Review length. Too short cuts off multi-issue reviews; too long produces rambling output. 150-300 tokens works well for single-function review.
prompt structure: Framing the review with explicit categories (correctness, style, documentation) guides the model to address all aspects rather than dwelling on the first issue it notices.
model selection: Instruction-tuned checkpoints trained on human feedback produce substantially better reviews than base language models, even at smaller scale.

Diff-Level vs. Function-Level ReviewLink Copied

There are two natural units for code review: the individual function or file (function-level) and the complete set of changes in a pull request (diff-level). Function-level review is simpler but misses review concerns that require understanding the interaction between changes across files. Diff-level review is richer but requires the model to handle longer contexts with deletions, additions, and unchanged context lines mixed together in unified diff format.

Models fine-tuned on diff-level data learn to reason across the change set. They can identify cases where a new function was added but the corresponding test was not, or where a change to a helper function is inconsistent with how callers in other files use it. These cross-file observations are much more valuable than per-function comments in isolation, but they require much more sophisticated context handling and are correspondingly harder to train for.