Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Code Understanding › Code Implementation: Generating Explanations

Michael BrenndoerferFebruary 2, 20262 min read

Language AI HandbookAll chapters

Part 47 of 67Code Generation

Chapter 358Code Understanding › Code Implementation: Generating Explanations

Reading section 4 of 13 · Code Implementation: Generating Explanations · 2 minSection 4 of 13 · 2 min

Section 4 of 13 · 2 min read

Code Understanding › Code Implementation: Generating ExplanationsOne section · Sections in this chapter4 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's implement code explanation using the Hugging Face transformers library with a code-aware model.

We start by loading a small instruction-tuned model capable of code understanding. We will use Salesforce/codegen-350M-mono for lightweight demonstration, but larger models like codellama/CodeLlama-7b-Instruct-hf produce substantially better results in practice.

In[5]:

Code

import warnings

warnings.filterwarnings("ignore")

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Salesforce/codegen-350M-mono"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model_explain = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.float32
)
model_explain.eval()

In[6]:

Code

def explain_code(code_snippet: str, max_new_tokens: int = 120) -> str:
    """Generate a natural language explanation for a code snippet."""
    prompt = (
        f"# Python function\n{code_snippet}\n# What this function does:\n# "
    )
    inputs = tokenizer(prompt, return_tensors="pt")
    with torch.no_grad():
        outputs = model_explain.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.2,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id,
        )
    generated = tokenizer.decode(outputs[0], skip_special_tokens=True)
    explanation_start = generated.find("# What this function does:\n# ") + len(
        "# What this function does:\n# "
    )
    explanation = generated[explanation_start:].strip().split("\n")[0]
    return explanation

In[7]:

Code

snippets = {
    "find_duplicates": """\
def find_duplicates(lst):
    seen = set()
    duplicates = set()
    for item in lst:
        if item in seen:
            duplicates.add(item)
        else:
            seen.add(item)
    return sorted(duplicates)""",
    "binary_search": """\
def binary_search(arr, target):
    lo, hi = 0, len(arr) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            lo = mid + 1
        else:
            hi = mid - 1
    return -1""",
}

explanations = {name: explain_code(code) for name, code in snippets.items()}

Out[8]:

Console

Function: find_duplicates
Explanation: # 1. Find all the duplicates in the list.

Function: binary_search
Explanation: #   - arr[mid] == target

The model generates completions that continue the comment begun in the prompt. For well-known algorithmic patterns like binary search, even a 350M-parameter model often produces accurate explanations. For more specialized code, larger models with instruction-following fine-tuning produce noticeably better results.