Interpretability Goals › Summary

Interpretability research aims to understand what is happening inside neural language models, connecting observed outputs to the internal computations that produced them. The goals fall into four main categories:

• Debugging and error analysis uses interpretability to diagnose why models fail on specific inputs and identify systematic errors, enabling targeted fixes rather than blind retraining.
• Trust and accountability uses interpretability to produce explanations for model decisions and audit models before deployment, supporting both user confidence and regulatory compliance.
• Safety and alignment uses interpretability to verify that models have not learned dangerous patterns and to check that safety training is modifying the right internal circuits rather than adding superficial filters.
• Scientific understanding uses interpretability to characterize the representations and algorithms that language models implement, adding to the broader science of how statistical learning produces semantic competence.

The core methodological challenge across all these goals is faithfulness: ensuring that explanations accurately reflect the model's computational process rather than merely being consistent with the observed output. A plausible-sounding unfaithful explanation is worse than no explanation, because it creates false confidence. The gold standard for faithfulness testing is causal intervention: modify what the explanation says is important and verify that the output changes as predicted.

Interpretability methods operate at different granularities, from token-level attribution to circuit-level mechanistic analysis, and the appropriate level depends on the question. Model-level analysis provides an efficient entry point; circuit-level analysis provides the most mechanistically detailed answers. Most real investigations combine multiple levels and multiple methods, using each for its comparative advantage.

The main technical approaches, each covered in depth in the following chapters, are:

• Attention visualization (fast, widely applicable, but limited faithfulness guarantees)
• Probing classifiers (tests whether concepts are encoded; layer probing tracks their evolution across depth)
• Causal intervention methods: activation patching and the logit lens, which test causal roles directly
• Sparse autoencoders (finds interpretable features by decomposing activations into overcomplete dictionaries)
• Mechanistic analysis (reverse-engineers specific circuits to algorithmic precision)
• Activation steering (applies interpretability findings to modify model behavior at generation time)

Each technique has a different strength and is suited to different interpretability goals. As you work through the specific methods in this part, keep in mind what question each technique is designed to answer, what its faithfulness guarantees are, and how it fits into the larger interpretability research program.