Attribution and Citation: Sourcing LLM Outputs › Summary

Attribution and citation form the accountability layer of language AI systems. The core insight is that correctness and verifiability are different properties: a model can be correct without being verifiable, and an attributed model provides verifiability as an additional guarantee on top of correctness.

Key takeaways from this chapter:

• Attribution connects generated claims to supporting sources. Forms range from document-level (which sources were used) to span-level (exactly which passage supports each phrase), with different granularities offering different tradeoffs between implementation complexity and user transparency.
• In RAG systems, attribution is the additional requirement that models indicate which retrieved passages support which parts of their output. The pipeline has distinct stages for retrieval, prompt construction, generation, and attribution verification.
• Attribution accuracy failures take several forms: unsupported citations, overclaiming, missing citations, citations to the wrong source, and unattributed copying. Each failure mode requires different detection strategies and has different consequences for users.
• Evaluation frameworks like AIS and ALCE assess both claim correctness and citation quality. AIS defines attribution as verifiability from cited sources alone, while ALCE evaluates attribution across multiple NLP tasks using a combination of automatic and human judgment.
• Citation precision and recall measure, respectively, the fraction of citations that are supportive and the fraction of supported claims that are cited. The F1 score summarizes the tradeoff between the two.
• NLI-based attribution evaluation automates the judgment of whether a passage entails a claim, but NLI models have their own error rates and cannot handle multi-hop inference. The entailment threshold is a key parameter that governs the precision-recall tradeoff.
• Source verification is conceptually distinct from attribution evaluation: attribution checks whether a claim matches its source, not whether the source is itself correct. Both layers are needed for full reliability.
• Training models for attribution requires supervised data with verified citation chains. Prompt engineering can improve attribution behavior at inference time, but does not fully substitute for training-time exposure to attributed generation examples.
• Key tensions include the fluency-attribution tradeoff (synthesis improves response quality but complicates attribution), the precision-recall tradeoff (conservative citation improves precision but reduces coverage), and the granularity tradeoff (finer attribution is more informative but harder to produce and evaluate).

The next chapter covers uncertainty quantification: how models can communicate their own confidence in their outputs, which complements attribution by adding calibrated probability estimates alongside source evidence.