Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Attribution and Citation: Sourcing LLM Outputs › Visualizations

Michael BrenndoerferFebruary 16, 20264 min read

Language AI HandbookAll chapters

Part 56 of 67Hallucination and Factuality

Chapter 433Attribution and Citation: Sourcing LLM Outputs › Visualizations

Reading section 10 of 16 · Visualizations · 4 minSection 10 of 16 · 4 min

Section 10 of 16 · 4 min read

Attribution and Citation: Sourcing LLM Outputs › VisualizationsOne section · Sections in this chapter10 / 16 ↓

Sections in this chapter16

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let us visualize citation evaluation behavior across a range of entailment thresholds and examine how different claim types behave under NLI-based evaluation.

Out[11]:

Visualization

Bar chart comparing entailment scores for three claim-citation pairs, showing correct attribution, date mismatch, and wrong source. — Entailment scores for three claim-citation pairs in the simulated attribution evaluation. The well-supported claim (transformer year) shows high entailment from its correctly cited passage. The date-mismatched claim (BERT year) shows lower entailment because the passage contradicts the stated year. The completely wrong citation (Amazon passage for a self-attention claim) scores near zero, correctly flagging hallucinated attribution. The dashed line at 0.5 marks the threshold used to classify a citation as supportive.

Out[12]:

Visualization

Line chart showing citation precision, claim recall, and F1 score across entailment threshold values from 0.1 to 0.9. — Citation precision, claim recall, and F1 score as functions of the NLI entailment threshold on a simulated dataset of 200 claim-citation pairs. At low thresholds, recall is high but precision is poor because many borderline passages are counted as supportive. As the threshold rises, precision improves sharply while recall declines. The F1 curve peaks near threshold 0.5 to 0.6, indicating the balanced operating region. The dotted vertical line marks threshold 0.5.

Out[13]:

Visualization

Overlapping kernel density plots for four citation quality types, showing entailment score distributions across the 0-to-1 range. — Simulated NLI entailment score distributions for four citation quality categories: well-supported, unsupported, overclaiming, and wrong-source citations. Well-supported citations concentrate near 1.0 with most mass above the 0.5 threshold. Unsupported citations cluster in the low range, below 0.3. Overclaiming citations occupy a middle range reflecting partial support. Wrong-source citations spread broadly because the passage is often topically related without directly entailing the specific claim.