Attribution and Citation: Sourcing LLM Outputs › The Attribution-Fluency Tradeoff

One subtle tension in attributed generation is the tradeoff between attribution accuracy and response fluency. Accurate attribution requires the model to closely track what specific passages say and only claim what those passages support. But language models are trained to produce fluent, coherent responses, which often requires synthesizing, paraphrasing, and interpolating across multiple sources.

This synthesis process is exactly what makes language models useful for response generation, but it is also what makes attribution hard. When a model produces a sentence that synthesizes information from three different passages, no single citation fully supports that sentence. The synthesized claim may be more informative than any individual source, but it is also harder to attribute accurately. The model has done cognitive work to combine the sources, and that work cannot easily be traced back to any one passage.

Consider a concrete example. Passage 1 says "Transformer models use self-attention mechanisms." Passage 2 says "Self-attention scales quadratically with sequence length." The model produces the sentence: "Transformer models use self-attention, which scales quadratically with sequence length." This sentence is correct and is a natural synthesis of passages 1 and 2. But it requires two citations, and the attribution logic must recognize that the first clause comes from passage 1 and the second from passage 2. A model that cites only one passage, or cites both for the entire sentence rather than assigning each part to its source, has technically produced incorrect attribution even though the synthesized claim is fully accurate.

Several approaches attempt to manage this tradeoff:

Extractive generation: Rather than having the model freely generate attributed text, constrain it to produce responses that are maximally close to exact extracts from source documents. This makes attribution trivial because the text is essentially the citation, but it produces less fluent, more fragmented responses. Users often find extractive responses harder to read than fluently synthesized ones, even when the information content is the same.

Claim decomposition: Decompose complex claims into atomic sub-claims, each of which can be independently verified and attributed. The model might first generate a free-form response, then a second step decomposes each sentence into its constituent atomic claims, and a third step matches each claim to a supporting passage. This improves attribution precision at the cost of response coherence: the final attributed response may feel choppy if it closely follows the claim-by-claim structure.

Citation confidence thresholding: Have the model provide citations only when it has high confidence that a passage supports a claim, leaving some claims uncited rather than providing uncertain citations. This trades citation recall for citation precision: the user will see fewer citations, but those they do see are more likely to be correct. For applications where false citations are particularly harmful, this conservative strategy may be preferable.

Constrained decoding: At inference time, restrict the model's generation to only produce claims that can be grounded in the current context. This is related to the constrained decoding methods discussed in the GPT architecture chapter. By constraining the generation space, you prevent the model from making claims that cannot be attributed, at the cost of potentially less fluent or less informative responses.

Each of these strategies represents a different point on the attribution-fluency frontier. The right choice depends on the application, the users' ability to tolerate reduced fluency in exchange for increased reliability, and the costs associated with false citations versus missing citations.