Attribution and Citation: Sourcing LLM Outputs › The Role of Prompt Engineering

Even without fine-tuning, prompt engineering can significantly affect how well a language model attributes its claims. Models that have been instruction-tuned to follow detailed prompts can be directed to produce attributed responses by specifying the citation behavior explicitly in the prompt.

Several prompt patterns have been found effective in practice. Providing numbered passages with clear delimiters, then explicitly instructing the model to cite passage numbers, typically produces better attribution than simply providing context without instructions. Including a formatting example in the prompt (few-shot attribution examples) further improves citation consistency. Asking the model to explain its reasoning before giving its final answer can also improve attribution quality, because the reasoning step forces the model to identify which passages it is drawing on before it writes the cited response.

However, prompt engineering for attribution has limits. A model that has not been trained on attributed generation tasks will frequently cite passages selectively, misattribute claims, or produce citations that are syntactically correct but semantically wrong. The model may have learned the surface form of citation (putting [1] after a claim) without learning the underlying behavior (ensuring the claim is entailed by the cited passage). Distinguishing surface-form citation from valid attribution behavior is one reason evaluation frameworks like ALCE and AIS are important: they look past the presence of citations to assess whether those citations are correct.