Information Extraction › Structured Output Generation with LLMs

The traditional IE pipeline (NER, RE, EE) is powerful but brittle. Each stage introduces its own errors, and adapting to a new domain or a new schema requires retraining multiple models. Large language models offer a different approach: instruct the model to extract all relevant information in one pass and return a structured JSON object.

This approach uses the LLM's broad world knowledge and flexible reasoning. A model that has read millions of news articles already "knows" that acquisitions involve buyers, sellers, prices, and dates. It understands that "deal," "transaction," and "purchase" are all synonyms for an acquisition event, and that "five billion dollars," "$5B," and "USD 5,000,000,000" all express the same monetary value. You can directly prompt it to structure this knowledge without annotating training data.

The tradeoffs are significant. LLM-based IE is easier to adapt to new schemas and domains, but each inference call costs more than running a fine-tuned sequence tagger, and the model may hallucinate fields that are not present in the source text. For high-volume production workloads processing millions of documents, the cost difference matters. For exploratory work or low-volume high-complexity extraction, LLM-based approaches are often the practical choice. A common production pattern is to use an LLM to validate and develop the schema on a sample of documents, then train a fine-tuned model for high-volume extraction, and use rule-based post-processing to enforce hard constraints on both.

The key to reliable structured extraction is a precise, schema-defining prompt. The schema should specify the exact fields expected, their types, and what to include when information is absent. Vague prompts produce inconsistent output. A prompt that says "Extract the key information" may return a narrative summary; a prompt that defines a specific JSON schema returns a consistently structured object.

Equally important is the instruction for handling missing information. If you say "Extract the deal value," a model that finds no deal value in the text may either return null or invent a plausible number. Explicitly instructing the model to "Set any field to null if the information is not present in the text" substantially reduces hallucination, because the model is given a valid, expected output for the case where information is absent.

The extracted JSON is machine-readable and maps directly to a database schema. This one-pass approach is particularly powerful when the schema is complex or when fields are expressed in varied natural language patterns that would require many separate pattern rules to capture. Notice that the LLM has normalized "May 2014" to a proper ISO date format ("2014-05-01"), converted "$3 billion" to an integer number of USD, and added the implied title "CEO" for Tim Cook, all without being explicitly instructed to do so.

This normalization behavior is one of the most practically valuable properties of LLM-based extraction. Traditional rule-based systems require explicit normalization rules for every surface form: separate patterns for "May 2014," "05/2014," "the fifth month of 2014," and "Q2 2014." An LLM understands all of these as referring to the same time period and normalizes them to ISO format if the schema specifies an ISO date field. The same applies to monetary values expressed as "$3B," "$3 billion," "three billion dollars," or "USD 3,000,000,000."

Modern LLM APIs provide structured output features that guarantee syntactically valid JSON regardless of what the model generates. OpenAI's function calling mechanism and structured output mode, and similar features in Anthropic's API and Google's Gemini, work by constraining the model's token sampling to only produce outputs that conform to a provided JSON Schema.

The mechanism is elegant: during decoding, the sampling process is constrained so that only tokens compatible with the current parse state of the expected JSON Schema can be generated. If the schema says the "amount" field must be an integer, the model cannot generate a string in that position. This is implemented via constrained decoding with a finite state machine derived from the JSON Schema. As we saw in the chapter on constrained decoding, this approach guarantees syntactic validity while allowing the model's full semantic capability to operate within the constraint.

The practical benefit is that you no longer need to handle JSON parse errors in your application code. Without structured output mode, a simple retry loop catches the occasional malformed JSON. With structured output mode, every response is guaranteed to be valid JSON that conforms to the schema. This is especially valuable for production systems where a malformed response that crashes a parsing pipeline at 2 AM is a reliability incident.

When structured output mode is not available, adding "Return only valid JSON. Do not include any explanation." to the prompt and validating with json.loads() with a retry on parse failure achieves similar reliability. A simple retry loop that re-prompts with "The previous response was not valid JSON. Try again." handles most failure cases within two attempts. For the rare case where the model consistently fails to produce valid JSON for a particular input, logging the failure and falling back to a rule-based extraction or manual review is the appropriate response.

Many real documents contain multiple events or multiple entities of the same type. A single news article may mention five acquisitions. The schema must accommodate lists, and the prompt must be explicit about including all instances.

The challenge is completeness: LLMs tend to focus on the most prominent event in a document and undercount secondary events. Explicitly asking for "a list of ALL events" and specifying that the list may contain multiple items reduces this problem. For critical applications, running a separate extraction pass per candidate entity pair (found by an NER scan) and then deduplicating is more reliable than a single all-events extraction.