Information Extraction › Relation Extraction

Once you have a set of entity mentions, the next question is how they relate. Relation extraction identifies and classifies semantic relationships between entity pairs mentioned in the same text. The output is a set of triples: (subject entity, relation type, object entity).

From the sentence "Apple acquired Beats Electronics for $3 billion in May 2014," relation extraction should produce the triple (Apple, acquired, Beats Electronics). Combined with NER output, the full structured record becomes:

This triple format, borrowed from the Resource Description Framework (RDF), is the basis of knowledge graphs. Systems like Google's Knowledge Graph and Wikidata are populated partly through automated relation extraction at scale. A knowledge graph that connects millions of entities through typed relations supports queries that would be impossible with keyword search alone: "List all AI companies acquired by tech giants for more than $1 billion since 2020."

Relation extraction is harder than NER for a fundamental reason: it is a binary task over pairs of entities, and the number of candidate pairs grows quadratically with the number of entities in a document. A document with 20 entities has $20 \times 19 / 2 = 190$ candidate pairs, most of which express no relation at all. The system must distinguish the small set of true relations from the large set of unrelated entity co-occurrences. This makes precision a particularly difficult metric to optimize: a system that predicts "no relation" for all pairs achieves perfect precision on the "no relation" class but extracts nothing useful.

The most common formulation treats RE as a classification problem. Given a sentence and two marked entity spans, predict the relation type (or "no relation" if the entities are unrelated). The entity types from NER serve as strong features: if both entities are ORG, the system focuses on organization-organization relations (acquired, merged_with, subsidiary_of), while ORG-PER pairs suggest employment or founding relations (founded_by, CEO_of, employed_by).

This type-based filtering is important for efficiency and precision. Restricting to plausible type combinations reduces the search space dramatically and avoids nonsensical pairings like checking whether a date is the CEO of a location. In practice, the mapping from entity type pair to plausible relation types is determined by the schema you are using. For the ACE relation set, the mapping is explicit in the task definition. For open-domain extraction, you can use the relation type definitions themselves to infer which entity types are compatible.

Mathematically, given entity spans $e_1$ and $e_2$ within sentence $\mathbf{x}$, a classifier computes the probability of each relation type $r$ as:

$$P(r \mid \mathbf{x}, e_1, e_2) = \text{softmax}(\mathbf{W} \cdot \mathbf{h}_{e_1, e_2} + \mathbf{b})$$

where:

• $r \in \mathcal{R}$ is a candidate relation type from a fixed label set $\mathcal{R}$ (e.g., "acquired," "founded_by," "located_in," "no_relation")
• $\mathbf{h}_{e_1, e_2} \in \mathbb{R}^d$ is a combined representation built from the encoder's [CLS] token embedding and the pooled embeddings of both entity spans
• $\mathbf{W} \in \mathbb{R}^{|\mathcal{R}| \times d}$ and $\mathbf{b} \in \mathbb{R}^{|\mathcal{R}|}$ are learned classification parameters

The entity span representations are typically obtained by pooling the token embeddings within each span, or by using special marker tokens (such as <e1> and </e1>) inserted around the entity mentions before encoding. These markers serve as positional cues that make the entity boundaries salient to the attention mechanism. The TACRED and DocRED datasets are the standard benchmarks for relation extraction. TACRED covers 42 relation types in a news and web context; DocRED extends this to document-level extraction where the subject and object entities may appear in different sentences. Models based on BERT or RoBERTa achieve F1 scores in the mid-70s on TACRED, showing the difficulty of the task.

The chart below compares how relation extraction F1 degrades across different sentence-level distances between entity pairs, illustrating that RE becomes substantially harder when entities are far apart in the sentence.

Annotating relation extraction training data is expensive: every sentence requires a human to identify entity pairs and label their relationship. Distant supervision offers an alternative. The core assumption is: if a knowledge base asserts the triple (Apple, acquired, Beats), and a corpus sentence contains both "Apple" and "Beats," that sentence likely expresses the acquired relation. This auto-generates millions of labeled examples without human annotation.

The assumption is noisy. Many sentences mentioning two entities do not express the expected relation, or express a different one. "Apple and Beats collaborated on audio products before the acquisition" mentions both entities but expresses collaboration, not acquisition. "Beats was founded by Dr. Dre before Apple existed as a company" mentions Apple and Beats but expresses no relation relevant to the acquisition. Models trained on distantly supervised data must therefore be robust to label noise.

Multi-instance learning approaches address this noise by aggregating evidence across multiple sentences about the same entity pair. If a knowledge base asserts (Apple, acquired, Beats), and ten sentences in the corpus mention both entities, at least some of them likely do express the acquisition relation. A model trained to classify the entity pair, using the bag of sentences as input rather than any single sentence, is more robust to individual mislabeled examples. The model learns to identify the most expressive sentence in the bag, ignoring the noisy ones, rather than treating every sentence as equally reliable evidence.

This aggregation idea connects distant supervision to a broader principle in IE: complementary evidence. A single sentence may be ambiguous, but multiple sentences mentioning the same entity pair tend to cluster around the most frequent true relation. Aggregation exploits this statistical regularization without requiring any human labels beyond the knowledge base itself.

Separating NER and RE into pipeline stages introduces error propagation: NER mistakes create bad entity boundaries that make RE fail even with a perfect relation classifier. If NER labels "Beats Electronics" as two separate entities, "Beats" (ORG) and "Electronics" (something else), the relation classifier never sees the correct entity pair and cannot produce the acquisition relation.

Joint models tackle both tasks simultaneously, sharing encoder representations and training with a combined objective. The encoder processes the sentence once, and separate prediction heads for entity typing and relation classification share the same contextual representations. This sharing means the model can learn that the presence of the word "acquired" between two ORG entities is a strong signal for both the entity type classification and the relation type classification, rather than learning each independently.

Modern joint extraction models often frame the problem as span prediction: enumerate all possible spans up to some maximum length, score each as an entity or non-entity, then score all entity-pair combinations for relation types. The table-filling approach represents this as a matrix where rows and columns are tokens and each cell encodes the relationship between the span starting at that row and ending at that column. This matrix representation enables efficient computation and naturally handles overlapping entities. SpERT (Span-based Entity and Relation Transformer) is a representative joint model that operates in this way, processing all spans and entity pairs in a single forward pass through a BERT encoder and achieving strong performance on both the CoNLL-2004 and SciERC benchmarks.

The tradeoff between pipeline and joint models is not always in favor of joint models. Pipeline models are easier to debug, easier to update (you can swap the NER model without touching RE), and easier to train because you can use separate datasets for each task. Joint models require co-annotated data where entities and relations are labeled in the same document, which is harder to obtain. For most production systems where well-labeled data is available and latency matters, a carefully designed pipeline outperforms a joint model despite the error propagation risk.