Relation, Event, and Temporal Extraction › Limitations and Impact

Schema design limits the represented output. Choosing relation types, event roles, and temporal distinctions affects annotation and downstream queries. Useful schema work includes defining negative cases and revising distinctions annotators cannot apply consistently. Narrowing a schema can make a task easier, but results on that narrower task do not establish that a hundred examples outperform ten thousand on a different task. Compare schemas, annotation quality, and training size separately.

The second limitation is noisy supervision and evaluation labels. Distantly supervised relation labels can disagree with individual sentences. Separately, the TACRED re-annotation studies discussed above found errors in a human-annotated sentence-level benchmark; they are not evidence that TACRED was created by distant supervision. Event and temporal annotations also require explicit definitions and agreement checks. Report per-class results and simple baselines, and state how candidate pairs and negative examples were selected.

The third limitation is error propagation through a pipeline. A missed mention can remove an argument, and an incorrect date can create a temporal contradiction. Preserve provenance and check for cycles in strict before relations. Compare an event with publication time only under appropriate factuality conditions: a completed past event should not become future, whereas a planned event may legitimately follow publication. Treat contradictions as reasons to revisit upstream evidence. General interval-algebra satisfiability is NP-complete; path consistency can detect some contradictions without establishing global consistency in every case.

Concrete applications illustrate different scopes. NELL (Mitchell et al., 2018) learns beliefs from web text. The BioNLP 2009 task extracts molecular events, including Regulation with event-valued arguments. FDA describes testing and implementing NLP and machine-learning methods to support postmarket safety studies and review. Ding et al. (2015) evaluate news-event features for stock prediction and a market simulation. GDELT and POLECAT provide machine-coded political-event data under their respective schemas. Temporal applications also include paragraph-level temporal question answering (Stricker, 2021) and event-graph-based timeline summarization (Li et al., 2021). Zep (Rasmussen et al., 2025) separates entity, fact, and temporal extraction in an agent-memory architecture. Time anchoring helps distinguish "last Tuesday" from "the trip in March", while identifying the requested action and its referents remains additional work.

A prompted model can attempt extraction without task-specific training examples. To report semantic precision, its predictions still need correctness judgments under a stated annotation policy; syntax and substring checks alone do not supply those judgments. Outputs can include invented arguments, inconsistent dates, invalid labels, and prompt-sensitive decisions. Schema validation, copied evidence, normalization checks, and per-relation or per-role evaluation address different parts of that problem. The implementation demonstrates substring and date-recomputation checks; it does not call a generative language model, constrain decoding, or establish semantic correctness.

There is also an impact dimension that deserves explicit mention. Extraction systems make decisions about people and organizations, and those decisions can be wrong in ways that matter. A mis-extracted relation can link two entities that should not be linked. A mis-normalized date can place an event in the wrong year. A missed negation can turn a denied event into an asserted one. Provenance preserves the claimed source, validation checks selected invariants, and human review can assess claims that those checks leave unresolved. These controls have different scopes; passing them is not a guarantee that every extracted statement is correct.