Relation, Event, and Temporal Extraction › Events

An event record describes an occurrence with participants in specific roles and can include time, place, and factuality attributes. Those fields support queries beyond the binary relation, and each creates a separate correctness condition. A record for a planned or denied event does not assert that it occurred.

A binary relation fits a single edge in a knowledge graph. In a small event subgraph, labeled role edges connect the event record to its arguments. A mention record links the event to its trigger span, and the event record can carry attributes. That extra structure allows a system to answer questions like "who bought whom, for how much, and when?" An evaluation can assess trigger identification, event typing, argument identification, role assignment, and factuality separately.

In the ACE program's formulation, an event mention consists of:

• A trigger: the word or phrase that most clearly evokes the event being described, whether asserted, negated, or hypothetical. It can be a verb ("acquired", "died", "resigned"), a nominalization ("acquisition" in "the acquisition", "resignation" in "his resignation"), or an adjective ("married").
• Arguments: entity, time, or value mentions filling roles allowed by the event type. ACE tags these only when they occur within the event extent.
• Event attributes: polarity (explicit negation versus positive), tense, genericity, and modality (asserted versus other, under ACE's definitions). These describe the source's presentation, rather than independently verifying what happened.

For example, take the sentence "The insurgents attacked the convoy outside Mosul on Tuesday morning." Under the ACE Attack type, selected roles are:

Selected attributes for this event are Polarity=POSITIVE, Tense=PAST, and Genericity=SPECIFIC.

Notice how much of this connects to what you already know. In this ACE example, trigger and argument mentions have spans in the text. Trigger detection can be formulated as sequence labeling, as in NER, covered in Part VI. The role inventory is a semantic-role inventory, close in spirit to the PropBank argument labels we discussed in Semantic Roles and Argument Structure, with the important difference that event roles are event-type-specific (Attacker for Attack, Buyer for Transfer-Ownership) while PropBank reuses numbered core labels such as ARG0 and ARG1, whose meanings are specified by predicate sense; general modifiers use labels such as ARGM-LOC and ARGM-TMP.

The table also shows why event extraction is not just relation extraction with more slots. The Instrument role is unfilled in this example, and that is not an error. The text simply does not mention an instrument. A system that forces every role to be filled will hallucinate arguments, while a system that handles missing roles correctly must distinguish between "not mentioned" and "mentioned but not extracted." That distinction matters when evaluating event extraction.

Trigger ambiguity includes word-sense and schema decisions. "Acquired" can describe purchasing a company, gaining a skill, or contracting an illness. A detector using only that word has no way to distinguish those examples. Arguments and surrounding context provide evidence; a business schema must also define how ownership transfers differ from organization mergers.

The ambiguity is not limited to verbs. "Acquisition" can refer to a business deal or to the process of learning a skill. "Strike" can be a labor action, a military attack, or a baseball pitch. "Deal" can be a business agreement or a turn at dealing cards. The same trigger string can occur with different senses. Its arguments and surrounding context supply evidence for the intended event type. A joint model can use evidence from trigger and argument predictions together, as Li et al. and JRNN illustrate. The words identifying buyers, sellers, or military targets provide context; their presence alone does not establish the event's type or factuality.

Nominal triggers can be nouns inside longer phrases, such as "merger" in "the merger", "layoffs" in "the layoffs", and "announcement" in "the announcement". A nominal mention can leave participants unstated in its local sentence. "The acquisition expanded the cloud business" names the event without naming its buyer.

Nominal triggers also interact with coreference. "The acquisition" may refer to an event mentioned earlier. Resolving that reference requires context beyond the nominal mention, although an event explicitly described in a single sentence can still be extracted at sentence level.

Negative and hypothetical events introduce factuality distinctions. "Acme considered acquiring Bolt" and "Acme denied that it had acquired Bolt" both contain the trigger and both arguments, but neither asserts the event. Polarity, modality, and reporting context are therefore part of the extraction target when an application needs to distinguish assertions, plans, and denials.

The failure is easy to imagine. A financial monitoring system that extracts "Acme acquired Bolt" from a sentence saying "Acme denied that it had acquired Bolt" will report an acquisition that the sentence does not affirm. A representation may identify the participants correctly while mishandling the denial context. Whether that is scored as a relation, factuality, or attribute error depends on the annotation scheme. Systems that skip attribute classification are therefore vulnerable to a whole class of errors that are invisible in trigger-level F1.

Implicit arguments are not syntactically connected to their predicates; some can be recovered from discourse (Cheng and Erk, 2018). For example, "The acquisition closed two weeks later" has no explicit buyer, but the buyer is the buyer of the acquisition the sentence refers to. Recovering that omitted buyer requires connecting this mention to its antecedent event. That is a document-context requirement of this example, rather than of every event mention.

A later sentence can refer to an introduced event with a definite description that omits its participants. "The deal closed," "the merger was approved," and "the acquisition would expand the cloud business" can be used this way when they refer back to an earlier mention. An extractor needs that earlier context to recover the omitted participants in those cases.

A trigger-to-argument pipeline detects triggers first, then classifies arguments for each predicted trigger. Its components can be inspected or replaced separately while their interfaces remain compatible. In a trigger-gated implementation, a missed trigger prevents later role predictions: if trigger detection misses "acquired", no argument extraction happens, no matter how good the argument model is.

A one-way pipeline cannot feed later predicted role labels back into an earlier trigger decision. Its trigger encoder can still inspect the sentence, including words that will become argument candidates. A joint model or iterative pipeline can additionally exchange predicted structural information.

A joint model represents trigger and argument outputs together. The classic formulation (Li, Ji, and Huang, 2013) is a structured prediction over triggers and arguments with global features, and the neural versions followed quickly: JRNN (Nguyen, Cho, and Grishman, 2016) jointly predicted triggers and argument roles from bidirectional sequence representations, with dependency-relation features and memory of earlier decisions; dbRNN (Sha et al., 2018) added dependency-tree shortcuts between syntactically related word states in its bidirectional RNN. DyGIE++ (Wadden et al., 2019) generalized the approach to entities, relations, and events in a single span-graph model with contextualized span representations, which illustrates shared span representations across these extraction tasks.

Joint modeling provides a way to share evidence across trigger and argument predictions. Whether it improves a particular pipeline, and by how much, requires a comparison with the same data, encoder, and evaluation protocol.

Let $x$ be the input and $y^{tr}$ the complete trigger labeling. For this document, fix a set of proposed trigger–argument pairs and let $j$ index those pairs. The variable $y^{arg}_j$ gives the pair’s role label, with a no_role outcome for pairs with no role. Write $y^{arg}$ for the collection of pair labels. To enforce the event schema in this model, give no_role probability one in the conditional factor for each pair whose trigger is labeled absent. For a present trigger, restrict that factor to no_role and the roles allowed by its predicted type. The factored model defines this joint conditional distribution.

$$P(y^{tr}, y^{arg} \mid x) = P(y^{tr} \mid x) \prod_{j} P\bigl(y^{arg}_j \mid x, y^{tr}\bigr)$$

A CRF over the complete output structure uses this form.

$$P(y \mid x) = \frac{1}{Z(x)} \exp\!\left(\sum_k \theta_k f_k(y, x)\right)$$

where $y$ is a complete output structure, $f_k$ are features, $\theta_k$ their weights, and $Z(x)$ sums the unnormalized scores over allowed structures. This follows the CRF construction from Conditional Random Fields. Finite feature weights express preferences. A hard constraint requires excluding structures from the support or assigning them zero probability, for example disallowing roles outside the event type's declared inventory. A target unmentioned in the text does not by itself invalidate an Attack mention.

In the displayed factored model, the pair-level role labels are conditionally independent given $x$ and the complete trigger labeling $y^{tr}$. A CRF can retain this factorization or add features coupling those labels. Its inference cost depends on the allowed structures and factorization; adding a CRF does not by itself establish that inference is slower. A shared encoder can be used in either design. The factored distribution allows both trigger-first and joint decoding. Suppose competing complete trigger labelings A and B have probabilities 0.6 and 0.4, and their best conditional role assignments have probabilities 0.5 and 0.9. Selecting the trigger labeling first picks A. Maximizing the joint probability picks B because $0.4 \times 0.9 = 0.36$ exceeds $0.6 \times 0.5 = 0.30$.

In Nguyen, Cho, and Grishman (2016), Table 3, JRNN reports 69.3 trigger-classification F1 and 55.4 argument-role F1 on its ACE 2005 blind test, a difference of 13.9 points. That comparison concerns one system and protocol. Argument recovery introduces several distinct decisions:

• Multiple mentions, one event. The argument "Acme Corp" may appear as "Acme", "the company", or "it" within the event's scope. Coreference connects mentions that refer to the same entity (see Coreference Resolution).
• Span reuse and event-valued arguments. One span can fill roles for multiple triggers, with a separate variable for each trigger–argument pair. If the schema permits multiple roles for the same pair, extend its variable to range over allowed role sets. Separately, a schema can permit an event as a role filler. For example, the illustrative Announcement schema can represent "Acme announced the acquisition of Bolt" as an Announcement whose Content is an Acquisition event.
• Long-distance roles. In "the company, which was founded in 2011 by Sam Okafor and which had grown to 40 employees, was acquired by a competitor", the company mention and the acquisition trigger "acquired" are separated by two relative clauses.
• Unfilled roles and evaluation. A role can be absent from the text. In set-based argument extraction F1, predicting no argument contributes neither a true positive nor a false positive when the gold set is also empty. It does not earn an extra true positive. A fixed-slot application may additionally report correct empty slots, but that is a separate evaluation choice.

The first bullet concerns reference resolution. In "Acme acquired Bolt. The company later sold it," the second sentence refers to participants by a description and a pronoun. Recovering their canonical identities requires deciding what "the company" and "it" refer to. Merely concatenating the sentences does not supply those decisions.

In the third bullet, two relative clauses intervene in the company subject phrase, while the main clause still links "company" to "was acquired". A window narrower than this surface distance excludes the company mention. A system that uses dependency paths can follow the syntactic link from the argument to the trigger even when the surface distance is large.

The fourth bullet separates extraction F1 from slot-level accuracy. For extraction F1 scored against a gold set, a gold argument with no match is a false negative and a predicted argument with no match is a false positive. Empty gold and predicted sets add no extracted items to either count. If an application reports slot-level accuracy, state the candidate-slot universe and report recall on filled gold slots separately. A large number of correctly empty slots can make slot accuracy high even when the system misses filled arguments.

Two other formulations express event outputs as answers or generated structures.

Event extraction as QA (Du and Cardie, 2020) turns each role into a natural-language question, in the same spirit as QA-based relation extraction. For an Attack event triggered by "attacked", the question "Who was attacked?" is posed against the sentence, and a reading comprehension model extracts the answer span. The answer can be a span elsewhere in the input, and the model includes a no-answer decision for absent arguments.

The QA formulation also gives a natural way to handle absent roles. If the question "Who was attacked?" has no answer in the text, the reading comprehension model can return "no answer," which is exactly the correct behavior for an unfilled role. In Du and Cardie's model, the first token ([CLS]) represents the no-answer case; span candidates are compared with that score. This is also a learned null decision, not a guarantee that empty roles are recognized correctly.

Event extraction as generation (Lu et al., 2021, Text2Event) casts the output as a linearized structure. Here is one way to illustrate such a record. Time is a simplified field in this illustration.

Text2Event trains a sequence-to-sequence model to generate a linearized event structure. The decoder predicts a serialized structure, including the roles present, rather than separately classifying every trigger-argument candidate. Decoding still searches over output sequences. Without structural controls it can emit invalid structures or unsupported arguments; Text2Event uses constrained decoding to restrict the allowed productions. At a role position such as Target in this example, the Text2Event trie limits the decoder to role names allowed by the schema.

Generation can encode event attributes in the output sequence. An output schema extended to include these attributes can serialize Polarity=NEG or Tense=PAST alongside the roles without requiring a separate attribute-output classifier. That is convenient, but it also means that an error in one part of the sequence can cascade to later parts. Constrained decoding helps, but it does not eliminate the need for validation.

An event schema defines the types and roles the system will predict. Several established resources use different inventories:

• FrameNet (Baker, Fillmore, and Lowe, 1998) organizes meaning into frames, each with frame elements. The Commerce_buy frame has Buyer, Goods, Seller, Money. FrameNet describes lexical-semantic frames; the released FrameNet 1.7 index contains 1,221 frames.
• PropBank (Palmer, Gildea, and Kingsbury, 2005) labels predicate arguments with numbered roles (ARG0, ARG1, ...) tied to each verb's frame file, plus adjunct labels (ARGM-TMP, ARGM-LOC). The original project annotates predicate roles in the Penn Treebank and supplies verb-specific framesets.
• ACE 2005 events use eight top-level types and 33 subtypes (Conflict:Attack, Life:Die, Transaction:Transfer-Ownership, Personnel:Start-Position, ...). Text2Event (Lu et al., 2021), for example, calls these 33 labels "event types". The ACE annotation guidelines define their roles and attributes; the corpus includes news and other genres.
• Domain schemas can define application-specific records. The BioNLP 2009 shared task, for example, defines Gene expression, Binding, and Regulation events. A separate illustrative financial schema might define Earnings, Merger, and Guidance; those names are design choices, not labels inherited from ACE.

Evaluate agreement for the actual schema and annotation instructions. A narrower label set changes the task; it does not guarantee higher agreement or better extraction. The TACRED re-annotation results above show why labels themselves need review, even when the output inventory is fixed.

The schema also determines whether an argument can itself be an event. BioNLP 2009 permits an event to fill the Theme or Cause of a Regulation event. Our custom Announcement schema can likewise use an acquisition as Content. This is different from FrameNet's frame-inheritance relations; inheritance alone does not establish that a frame element accepts a nested event. State the allowed argument types when defining a nested representation.

Political event coding reduces news reports to structured records such as (Actor, EventType, Target) with temporal and geographic metadata for statistical analysis. CAMEO's authors describe developing their scheme in research on third-party mediation.

• CAMEO (Conflict and Mediation Event Observations) provides a hierarchical taxonomy of about 300 event types for coding political interactions. CAMEO codes the source and target with the same actor coding scheme.
• TABARI uses verb-phrase patterns and a sparse parse to code political interactions. Its manual describes this pattern-based design; the architecture should not be generalized to every later event coder.
• GDELT (Leetaru and Schrodt, 2013) applies this at global scale, coding news into event records with tone and geography, whose 2013 codebook describes daily update files beginning on 1 April 2013.
• ICEWS provides historical machine-coded political event data; its dataset metadata states that the series was discontinued on 11 April 2023. POLECAT, described by Halterman and colleagues (2023), uses the PLOVER ontology and the Next Generation Event Coder. The authors describe a combination of NLP tools, transformer networks, expert annotations, and actor information from Wikipedia.

Political event coding includes both pattern-based coders and systems such as NGEC that use transformers. For longitudinal analysis, retain the coder and schema versions as well as the event records. A change in coding behavior can alter the measured trend even when the underlying reports stay fixed.

Evaluate that stability explicitly. Recode the same reference collection with each version. Compare records and per-class scores using the same scoring protocol and shared class definitions or a documented mapping. Report unmatched classes separately. A single aggregate F1 score on one version does not measure change between versions. Once labels are aligned, agreement F1 between the two runs can measure record overlap. Neither a rule-based nor a neural architecture guarantees stable outputs when its rules, weights, resources, or inputs change.

Two mentions in different places may refer to the same event. "The deal closed two weeks later" and "the acquisition completed on 17 March" may describe one closing. Event coreference clusters mentions judged to refer to the same occurrence. The ECB+ work (Cybulska and Vossen, 2014) distinguishes event mentions from event denotations and considers action, time, location, and participants.

Events can have names, but they can also be referred to through verbs, nominalizations, and paraphrases. In an acquisition example, "the acquisition", "the deal", and "the transaction" can refer to the same occurrence. Group "the merger" with them only if the source and schema establish that identity. Clustering asks whether the mentions refer to the same occurrence. Participants, time, location, and other evidence in context help answer that question.

Events also stand in relations to each other beyond identity:

• Temporal: before, after, simultaneous, during. Covered in the next section.
• Causal: cause, enable, prevent. BECauSE annotates expressed causal language using its own type and degree distinctions; event-link datasets such as Ning et al.'s (2018) joint temporal–causal dataset use their own causal-link inventory.
• Subevent: a component occurrence is part of a larger event, if the chosen event model defines that part-whole relation. An acquisition appearing as the Content of an announcement is instead an event-valued argument; that sentence alone does not establish a subevent relation. A schema can treat the closing as one stage of a larger transaction process. That process is distinct from the earlier reported acquisition in the worked example.

These links organize event records by showing how the occurrences are connected. Discourse Relations and Coherence is a planned later chapter.