Relation, Event, and Temporal Extraction

An article can mention Acme Corp and Bolt Systems without saying that either bought the other. To build an acquisition record, recover the buyer and target when the text identifies them, retain the supporting text, and record timing at the precision the source supplies. Leave missing participants or timing unknown. You also need to distinguish a completed acquisition from a reported intention to acquire. Entity names alone cannot answer those questions.

A list of names can tell you that an article mentions Acme and Bolt, but it cannot tell you whether Acme bought Bolt, whether Bolt bought Acme, whether the two are partners, or whether the article merely compares them. Recovering the statement adds information that a list of names omits: argument roles, the predicate, and whether the text asserts it. An extracted record states a claim. Whether the source supports it, and whether it is true in the world, are separate questions.

That structure comes in three related forms. Each can be studied and evaluated separately: a relation classifier, for example, can predict a label for two supplied entity spans without extracting events or dates. The worked examples here connect their outputs at document level.

• Relations are links between entities: (Acme Corp, acquired, Bolt Systems). The binary examples here represent statements about entities using a typed link. The schema supplies the name of the link, such as acquired, founder_of, or chief_executive_of, and the extraction system decides whether the link is supported by a particular piece of text.
• Event records can represent described occurrences or situations with a trigger expression, arguments filling roles, and attributes such as time, place, and price. Under the ACE annotation guidelines, triggers are often verbs or nominalizations, with other forms allowed. Acquisition(trigger=acquired, Buyer=Acme Corp, Acquired=Bolt Systems, Price=$120M, Time=unknown). Such records can retain more structure than a binary link: multiple roles, event-valued arguments when the schema permits them, modality, and polarity.
• Temporal expressions and relations ground events on a timeline and order them relative to one another: E1 before E2. E3’s date may fall anywhere in 2011 when only the year is known. Temporal relations let us ask which events precede others and retain unresolved orderings.

Document context adds decisions about references and evidence aggregation to these tasks. In Named Entity Recognition and BIO Tagging we labeled spans inside sentences. Here the unit of analysis shifts upward. A relation's two arguments may sit four paragraphs apart. An event's time may be expressed as "two weeks later," which requires a reference time. A pronoun like "it" may be the buyer in the most important sentence of the document.

This shift from sentence-level to document-level extraction is the central theme. As we'll see, it changes the models, the evaluation, and the error analysis. A supplied-pair sentence classifier and a document-level entity-pair extractor have different inputs and prediction units. Compare their scores only after stating those units and the gold annotations. Document-level models can aggregate evidence across mentions and use reference information, as the DocRED examples below illustrate.

Confusing these levels has practical consequences. A model that is excellent at mention-level relation classification may look mediocre when its predictions are aggregated to the entity level, and that drop is not necessarily a modeling failure. Changing the prediction unit and aggregation rule can change F1 even when the underlying mention predictions stay fixed. Similarly, an event mention is not an event: the same acquisition may be mentioned in a headline, in the body of an article, and in a follow-up piece a week later. If several mentions refer to one occurrence, counting each mention as a distinct event overcounts occurrences. Merging those references is the event-coreference task.