Relation, Event, and Temporal Extraction › Time

Temporal records can retain calendar values and relative orderings. Normalizing "two weeks later" to a calendar date requires choosing its reference point. This section covers the representation standard, the anchoring problem, and the pairwise ordering task.

Temporal extraction separates surface recognition from semantic interpretation. The string "3 March 2019" can be recognized as a date. Assigning its value, relating it to other dates in the document, and deciding whether an event is represented as a point or an interval are additional decisions. The representation standard gives you a vocabulary for those decisions, and the anchoring problem tells you which context you need to make them.

TimeML (Pustejovsky et al., 2003) defines an annotation language for temporal information in text. ISO 24617-1:2012 specifies ISO-TimeML as a standard for temporal semantic annotation. The following tags are from TimeML 1.2.1:

• TIMEX3: temporal expressions, tagged with a type and either a normalized value or a valueFromFunction reference.
• EVENT: event expressions with an identifier and event class.
• MAKEINSTANCE: event instances linked to EVENT, carrying tense, aspect, and applicable polarity and modality attributes in TimeML 1.2.1.
• SIGNAL: expressions, typically function words, indicating temporal relations or quantification (for example, "before", "after", "during", and "twice").
• TLINK: temporal links between event instances or TIMEX3s. A document creation time participates through its TIMEX3.

TIMEX3 has four types:

TimeML values use ISO 8601 conventions together with task-specific extensions for underspecified dates. Keep those extensions separate from a claim of strict ISO conformance:

• Dates include full values such as 2019-03-03 and reduced-precision values such as 2019-03 (a month) or 2019 (a year). The related TIDES TIMEX2 guide uses 1999-06-XX for an unspecified day in June 1999.
• Times: bare clock-time forms such as T18:00 (6:00 p.m.).
• Durations: P followed by components: P2W (two weeks), P3D (three days), P1Y6M (one year six months).
• A start/end pair can also be written in ISO-style interval notation, such as 2019-03-03/2019-03-17. Do not substitute that string for a TimeML 1.2.1 value without checking its datatype grammar. TimeML can anchor a duration using beginPoint and endPoint references to TIMEX3 identifiers.

The mod attribute captures temporal modifiers: "more than 3 years" gets mod="MORE_THAN", "approximately two weeks" gets mod="APPROX". If only the normalized duration value is kept, that value cannot distinguish an exact duration from the same number with a modifier.

The distinction matters because temporal expressions are not only about dates. "More than three years" and "three years" denote different durations, and the difference is relevant to questions about how long something lasted. A system that normalizes both to P3Y loses the distinction. Keeping the mod attribute preserves it, at the cost of a slightly more complex value representation.

Replacing an unknown date within 2011 with 2011-01-01 invents precision. It can create an unsupported ordering against another event in that year. Preserve both the normalized year and the fact that it only bounds the event's date.

Normalizing "two weeks later" requires deciding what it is two weeks later than, in addition to identifying the expression.

A normalization interface that supports both document-relative and discourse-relative expressions can take the form

$$\text{value} = \text{Normalize}(e,\; t_{\text{DCT}},\; t_{\text{ref}})$$

where $e$ is the expression, $t_{\text{DCT}}$ is the document creation time, such as the time an email was written, and $t_{\text{ref}}$ is a reference time supplied by context.

Three cases illustrate different reference requirements:

• Fully specified absolute dates need no anchor. "3 March 2019" is 2019-03-03 regardless of when the document was written.
• Deictic expressions need a temporal perspective. In the isolated-expression example below, "yesterday" resolves against the document creation time $t_{\text{DCT}}$. Quoted speech may instead require the quoted utterance time. Substituting the system clock for the intended reference can change the result when those dates differ; absolute dates do not have that dependency.
• Relative expressions anchor to $t_{\text{ref}}$, an event or time supplied by context. "Three days after the merger" names its reference locally; "two weeks later" can require looking back to an earlier sentence. Resolving the reference can involve event extraction or coreference when the antecedent is an event mention.

A system limited to absolute and document-date-relative expressions cannot generally resolve discourse-relative expressions using their intended references. Its coverage depends on the corpus and annotation conventions; no fixed percentage follows from this taxonomy.

The failure is easier to audit when the two candidate anchors are placed on the same axis. The following computation compares two anchors for the closing. In the worked reading, the reference is the announcement on 3 March. Using the publication date instead gives a different result.

The anchor error is subtle because the system can still produce a valid date. For the announcement-anchored reading, substituting the document date produces a well-formed ISO date that disagrees with the intended value. A normalizer that checks only date validity misses this error. Compare normalized values against gold values to test the anchor decision.

For English tense analysis, Reichenbach (1947) distinguished three temporal points:

• Speech time $S$: when the utterance (or document) is produced.
• Event time $E$: the time assigned to the event in the tense interpretation.
• Reference time $R$: the temporal vantage point from which the event is located.

Tense is then a constraint on the ordering of these three. In Reichenbach’s schematic analysis, simple past has $E = R < S$. Present perfect asserts $E < R = S$: the event precedes a reference time that coincides with speech. Past perfect asserts $E < R < S$.

The worked example says, "Acme Corp announced on 3 March 2019 that it had acquired Bolt Systems for $120 million." Within Acme’s report, the past perfect places the acquisition ($E$) before the announcement ($R$). The worked chronology places that announcement before the article’s assumed narration time ($S$). The report leaves the acquisition date unspecified. It could refer to an acquisition earlier that day or on an earlier date.

Reichenbach's framework is useful because it separates two questions that are easy to conflate. The first is what absolute time or interval, if any, the text assigns to the described event. The second is how the text locates the event relative to other events and to the moment of speech. An application that needs both absolute values and discourse order has both prediction targets. If it only answers the second, it can produce a relative ordering without absolute dates. If it only answers the first, it can miss the narrative structure that the tense encodes.

Several temporal normalization systems use grammars and rules:

• SUTime (Chang and Manning, 2012) is a compositional rule-based recognizer built on token regex patterns. Its paper describes temporal objects for times, durations, intervals, and sets. Rules combine recognized components, and a resolution stage uses a reference date for relative expressions.

• HeidelTime (Strötgen and Gertz, 2013) is a rule-based, multilingual tagger with language resources and domain-sensitive normalization strategies. Their 2012 domain comparison evaluates different normalization strategies for news, narratives, colloquial text, and scientific text; the domain can affect which reference-time strategy is appropriate.

• Bethard (2013) pushed the grammar idea further with a synchronous context-free grammar mapping expression words to formal temporal operators that are evaluated to obtain values. The shared derivation ties parsing the supplied expression to its normalization, but it does not guarantee a semantically correct parse or value.

TempEval-3 provides a bounded comparison: on its newswire test data, statistical systems performed best under strict span matching and rule-engineered systems under relaxed matching. All participating normalization components used rule-engineered approaches. Those results do not establish a ranking for every domain or later model.

The lesson is not that neural models are unnecessary. It is that calendar arithmetic and explicit anchor rules can be implemented and checked separately from span recognition. A hybrid system that uses a neural model for recognition and a grammar for resolution can combine these functions. Given a reference time selected from context, the grammar can apply calendar arithmetic and explicit anchor rules. The neural model can recognize varied surface forms.

Temporal relation extraction assigns a relation to a pair of event instances or time expressions. A relative order can be recoverable even when their calendar dates are unknown.

A TLINK records a temporal relation between event instances or times. TimeML 1.2.1 declares the following relType values.

BEFORE, AFTER, INCLUDES, IS_INCLUDED, DURING, SIMULTANEOUS, IBEFORE, IAFTER, BEGINS, BEGUN_BY, ENDS, ENDED_BY, and IDENTITY.

Use the label definitions of the chosen annotation scheme. MATRES's VAGUE label, for example, is absent from the TimeML 1.2.1 list above. Its BEFORE label shares the spelling of a TimeML label.

The following resources use different temporal targets and annotation protocols:

• TimeBank 1.2 and the temporally annotated AQUAINT material used in TempEval-3 are resources for temporal extraction. Do not confuse the original TimeBank with TimeBank-Dense: Chambers and colleagues (2014) describe the original corpus as labeling only a small subset of possible temporal pairs.
• TempEval-1/2/3 (Verhagen et al., 2007, Verhagen et al., 2010, and UzZaman et al., 2013) defined shared tasks. In TempEval-3 Task ABC, systems start from raw text, identify events and time expressions, choose linked pairs, and classify the temporal links.
• TAC KBP temporal slot filling sought temporal bounds on when an entity's slot-filler relation held true, such as a person holding a job title. Its evidence could be distributed across documents.
• MATRES (Ning, Wu, and Roth, 2018) organizes events on temporal axes and compares their start points, rather than attempting full interval-endpoint judgments. The released main-axis data uses before, after, equal, and vague labels. "Multi-axis" does not mean a separate axis for each relation label. The paper reports improved agreement in a two-expert pilot on the main axis.
• Ning and colleagues (2018) evaluated their Temporal and Causal Reasoning (TCR) framework on a newswire corpus annotated with both temporal and causal links. TDDiscourse (Naik et al., 2019) augments TimeBank-Dense with discourse-level temporal annotations, including pairs more than one sentence apart.

The following examples cover feature-based and neural classifiers. Early feature-based classifiers included systems in TempEval 2007. Bethard’s ClearTK-TimeML used event tense, aspect, and constituency paths for same-sentence event pairs. Other classifiers used temporal signal features or VerbNet classes. CAEVO (Chambers et al., 2014) ordered twelve sieves by precision, passed earlier graph constraints to later sieves, and added transitive links after each sieve. Neural examples include bidirectional LSTMs over dependency paths (Cheng and Miyao, 2017) and BERT-based clinical temporal classifiers (Lin et al., 2019) that mark the candidate spans in the input.

Two problems matter when evaluating temporal systems.

Label imbalance and a document-order baseline. Yao and Huang (2018) use the chronological tendency of narrative stories after identifying narrative paragraphs. This does not establish the label distribution of arbitrary documents. If a declared single-label evaluation set has 90 percent BEFORE labels, always predicting BEFORE gives 90 percent accuracy and zero recall for every other class with gold examples. With one prediction per item and all classes included, micro-F1 is also 0.90. Measure the actual baseline and report per-class scores.

Consistency. An independent pairwise baseline predicts each pair separately, so its outputs can be globally inconsistent: $A$ before $B$, $B$ before $C$, and $C$ before $A$. A joint inference constraint or a repair pass can reject such a cycle. A design that predicts one global order can also induce pairwise labels from that order.

A single incorrect label can create such a cycle. For a true order $A<B<C$, predicting $A$ before $B$, $B$ before $C$, and $C$ before $A$ reverses just one pair. The cycle is a clear signal that something is wrong, and it can be used to trigger a repair pass or to flag the document for human review. Independent predictions can form an impossible cycle; they need not do so on every input.

The formal foundation for interval orderings is Allen's (1983) algebra. For proper intervals with $a^- < a^+$ and $b^- < b^+$, thirteen relations exhaust the possible endpoint orderings: six relations, their six inverses, and equality. Write $A = [a^-, a^+]$ and $B = [b^-, b^+]$, where minus and plus denote start and end coordinates:

Six representatives and equality look like this:

• before: A entirely to the left of B.
• meets: A ends exactly where B begins.
• overlaps: A starts first, they share some time, B extends beyond.
• starts: same start, A ends first.
• during: A strictly inside B.
• finishes: same end, A starts later.
• equals: identical extents.

The algebra is closed under composition: from the relation between $A$ and $B$ and between $B$ and $C$, you can derive a set of possible relations between $A$ and $C$. Composition tables let you propagate constraints and detect contradictions.

Two important caveats. First, the point algebra (Vilain and Kautz, 1986) uses instants as its variables, with base relations <, =, > and their disjunctions. Separate start and end variables can encode each individual Allen relation without collapsing the intervals; not every disjunction of interval relations can be expressed as a conjunction of point-algebra constraints on those endpoints. MATRES's start-point comparisons illustrate a point-based choice: start order alone cannot distinguish intervals that meet from ones that overlap. Second, general satisfiability of interval algebra constraint networks is NP-complete (Vilain, Kautz, and van Beek, 1990). This is a worst-case result, not a claim that every network is difficult. Exact checking remains an option for small instances or tractable restricted classes. Path consistency is sound but incomplete for the unrestricted algebra: it detects some contradictions and misses others.

Granularity matters when interpreting the algebra. The calendar-year window contains 3 March, but an event localized only to that window is not thereby known to contain an event on 3 March. Allen relations describe actual interval extents. Uncertain localization requires bounds or sets of possible relations; it cannot be substituted for event duration.

Yao and Huang's narrative-paragraph method uses a tendency for narrated events to follow chronological order. The sequence "we went to the restaurant, we ate, we left" illustrates an order that a reader can infer without an explicit "before" between each clause. Reordering those clauses can change that reading.

Several consequences follow.

• First, document order supplies a candidate weak labeler for the narrative setting. Its accuracy must be measured on the selected texts; the narrative tendency does not make it correct for every event pair.
• Second, not every temporal relation has an explicit signal. An ordering can be expressed without the words "before" or "after". The past-perfect example above uses tense and reference time to constrain the acquisition relative to the announcement.
• Third, flashbacks can reverse mention order, while reported speech can shift the temporal perspective. "Acme announced in 2019 that it had been founded in 2011" puts the later announcement trigger before the earlier founding trigger. Document order alone cannot recover that flashback. Explicit dates, tense, reference structure, or other contextual evidence can supply a different ordering; no particular parser feature is mandatory for every system.

Document order supplies a candidate baseline in the narrative setting, but the flashback above defeats an unqualified BEFORE rule. An unqualified document-order BEFORE rule fails on a pair whose mention order reverses its temporal order, as the flashback example demonstrates. The document-order preference should be defeasible. An explicit feature is one way to let other evidence override it.

Beyond pairwise classification, temporal reasoning asks what follows from the set of extracted constraints: can these events be arranged into a consistent timeline? Which events precede which? Is there a unique total order?

The machinery is constraint propagation over the interval or point algebra. For a pair $A,C$ and an intermediate event $B$, compose the allowed relations from $A$ to $B$ and from $B$ to $C$. Intersect the result with the current set for $A,C$. Cycle through the relevant triples until no set changes. If a sound propagation step empties a pair's allowed relation set, the supplied constraints are jointly contradictory under the representation's assumptions. Recheck the source assertions, extracted relations, normalization, and interval assumptions; the contradiction need not originate in the extractor.

Constraint checks can flag records for review. For an acyclic graph of strict BEFORE constraints, a topological sort returns one linear extension. It does not establish a unique true order, resolve unknown pairs, or enforce interval relations such as overlaps. Keep unresolved relations explicit rather than treating an arbitrary tie-break as evidence.

Constraint propagation can also supplement evaluation. A jointly unsatisfiable set of predicted relations cannot serve as one consistent timeline under the declared assumptions, even when each pair looks plausible. Pairwise scores alone can miss this incompatibility. Recheck source contradictions and factuality as well as extraction: faithfully recording conflicting reports is different from asserting that both reports are true.

Evaluating temporal extraction involves three subtasks with different metrics:

• Timex recognition and normalization is evaluated with span-level precision, recall, and F1 for recognition, and value accuracy (exact match of the normalized temporal value) for normalization, reported separately. A correct span can have an incorrect value, so report both. State the span alignment and the denominator used for value accuracy.

• Temporal relation extraction is evaluated with precision, recall, and F1 over labeled pairs, sometimes with a "reduced" label set that merges fine distinctions. Report class-level behavior alongside micro-F1. In the majority-only example above, one prediction per item and micro-averaging over all classes give 0.90 micro-F1. Every other class with gold examples has zero recall. Macro-F1 and a confusion matrix expose aspects of that result that the micro aggregate alone does not identify.

• Timeline ordering can be assessed against a declared reference order, for example by the fraction of comparable event pairs ordered correctly. Kendall's tau is another option for total orders, with a stated tie convention when needed. These ordering checks do not replace evaluating event identity, extraction, and time normalization.

State which pairs are annotated and evaluated. The original TimeBank labels a subset of possible pairs; TimeBank-Dense and TDDiscourse add annotations under different selection policies. A sampled evaluation set can have a different class balance from all candidate pairs at inference.

The sampling issue is worth emphasizing because it can make a system look better or worse than it is. If the selected pairs have a higher error rate for the evaluated system than the full candidate set, measured accuracy decreases; a lower selected error rate has the opposite effect. Report the selection policy and its label distribution instead of assuming that a named pair category is harder for every model. Papers that report only aggregate F1 without describing the sampling procedure are hard to compare.