Relation, Event, and Temporal Extraction › Implementation
Language AI HandbookAll chapters Relation, Event, and Temporal Extraction › ImplementationSection view · Sections in this chapter8 / 12 ↓
Reading tools Definitions, links & notes
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
We'll implement dependency rules, reference heuristics, temporal normalization, and interval reasoning, then apply grounding checks to a hand-written response to an extraction prompt.
The dependency rules expose the selected syntactic conditions. The pronoun heuristic supplies a buyer for this document and has explicit failure cases. The temporal calculation compares two anchors. The final checks inspect selected strings and a normalized date in the supplied response.
Setup
Run these cells in the book environment with book_plot_style on the Python path and spaCy’s English en_core_web_sm model installed. The worked code also uses Matplotlib, pandas, dateparser, and python-dateutil.
PythonIn [11]Expand
import json
import re
from datetime import datetime
import dateparser
import pandas as pd
import spacy
from dateutil.relativedelta import relativedelta
doc_text = (
"Acme Corp announced on 3 March 2019 that it had acquired Bolt Systems "
"for $120 million. The deal closed two weeks later. Acme's chief executive, "
"Jane Ruiz, said the acquisition would expand the company's cloud business. "
"Bolt Systems, founded in 2011 by Sam Okafor, employed about 40 people at "
"the time."
)
nlp = spacy.load("en_core_web_sm")
doc = nlp(doc_text)ConsoleOut [12]Expand
Sentences: 4 Tokens: 62 Entities: Acme Corp ORG [0, 9) 3 March 2019 DATE [23, 35) Bolt Systems ORG [57, 69) $120 million MONEY [74, 86) two weeks later DATE [104, 119) Acme ORG [121, 125) Jane Ruiz PERSON [145, 154) Bolt Systems ORG [220, 232) 2011 DATE [245, 249) Sam Okafor PERSON [253, 263) about 40 CARDINAL [274, 282)
The displayed counts come from the installed spaCy model. Its named mentions include repeated organizations and temporal expressions; these are mentions, rather than a deduplicated entity inventory.
Dependency Rules for Subject-Verb-Object Relations
The first rule handles selected subject-verb-object constructions. We build helpers with explicit fallbacks, then iterate over tokens whose lemmas appear in a trigger lexicon. The fallbacks cover the branches shown below, rather than arbitrary parsing variation.
PythonIn [13]Expand
chunk_by_root = {chunk.root: chunk for chunk in doc.noun_chunks}
def span_for(token):
"""Return the noun chunk rooted at `token`, or the token itself."""
chunk = chunk_by_root.get(token)
return chunk.text if chunk is not None else token.text
def subject_head(verb):
for child in verb.children:
if child.dep_ in ("nsubj", "nsubjpass", "csubj"):
return child
# Reduced relative clauses: "Bolt Systems, founded in 2011 ..."
if verb.dep_ in ("acl", "advcl", "relcl") and verb.head.pos_ in (
"NOUN",
"PROPN",
):
return verb.head
return None
def agent_head(verb):
"""Look for a passive agent using the supported dependency labels."""
for child in verb.children:
if child.dep_ == "agent":
return next(
(item for item in child.children if item.dep_ == "pobj"), None
)
for child in verb.children:
if child.dep_ == "prep" and child.lower_ == "by":
for grandchild in child.children:
if grandchild.dep_ == "pobj":
return grandchild
return NoneThe loop emits candidates from a lemma lookup and dependency conditions. The acquisition branch takes the subject as its head and emits a candidate for each supported object child. A passive acquisition with a passive subject and a by-agent but no such child yields no candidate. The founding branch uses the passive-agent helper. If the rule accepts the active subject in “Sam founded Nova” and finds no passive agent, it emits (?, founder_of, Sam). The rule puts the subject in the tail and ignores the direct object. The earlier root, negation, and modality checks illustrate ways to assess factuality. This loop skips those checks and only emits candidates. We keep the trigger's token index for accessing the parsed token. At the pronoun-resolver call site, Token.idx supplies its character offset, which is compared with each mention's Span.end_char.
PythonIn [14]Expand
TRIGGERS = {"acquire": "acquire", "found": "founder_of"}
raw_triples = []
for token in doc:
relation = TRIGGERS.get(token.lemma_.lower())
if relation is None:
continue
subject = subject_head(token)
if subject is None:
continue
if relation == "acquire":
for child in token.children:
if child.dep_ in ("dobj", "obj", "attr"):
raw_triples.append(
{
"head": span_for(subject),
"relation": relation,
"tail": span_for(child),
"trigger_index": token.i,
}
)
else:
agent = agent_head(token)
raw_triples.append(
{
"head": span_for(agent) if agent is not None else "?",
"relation": relation,
"tail": span_for(subject),
"trigger_index": token.i,
}
)ConsoleOut [15]Expand
'it' --[acquire]--> 'Bolt Systems' 'Sam Okafor' --[founder_of]--> 'Bolt Systems'
The dependency rules emitted two candidates. The acquisition candidate still has “it” as its head. That is the grammatical subject of “acquired” in the text. Resolving the pronoun to the buyer takes a separate coreference step.
A Minimal Pronoun Resolver
The worked example uses a minimal heuristic to illustrate one antecedent search rule. The toy resolver selects the recognized ORG, PERSON, or GPE mention with the latest end offset at or before the cutoff. Here the caller uses the trigger offset, which can fall after the pronoun. An eligible mention between the pronoun and trigger can then be selected as the antecedent. It does not test compatibility separately for each pronoun.
PythonIn [16]Expand
PRONOUNS = {"it", "its", "they", "them", "he", "she", "this", "that"}
def resolve_pronoun(mention, char_position):
"""Resolve a toy pronoun using a character offset as the cutoff."""
if mention.lower() not in PRONOUNS:
return mention
best = None
for ent in doc.ents:
if ent.end_char <= char_position and ent.label_ in (
"ORG",
"PERSON",
"GPE",
):
if best is None or ent.end_char > best.end_char:
best = ent
return best.text if best is not None else mention
relations = []
for triple in raw_triples:
relations.append(
{
"head": resolve_pronoun(
triple["head"], doc[triple["trigger_index"]].idx
),
"relation": triple["relation"],
"tail": triple["tail"],
}
)ConsoleOut [17]Expand
'Acme Corp' --[acquire]--> 'Bolt Systems' 'Sam Okafor' --[founder_of]--> 'Bolt Systems'
The display shows candidate tuples. Its acquisition tuple carries neither the reporting clause nor a status field. The resolver chose "Acme Corp", the only recognized mention of an allowed type before this trigger's character cutoff. That answer is correct for this document, but the heuristic cannot distinguish several plausible antecedents or enforce agreement. Broader systems may use learned coreference models or domain rules. Coreference Resolution is a planned later chapter.
Notice also the asymmetry in what the resolver sees: "Bolt Systems" also appears in the text, but after the acquisition trigger, so it falls outside this search cutoff. Direction of resolution matters.
Lexico-Syntactic Patterns for Possessive Appositives
A surface regex can recognize the selected "X's ROLE, Y" construction directly, without a dependency parse. The expression below recognizes two role strings and captures a two-word person name. The capture can be only the prefix of a longer name. Whether it is more accurate or robust than a dependency rule requires a matched evaluation.
PythonIn [18]Expand
ROLE_PATTERN = re.compile(
r"([A-Z][\w&\-]*(?: [A-Z][\w&\-]*)*)'s (?:chief executive|CEO),"
r" ([A-Z][a-z]+ [A-Z][a-z]+)"
)
for organization, person in ROLE_PATTERN.findall(doc_text):
relations.append(
{"head": person, "relation": "chief_executive_of", "tail": organization}
)ConsoleOut [19]Expand
'Acme Corp' --[acquire]--> 'Bolt Systems' 'Sam Okafor' --[founder_of]--> 'Bolt Systems' 'Jane Ruiz' --[chief_executive_of]--> 'Acme'
The demonstration now emits three relation candidates. Note the alias problem in the last one: the organization tail is "Acme", not "Acme Corp". An aggregation that treats the two surface strings as distinct identifiers will keep them separate unless it reconciles the alias. Entity linking or another canonicalization step can perform that reconciliation.
Extracting Numeric Values
To use a price numerically, convert the matched amount to a number and apply its scale. This toy regex searches for a dollar sign, at most one whitespace character, digits or commas, and an optional decimal point followed by digits. It allows whitespace before the optional listed scale. It can match a prefix of a longer string. For example, $5millionth uses the million scale and $5 trillion yields 5 with no scale multiplier. The search can skip a leading minus, reading -$5 million as positive 5 million. A comma-only amount such as $, matches. Removing the comma leaves an empty string, so float raises ValueError.
PythonIn [20]Expand
MULTIPLIER = {"thousand": 1e3, "million": 1e6, "billion": 1e9, None: 1.0}
money_values = []
for amount, scale in re.findall(
r"\$\s?([\d,]+(?:\.\d+)?)\s*(million|billion|thousand)?",
doc_text,
flags=re.IGNORECASE,
):
key = scale.lower() if scale else None
money_values.append(float(amount.replace(",", "")) * MULTIPLIER[key])ConsoleOut [21]Expand
Money mentions: [120000000.0]
The value is 120000000.0 dollars. Numeric values let you sort, compare, and aggregate prices across documents when the amounts share a currency. Convert other currencies to the common unit first.
Temporal Normalization
Now the temporal core. We set the document creation time, then normalize an absolute expression, then a deictic one, then derive the relative one by arithmetic.
PythonIn [22]Expand
DCT = datetime(2019, 3, 25) # document creation time (publication date)
# 1. Absolute expression: independent of the anchor.
announce_date = dateparser.parse("3 March 2019")
# 2. Deictic expression: resolved against the document creation time.
yesterday = dateparser.parse("yesterday", settings={"RELATIVE_BASE": DCT})
# 3. Relative expression: resolved against the antecedent EVENT, not the DCT.
close_date = announce_date + relativedelta(weeks=2)ConsoleOut [23]Expand
Document creation time : 2019-03-25 '3 March 2019' -> 2019-03-03 (absolute, anchor-independent) 'yesterday' -> 2019-03-24 (deictic, anchored to DCT) 'two weeks later' -> 2019-03-17 (relative, anchored to E1) If anchored to DCT instead: 2019-04-08
The same string "two weeks later" produces 17 March with the announcement anchor and 8 April with the document date. The first is the intended closing date in this worked reading.
The same isolated deictic expression, normalized against different base dates, yields different dates:
PythonIn [24]Expand
for base in (datetime(2019, 3, 25), datetime(2024, 1, 15)):
resolved = dateparser.parse("yesterday", settings={"RELATIVE_BASE": base})
print(base.date(), "->", resolved.date())ConsoleOut [24]Expand
2019-03-25 -> 2019-03-24 2024-01-15 -> 2024-01-14
Archive the document creation time when deictic expressions depend on it. Using today's date for "yesterday" in an archived article can produce the wrong result. An absolute date such as "3 March 2019" does not have that dependency.
Assembling the Event Table
We assemble the selected event records by hand for the reasoning example. The closing record reuses the date computed above. The time_kind field distinguishes an exact day, an uncertainty window, and an unknown date. The two endpoints of E3 are bounds on localization, not its duration. At this demonstration's day resolution, an exact day uses equal coordinates. These day points are not proper intervals. Disjoint date bounds can establish before or after; an interval label returned for a degenerate input uses the helper's explicit branch-priority convention described below.
PythonIn [25]Expand
events = pd.DataFrame(
[
{
"event_id": "E1",
"type": "Announcement",
"trigger": "announced",
"role_1": "Speaker: Acme Corp",
"role_2": "Content: acquisition of Bolt Systems",
"time_kind": "exact-day",
"start": datetime(2019, 3, 3),
"end": datetime(2019, 3, 3),
},
{
"event_id": "E2",
"type": "Closing",
"trigger": "closed",
"role_1": "Deal: the acquisition",
"role_2": "-",
"time_kind": "exact-day",
"start": close_date,
"end": close_date,
},
{
"event_id": "E3",
"type": "Founding",
"trigger": "founded",
"role_1": "Founder: Sam Okafor",
"role_2": "Organization: Bolt Systems",
"time_kind": "uncertainty-window",
"start": datetime(2011, 1, 1),
"end": datetime(2011, 12, 31),
},
{
"event_id": "E4",
"type": "Statement",
"trigger": "said",
"role_1": "Speaker: Jane Ruiz",
"role_2": "Content: the acquisition would expand ...",
"time_kind": "unknown",
"start": None,
"end": None,
},
]
)ConsoleOut [26]Expand
event_id type trigger role_1 time_kind start end
E1 Announcement announced Speaker: Acme Corp exact-day 2019-03-03 2019-03-03
E2 Closing closed Deal: the acquisition exact-day 2019-03-17 2019-03-17
E3 Founding founded Founder: Sam Okafor uncertainty-window 2011-01-01 2011-12-31
E4 Statement said Speaker: Jane Ruiz unknown NaT NaTThe selected records preserve missing information. E2 refers to the acquisition event, E3 stores a year-sized uncertainty window, and E4 has no assigned date. Neither an unknown date nor an uncertainty window should be mistaken for a measured event extent.
Allen Relations Over the Events
The allen helper distinguishes the thirteen endpoint relations for proper intervals. The graph builder below first checks for unknown times and disjoint bounds, then withholds an extent relation for overlapping uncertainty windows. Only the remaining pairs are passed to allen.
For proper intervals, exactly one of the thirteen endpoint relations holds. The helper also accepts equal start/end coordinates, as used for exact days here, and resolves overlapping endpoint predicates by the order of its if branches. For example, the point (2, 2) against (0, 2) returns met-by before reaching the finishes test, although both displayed endpoint predicates hold. That is an implementation convention for points, not a unique Allen relation. Use a point/interval formalism with explicit semantics if those mixed cases matter to the application.
PythonIn [27]Expand
def allen(a, b):
"""Classify proper intervals; point inputs follow the ordered branch priority."""
a0, a1 = a
b0, b1 = b
if a0 > a1 or b0 > b1:
raise ValueError("Interval start must not exceed its end")
if a0 == b0 and a1 == b1:
return "equals"
if a1 < b0:
return "before"
if b1 < a0:
return "after"
if a1 == b0:
return "meets"
if b1 == a0:
return "met-by"
if a0 == b0:
return "starts" if a1 < b1 else "started-by"
if a1 == b1:
return "finishes" if a0 > b0 else "finished-by"
if a0 > b0 and a1 < b1:
return "during"
if a0 < b0 and a1 > b1:
return "contains"
if a0 < b0 < a1 < b1:
return "overlaps"
return "overlapped-by"
temporal_graph = []
ids = events["event_id"].tolist()
for i in range(len(ids)):
for j in range(i + 1, len(ids)):
a = events.iloc[i]
b = events.iloc[j]
if a["time_kind"] == "unknown" or b["time_kind"] == "unknown":
relation = "undetermined"
elif a["end"] < b["start"]:
relation = "before"
elif b["end"] < a["start"]:
relation = "after"
elif "uncertainty-window" in (a["time_kind"], b["time_kind"]):
relation = "undetermined"
else:
relation = allen((a["start"], a["end"]), (b["start"], b["end"]))
temporal_graph.append((ids[i], relation, ids[j]))ConsoleOut [28]Expand
E1 before E2 E1 after E3 E1 undetermined E4 E2 after E3 E2 undetermined E4 E3 undetermined E4
Each label describes the earlier row by position in the table relative to the later row in that pair, regardless of the DataFrame’s index labels. The event IDs name those rows. E1 is before E2 but after E3, and E2 is also after E3. The founding window is entirely earlier than the 2019 dates, so these orderings hold despite its unknown exact date. The supplied date fields leave pairs involving E4 undetermined. If known bounds overlap or touch and either event has an uncertainty window, the code also returns undetermined.
For two known, positive-duration event extents, such as a tenure that contains a merger period, the allen helper can distinguish relations that start dates alone cannot. E3's window instead bounds a possible founding date.
Visualizing the Timeline
Before plotting, we precompute the coordinates. Keeping data preparation out of the visualization cell means the figure can be re-rendered for a different theme without touching any extraction logic.
PythonIn [29]Expand
import matplotlib.dates as mdates
timeline_rows = []
for _, row in events.iterrows():
if row["time_kind"] == "unknown":
continue
timeline_rows.append(
{
"label": f"{row['event_id']}: {row['type']}",
"x0": mdates.date2num(row["start"]),
"x1": mdates.date2num(row["end"]),
"time_kind": row["time_kind"],
}
)
dct_num = mdates.date2num(DCT)
x_min = mdates.date2num(datetime(2010, 6, 1))
x_max = mdates.date2num(datetime(2019, 12, 31))VisualizationOut [30]Expand
E3's hatched bar marks where its unknown founding date can fall. E1 and E2 lie two weeks apart on the horizontal axis, and the row labels give their exact days. The selected record for E4 has no date or plotted localization window, so the figure omits it. The dashed publication marker at 25 March sits eight days after Closing on 17 March. Those dates look close together on an axis that also includes 2011. The figure preserves both known dates and the limits of what the paragraph establishes.
Visualizing Allen's Relations
To see the endpoint comparisons, plot the seven rows from the table as interval pairs.
PythonIn [31]Expand
allen_demo = [
("before", (0, 2), (3, 5)),
("meets", (0, 2), (2, 4)),
("overlaps", (0, 3), (2, 5)),
("starts", (0, 3), (0, 5)),
("during", (1, 2), (0, 4)),
("finishes", (2, 4), (0, 4)),
("equals", (0, 4), (0, 4)),
]VisualizationOut [32]Expand
Notice how meets and overlaps differ by a single endpoint comparison, and why a classifier distinguishing these relations must compare the endpoints correctly. A timestamp near midnight can fall on different calendar dates in different time zones; an off-by-one arithmetic error can also move an endpoint by one day. An error in a relevant endpoint can change meets to overlaps, or the reverse. A common shift of both intervals preserves their relation.
LLM-Based Extraction and Grounding Verification
A prompt can ask a language model to extract structured records. Here we write one and check selected fields in a partial, hand-written response with two deliberate alterations.
PythonIn [33]Expand
SCHEMA = {
"relations": ["acquire", "founder_of", "chief_executive_of"],
"events": {
"Acquisition": ["Buyer", "Acquired", "Price"],
"Announcement": ["Speaker", "Content"],
"Closing": ["Deal"],
"Founding": ["Founder", "Organization"],
"Statement": ["Speaker", "Content"],
},
}
prompt = f"""Extract relations and events from the document below.
Relation types: {", ".join(SCHEMA["relations"])}
Event types and roles: {json.dumps(SCHEMA["events"])}
Return JSON with two keys, "relations" and "events". For every relation give
head, relation, tail, and an evidence string copied exactly from the document. Include a relation only if the document supports it.
For every event give type, trigger, roles, and time in ISO 8601 at the
precision supported by the document. Use null when the time is unknown.
Document:
{doc_text}
"""ConsoleOut [34]Expand
Extract relations and events from the document below.
Relation types: acquire, founder_of, chief_executive_of
Event types and roles: {"Acquisition": ["Buyer", "Acquired", "Price"], "Announcement": ["Speaker", "Content"], "Closing": ["Deal"], "Founding": ["Founder", "Organization"], "Statement": ["Speaker", "Content"]}
Return JSON with two keys, "relations" and "events". For every relation give
head, relation, tail, and an evidence string copied exactly from the document. Include a relation only if the document supports it.
For every event give type, trigger, roles, and time in ISO 8601 at the
precision supported by the document. Use null when the time is unknown.
Document:
Acme Corp announced on 3 March 2019 that it had acquired Bolt Systems for $120 million. The deal closed two weeks later. Acme's chief executive, Jane Ruiz, said the acquisition would expand the company's cloud business. Bolt Systems, founded in 2011 by Sam Okafor, employed about 40 people at the time.The prompt names allowed labels, requests copied evidence for each relation, and asks for a normalized time at the available precision or null for unknown time. The event time has its own field. The prompt leaves out the publication date and the worked choice to anchor "two weeks later" to the announcement. The check below covers the announcement’s absolute date. The requested schema has no event status field for distinguishing a completed event from a plan or a denial. Its acquisition relation has no field for who made the claim. The prompt alone does not constrain the decoder. A prompted model can still return an unknown label, invalid JSON, or unsupported evidence. A schema validator can check structure and labels when configured with the corresponding rules. Constrained decoding can enforce syntax and allowed productions. Checking truth requires evidence for the record.
The following hand-written response includes only the announcement in its event list. It contains an unsupported organization suffix and an incorrect announcement date. The prompt leaves the naming policy for relation arguments open. The executive row uses the full name Acme Corp for the Acme alias in its copied evidence. A policy that requires copied argument strings would keep Acme in that row. The document-wide substring check finds Acme Corp elsewhere. Checking the alias mapping and the support within the cited evidence requires separate validation.
PythonIn [35]Expand
model_output = {
"relations": [
{
"head": "Acme Corp",
"relation": "acquire",
"tail": "Bolt Systems",
"evidence": "Acme Corp announced on 3 March 2019 that it had acquired Bolt Systems",
},
{
"head": "Jane Ruiz",
"relation": "chief_executive_of",
"tail": "Acme Corp",
"evidence": "Acme's chief executive, Jane Ruiz",
},
{
"head": "Sam Okafor",
"relation": "founder_of",
"tail": "Bolt Systems Inc.",
"evidence": "founded in 2011 by Sam Okafor",
},
],
"events": [
{
"type": "Announcement",
"trigger": "announced",
"time": "2019-03-01",
"roles": {"Speaker": "Acme Corp", "Content": "the acquisition"},
},
],
}
GROUNDED_FIELDS = ("head", "tail", "evidence")
def normalize_surface(text):
return " ".join(text.lower().split())
def ungrounded_fields(record, text, fields):
haystack = normalize_surface(text)
return {
field: record[field]
for field in fields
if isinstance(record.get(field), str)
and normalize_surface(record[field]) not in haystack
}
grounding_report = [
ungrounded_fields(rel, doc_text, GROUNDED_FIELDS)
for rel in model_output["relations"]
]ConsoleOut [36]Expand
relation 0: no absent string reported
relation 1: no absent string reported
relation 2: absent strings: {'tail': 'Bolt Systems Inc.'}
Rows without an absent selected string: 2/3The third relation fails the check: the text says "Bolt Systems", not "Bolt Systems Inc.". This hand-written response adds an unsupported suffix; it is not an observed model error or evidence about where a model obtained the suffix. An exact-string aggregation could create a duplicate node, although entity linking might reconcile it. A metric that checks only relation labels would miss the argument error.
The substring check reports supplied string fields that are absent after normalization. All three selected fields in each fixture record are nonempty strings. For other responses, check missing, blank, and non-string values separately before interpreting the count. A wrong relation can pass when its strings occur. In a separate example, a supplied tail of Bolt System passes because it is a substring of Bolt Systems in the document. The check cannot establish offsets, semantic roles, or normalized dates. The response gives 2019-03-01 for the announcement. The next cell reads the first and only event, the hand-written Announcement, and compares its day with the day recomputed from the selected source phrase "3 March 2019". The code assumes that this field contains a day value. An empty event list raises an indexing error before either date is parsed. For arbitrary responses, check that the intended event exists, align it with its evidence, and handle unknown or reduced-precision times before comparing days. The acquisition date remains unknown.
PythonIn [37]Expand
extracted_time = datetime.fromisoformat(model_output["events"][0]["time"])
evidence_span = "3 March 2019"
recomputed = dateparser.parse(evidence_span)
time_ok = recomputed.date() == extracted_time.date()ConsoleOut [38]Expand
Response time : 2019-03-01 Evidence : '3 March 2019' Recomputed time: 2019-03-03 Match : False
The example checks substring presence for selected strings and recomputes one normalized date. The demonstrated substring and date checks require no language model. Their pass/fail flags record these two checks; they are not calibrated confidence scores or judgments of whether the source supports the relation.
For LLM extraction, retain evidence spans and recompute values whose derivations are explicit. Report what each check measures, such as selected-string presence or date agreement. Semantic precision requires correctness judgments under a stated policy, for example against reviewed gold annotations. For a nonempty output set, judging every prediction gives exact precision. Judging a uniform random sample of at least one prediction estimates the same fraction. Substring presence alone provides no precision estimate.
Comments
Add to the conversation
A question, a correction, or an example from your own work.
No comments yet. Be the first to share your thoughts!