PropBank (2005) › The Problem

By 2005, statistical NLP systems had made substantial progress in syntactic parsing, but syntax alone could not identify who did what to whom. Consider the sentences "John gave Mary a book" and "Mary received a book from John." Their syntax differs, yet both describe the same transfer of possession. A syntactic parser could identify grammatical relationships without recording that John is the agent, Mary the recipient, and the book the transferred theme. Those roles support tasks such as information extraction, question answering, and machine translation.

The problem extended beyond individual sentences. Different syntactic realizations could express the same semantic roles. In "The company bought the subsidiary," the subject "The company" is the buyer. But in "The subsidiary was bought by the company," the buyer appears in a prepositional phrase, while the object becomes the subject. Systems needed to recognize that despite different syntactic structures, the semantic roles remain the same. Traditional resources couldn't help with this. Syntactic parsers provided tree structures but not semantic roles. WordNet provided word relationships but not event structure. The gap between syntactic analysis and semantic understanding constrained NLP systems' capabilities.

This limitation became increasingly problematic as systems attempted more sophisticated applications. Information extraction systems needed to identify entities and their relationships, but without understanding semantic roles, they struggled to determine whether a person mentioned in a sentence was an agent, a patient, a beneficiary, or some other participant. Question answering systems needed to understand who performed actions, on what objects, for what purposes, but existing resources provided no framework for encoding this information systematically. Machine translation systems needed to preserve semantic relationships across languages, but word-level resources couldn't capture the event structures that verbs and other predicates evoked.

FrameNet, released in 1998, had demonstrated that semantic role annotation was valuable and feasible. FrameNet's frame-based approach with named roles like Buyer, Seller, and Goods provided detailed semantic structures, and it showed that large-scale semantic annotation of corpus data could produce useful resources. However, FrameNet's annotation process was labor-intensive, requiring detailed frame definitions for each lexical unit. By 2005, FrameNet had annotated around 13,000 lexical units, but English vocabulary includes hundreds of thousands of words. The coverage gap limited FrameNet's usefulness for unrestricted text processing, and its frame-based approach required substantial annotation effort for each new lexical unit.

Researchers at the University of Pennsylvania recognized that a different approach could provide broader coverage more efficiently. Rather than creating detailed named frames for each verb, a simpler scheme using numbered arguments could scale more effectively while still capturing the essential semantic information needed for semantic role labeling. This scheme could build directly on the Penn Treebank's existing syntactic annotations, avoiding the need to create a separate resource from scratch. The challenge was designing an annotation scheme that was simple enough to scale broadly but precise enough to capture meaningful semantic distinctions. PropBank addressed this challenge by combining verb-specific frame files with a general numbering scheme for arguments.