Conceptual Dependency - Canonical Meaning Representation for Natural Language Understanding

By the late 1960s, researchers working on natural language understanding systems faced a fundamental representational challenge. Programs could parse sentences to identify grammatical structures, but parsing alone didn't provide meaning. When a system encountered "John gave Mary a book" and "Mary received a book from John," it treated these as completely different sentences with different parse trees, despite expressing the same core meaning. The surface form of language, the particular words and grammatical constructions used, varies dramatically even when describing identical events. To build systems that truly understood language, researchers needed a way to capture meaning independently of linguistic form.

Roger Schank, then at Stanford University's Artificial Intelligence Laboratory, recognized that the key to language understanding lay in finding a canonical representation for meaning. Rather than working directly with natural language syntax, systems should translate sentences into a language-independent representation that captured what happened, not just how it was expressed. This representation would serve as an interlingua, a common meaning representation into which different surface forms could be translated and from which inferences could be drawn. In 1969, Schank introduced Conceptual Dependency, a theory that represented sentences as structured networks of conceptual primitives: actions, objects, states, and the relationships between them.

Conceptual Dependency proposed that the meaning of any sentence could be expressed using a small set of primitive actions like TRANSFER, PROPEL, INGEST, MOVE, and ATTEND. These primitives, combined with conceptual cases (roles like actor, object, recipient, direction), created a representation that was both language-independent and inference-friendly. "John gave Mary a book" and "Mary received a book from John" would both map to the same Conceptual Dependency structure: a TRANSFER action with John as actor, book as object, and Mary as recipient. This canonical form allowed systems to recognize semantic equivalence, draw inferences, and answer questions about the meaning of sentences rather than just matching their surface structure.

The framework's influence on natural language understanding was immediate and lasting. Schank's research group at Yale University (where he moved in 1974) developed several systems that used Conceptual Dependency, including MARGIE, SAM, and PAM, programs that could understand stories, answer questions, and make inferences. The theory demonstrated that meaning representation could be separated from syntactic form, establishing a pattern that would persist through decades of semantic parsing research. More fundamentally, it showed that language understanding required structured representations of events and relationships, not just syntactic trees.

The challenge that motivated Conceptual Dependency appeared whenever natural language systems tried to perform semantic reasoning. Traditional parsing systems produced syntactic structures, trees showing how words grouped into phrases and how phrases combined into sentences. But these structures didn't capture meaning in a way that enabled inference or cross-linguistic equivalence. Consider the sentences "The cat chased the mouse," "The mouse was chased by the cat," and "It was the cat that chased the mouse." All describe the same event, but traditional parsers treated them as completely distinct structures.

The problem became acute when systems tried to answer questions or make inferences. If a system understood "John gave Mary a book," it should be able to answer "What did John give to Mary?" or "Who received a book from John?" without needing to parse these questions and match them against the original sentence's syntax. It should recognize that "Mary received a book from John" means the same thing, even though the grammatical structure is reversed. And it should be able to infer that Mary now has the book, even though this information wasn't explicitly stated.

Early natural language systems attempted various solutions. Some tried to encode meaning directly in syntactic rules, creating grammar rules that included semantic annotations. But this conflated syntax and semantics in ways that made grammars unwieldy and difficult to maintain. Other systems used predicate logic as a meaning representation, translating sentences into logical formulas. But predicate logic was too fine-grained, creating different representations for semantically equivalent sentences that differed only in how information was expressed. The sentence "John killed Mary" might be represented as $kill(john, mary)$ while "Mary was killed by John" became $killed\_by(mary, john)$, obscuring the fact that these expressed the same event.

A deeper problem was inference. To answer questions or understand discourse, systems needed to draw conclusions from what was explicitly stated. If the system knew that "John gave Mary a book," it should infer that Mary now possesses the book, that John no longer has it (unless he had multiple copies), and that the transfer happened in the past. Predicate logic could represent these inferences, but only with extensive axioms about the properties of actions. What was needed was a representation where such inferences emerged naturally from the structure of the representation itself.

The representation also needed to handle paraphrases and translations. Natural language provides many ways to express the same meaning. "John sold Mary a car" and "Mary bought a car from John" describe the same transaction from different perspectives. A robust meaning representation should capture what happened, the transfer of ownership and money, in a way that made these paraphrases obviously equivalent. This required moving beyond surface structure to a deeper, more abstract level of representation.

Conceptual Dependency solved the meaning representation problem by introducing a small set of primitive actions that could be combined to represent any event. These primitives were chosen to be at the right level of abstraction: specific enough to distinguish different types of events, but general enough to capture semantic equivalence across diverse surface forms. The theory proposed approximately a dozen primitive actions, including:

• ATRANS: Transfer of abstract relationships (possession, ownership, control)
• PTRANS: Transfer of physical location (movement of objects)
• PROPEL: Application of physical force
• MOVE: Movement of a body part
• GRASP: Physical grasping of an object
• INGEST: Taking something into the body
• EXPEL: Removing something from the body
• MTRANS: Transfer of mental information
• MBUILD: Construction of new thoughts from old ones
• SPEAK: Production of sounds
• ATTEND: Focusing sense organs on a stimulus

Each action had associated conceptual cases, slots that could be filled with objects, actors, recipients, and other roles. These cases were similar to semantic roles or thematic roles in linguistics, but specifically designed for computational inference. The cases included:

• ACTOR: The entity performing the action
• OBJECT: The thing being acted upon
• TO and FROM: Direction of transfer (for TRANS actions)
• INSTRUMENT: The means used to perform the action
• RECIPIENT: The entity receiving something
• DIRECTION: Physical or abstract direction of movement

A Conceptual Dependency structure was a network connecting these primitives and cases. "John gave Mary a book" would be represented as an ATRANS action (transfer of possession) with John as ACTOR, book as OBJECT, and Mary as RECIPIENT (TO case). The structure might look like:

This same structure could represent "Mary received a book from John," "John passed a book to Mary," or any other paraphrase expressing the transfer of possession. The canonical form made semantic equivalence transparent.

The theory also included state primitives, conceptualizations of conditions rather than events. States included physical states (like location, possession) and mental states (like knowing, wanting). A sentence like "John has a book" would be represented as a state primitive indicating possession: John is in a POSS-BY state with respect to the book. These states provided a way to represent the results of actions and the conditions that enable or trigger them.

Inference rules operated directly on Conceptual Dependency structures. The system could infer consequences of actions: an ATRANS from ACTOR to RECIPIENT implied a change in the POSS-BY state, with the RECIPIENT gaining possession and the ACTOR losing it. Causal chains could be constructed: if someone PROPELLED an object toward a location, the object PTRANSed (moved) to that location. These inference rules were built into the representation itself, making common-sense reasoning more straightforward than with predicate logic where such rules had to be explicitly stated.

The representation handled complex sentences through embedding. "John persuaded Mary to give him the book" involved multiple actions: a MTRANS (mental transfer) representing the persuasion, with the content of that transfer being another Conceptual Dependency structure (the ATRANS of the book). This recursive structure allowed the theory to represent nested events and mental attitudes.

The first major application of Conceptual Dependency was MARGIE (Memory, Analysis, Response Generation, and Inference on English), developed by Schank and his colleagues at Stanford in the early 1970s. MARGIE took English sentences as input, parsed them, converted them to Conceptual Dependency structures, stored these structures in memory, and could then answer questions and generate paraphrases based on what it understood.

MARGIE demonstrated several capabilities that would become central to natural language understanding systems. It could recognize paraphrases: given "John gave Mary a book" and "Mary received a book from John," MARGIE would identify these as expressing the same meaning because they mapped to identical Conceptual Dependency structures. It could answer questions: asked "What did John give to Mary?" it would search its memory for Conceptual Dependency structures with John as ACTOR and Mary as RECIPIENT in an ATRANS, then extract the OBJECT case. It could make inferences: if told "John gave Mary a book," MARGIE could infer that Mary now possesses the book, even though this wasn't explicitly stated.

The system also generated paraphrases. Given a stored Conceptual Dependency structure, it could produce multiple English sentences expressing that meaning. This demonstrated that Conceptual Dependency truly served as an interlingua, a common representation from which various surface forms could be generated, just as it could be used to parse various surface forms into a common representation.

SAM (Script Applier Mechanism), developed at Yale in the mid-1970s, extended Conceptual Dependency to handle stories by incorporating scripts, expectation structures representing stereotypical sequences of events. Given a story like "John went to a restaurant. He ordered a hamburger. He ate it. He left," SAM would use its restaurant script to fill in expected events that weren't explicitly mentioned: John sat down, a waiter brought the hamburger, John paid the bill. These inferences came from understanding the Conceptual Dependency structures of explicitly mentioned events and recognizing which script they activated.

Scripts were essentially networks of Conceptual Dependency structures representing typical event sequences. When SAM encountered "John went to a restaurant," it activated its restaurant script, which included expectations about subsequent events (ordering, eating, paying) and their typical Conceptual Dependency representations. As the story unfolded, SAM matched events to script expectations, filling in missing information and resolving references based on script roles.

PAM (Plan Applier Mechanism), also developed at Yale, addressed stories that didn't fit stereotypical scripts by using plans, goal-directed sequences of actions. When PAM encountered a story about someone trying to achieve a goal, it would infer the plan being followed and use that to understand and predict actions. Plans were represented using Conceptual Dependency structures, with goals linked to the actions that could achieve them.

These systems demonstrated that Conceptual Dependency could support complex inference and discourse understanding. By representing meaning at the right level of abstraction, the theory enabled systems to recognize semantic equivalence, draw commonsense conclusions, and understand extended texts beyond individual sentences.

Despite its theoretical elegance, Conceptual Dependency faced practical limitations that would become evident as systems scaled to more complex domains. The most significant was the need to hand-craft representations. Converting sentences to Conceptual Dependency structures required extensive knowledge engineering: lexical rules mapping words to primitives, case assignment rules determining which entities filled which cases, and inference rules connecting actions to their consequences and prerequisites.

This knowledge engineering proved time-consuming and error-prone. The lexicon had to be manually constructed, with entries specifying how each verb mapped to Conceptual Dependency primitives. But verbs are polysemous. For example, "bank" could mean financial institution or river edge, and the mapping depended on context that wasn't always easy to determine. Creating a comprehensive lexicon for unrestricted text required enumerating countless word senses and their mappings, a task that didn't scale.

The choice of primitives also proved problematic. Schank's approximately dozen primitive actions were sufficient for many common events, but some concepts resisted decomposition into these primitives. Complex mental states, subtle social interactions, and abstract concepts often required awkward or unnatural representations using the available primitives. The theory's claim that all events could be decomposed into a small set of primitives worked well for physical actions but seemed less natural for cognitive or social phenomena.

The representation also struggled with ambiguity. Natural language sentences are often ambiguous: structurally ambiguous (different parse trees), lexically ambiguous (different word senses), or scope ambiguous (different quantifier or modifier scopes). A single sentence might map to multiple Conceptual Dependency structures, and the theory provided no principled way to choose among them. Systems relied on heuristics, scripts, or world knowledge to resolve ambiguity, but these weren't part of the core theory.

Inference, while more straightforward than with predicate logic, still required extensive hand-coded rules. Each type of inference, including consequences of actions, prerequisites, and causal chains, needed explicit rules. As the domain expanded, the number of inference rules grew, and interactions between rules became difficult to manage. The theory's promise of natural inference from representation structure was only partially realized.

The canonical form assumption, that semantically equivalent sentences should map to identical structures, also proved too strong in practice. "John killed Mary" and "Mary died because of John's actions" express related but not identical meanings. The first emphasizes John's agency, the second emphasizes Mary's death. Sometimes these differences matter for understanding. Forcing everything into a single canonical form could lose important nuances.

Finally, the theory lacked a clear learning mechanism. Unlike later statistical approaches that could learn representations from data, Conceptual Dependency systems required all knowledge to be manually encoded. This made them inflexible and unable to adapt to new domains or usages without extensive reengineering.

Conceptual Dependency's most lasting influence was demonstrating that meaning representation could be separated from syntactic form and that structured representations enabled inference and semantic equivalence recognition. These insights became fundamental to semantic parsing research, even as the specific primitives and notation evolved.

Frame semantics, developed by Charles Fillmore in the 1970s, shares Conceptual Dependency's emphasis on structured event representations with roles. Frames are similar to Conceptual Dependency structures but more linguistically motivated, with roles determined by the semantics of verbs rather than a fixed set of cases. Modern frame-semantic parsers, like those in the PropBank project, annotate sentences with frame structures that closely parallel Conceptual Dependency's action-role framework.

Abstract Meaning Representation (AMR), developed in the 2010s, represents sentences as directed acyclic graphs with labeled edges, a direct conceptual descendant of Conceptual Dependency's network structures. AMR uses a different set of primitives (rooted in linguistic predicates rather than cognitive primitives) but maintains the same goal: a canonical, language-independent representation that captures meaning rather than form. Like Conceptual Dependency, AMR enables paraphrase recognition and inference through structural matching.

Semantic role labeling, now a standard task in NLP, identifies the semantic roles of entities in sentences, answering questions like who did what to whom, where, when, and why. This task formalizes what Conceptual Dependency did with its cases: identifying actors, objects, recipients, instruments, and other roles. Modern systems use statistical or neural methods to label roles automatically, but the fundamental question, what semantic roles appear in events, remains the same one Conceptual Dependency addressed.

In knowledge representation, Conceptual Dependency's influence appears in event ontologies and knowledge graphs. Modern knowledge graphs represent events with predicates and arguments, maintaining structured representations similar to Conceptual Dependency's action-case structures. When a knowledge graph represents "John gave Mary a book" as give(John, Mary, book) with additional edges indicating this is a transfer event, it's using the same conceptual structure, just with different notation.

The theory's emphasis on inference from structured representations also persists. Modern question-answering systems that perform semantic parsing convert questions and documents into structured representations, then match these structures to find answers. Knowledge-base question answering systems use structured queries over knowledge graphs, essentially performing inference over structures that parallel Conceptual Dependency representations.

Neural approaches have largely replaced hand-crafted primitives with learned representations. But the question they address, how to represent meaning in a form that supports reasoning and equivalence recognition, remains the same. Transformer models learn distributed representations that capture semantic similarities, enabling paraphrase recognition and inference through learned similarity rather than structural matching. But the goal is unchanged: finding representations where semantically similar inputs produce similar representations.

In dialogue systems and conversational AI, the need for canonical meaning representations persists. Systems must recognize that "book a flight to Paris" and "I want to fly to Paris" express the same intent, even with different surface forms. Modern systems use learned embeddings or structured intent representations, but the fundamental challenge Conceptual Dependency addressed, mapping diverse expressions to a common meaning space, remains central.

Contemporary semantic parsing research continues to explore the questions Conceptual Dependency raised. Universal Decompositional Semantics attempts to decompose lexical meanings into primitives learned from data rather than hand-crafted. This work maintains Conceptual Dependency's decompositional approach but uses machine learning to discover primitives and mappings rather than requiring manual specification.

Frame-based parsing systems explicitly model event structures with roles, maintaining Conceptual Dependency's structured approach while using statistical methods for parsing and role assignment. PropBank, FrameNet, and related projects provide annotated resources that enable data-driven learning of frame structures, addressing the knowledge acquisition bottleneck that limited Conceptual Dependency systems.

Abstract Meaning Representation has become a standard for meaning representation in computational linguistics, used in tasks ranging from machine translation to question answering. AMR graphs serve as interlinguas, just as Conceptual Dependency structures did, enabling paraphrase recognition and cross-linguistic semantic analysis. Modern AMR parsers use neural networks to automatically convert sentences to AMR structures, combining Conceptual Dependency's representational insights with contemporary machine learning.

The theory's influence also appears in cognitive science and psycholinguistics, where researchers investigate whether human conceptual understanding uses primitive-like structures. The question of whether concepts decompose into more basic primitives remains active, though evidence suggests human conceptual structure may be richer and more flexible than the fixed primitives Conceptual Dependency proposed.

In practical NLP systems, structured meaning representations derived from Conceptual Dependency's insights continue to be important. Semantic role labeling remains a core preprocessing step for many NLP pipelines. Knowledge extraction systems that build structured representations of events in text use role-based frameworks directly descended from Conceptual Dependency's cases. Question-answering systems perform semantic parsing to structured forms, then reason over these structures to find answers.

The contrast between Conceptual Dependency's hand-crafted approach and modern learned representations highlights an important shift in NLP. Early systems like MARGIE, SAM, and PAM demonstrated what was possible with structured meaning representations, but their reliance on manual knowledge engineering limited their scalability. Modern systems learn representations from data, achieving broader coverage but sometimes losing the explicit structure and interpretability that made Conceptual Dependency powerful for inference.

When Roger Schank introduced Conceptual Dependency in 1969, he proposed a solution to a fundamental problem in natural language understanding: how to represent meaning in a form that enabled inference, paraphrasing, and cross-linguistic equivalence. The theory's core insight, that meaning could be captured using structured networks of primitive actions and roles, proved remarkably durable, even as the specific primitives and computational methods evolved.

Conceptual Dependency demonstrated that language understanding required more than syntactic parsing. Systems needed canonical meaning representations that abstracted away from surface form, enabling recognition that "John gave Mary a book" and "Mary received a book from John" express the same meaning. This insight became fundamental to semantic parsing, knowledge representation, and modern NLP systems.

The theory's limitations, including the knowledge acquisition bottleneck, the difficulty of choosing appropriate primitives, and the challenge of handling ambiguity, highlighted problems that subsequent research would address. But the questions it raised remain central: How should meaning be represented? What primitives or structures best capture semantic content? How can systems recognize semantic equivalence across diverse surface forms?

Modern NLP has largely moved away from hand-crafted primitives toward learned representations, but the fundamental challenge Conceptual Dependency addressed persists. Whether using frame semantics, Abstract Meaning Representation, semantic role labeling, or neural embeddings, contemporary systems still seek representations that capture meaning independently of form and enable reasoning about content. Conceptual Dependency provided an early, influential answer to these questions, establishing patterns that would shape decades of research in meaning representation and semantic understanding.

Ready to test your understanding of Conceptual Dependency? This quiz covers the key concepts, primitive actions, applications, limitations, and historical significance of this foundational theory for meaning representation in natural language understanding. Challenge yourself and see how well you've grasped these important ideas!