Grammars and Constituency Structure › Limitations & Impact

Context-free grammars are a major success of formal language theory. They give a precise definition of phrase structure, they support efficient parsing algorithms, and they connect syntax to recursion in a way that regular expressions cannot. CKY, Earley parsing, and probabilistic chart parsing all build on the CFG framework. The Penn Treebank paper describes the annotation of skeletal constituency structures used for syntactic analysis and parser research. The formalism also influenced compiler design, where context-free grammars describe programming language syntax and parser generators turn those grammars into working parsers.

But a CFG is a model, not a complete theory of human language. Its limitations become visible as soon as you try to cover real text. Our symbolic grammar lists its lexical rules explicitly. Broader coverage requires more entries or a strategy for unfamiliar words. A word may have multiple categories, and a fixed list cannot anticipate every new word. A grammar developed for one domain may under-cover text from another domain. Hand-written rules also encode assumptions about what is grammatical, and those assumptions can be brittle in the face of typos, fragments, code-switching, and informal syntax. Treebanks reduce the manual burden, but annotation is expensive and annotators disagree about borderline cases.

The formal power of CFGs is both a strength and a weakness. CFGs capture nested constituents and arbitrarily deep recursion. A small grammar with coarse categories may nevertheless omit agreement, case, or lexical valency constraints; finite category refinement can encode many such distinctions. The issue is the chosen grammar and the particular dependency pattern, not an inability to express any unbounded dependency. A rule like $S \to NP\ VP$ does not require the subject and verb to agree in number. You can encode agreement by splitting categories into $SgNP$, $PlNP$, $SgVP$, and so on. This duplicates compatible productions; with several independent finite features, their category combinations can multiply. Feature-based grammars and unification-based formalisms add features and constraints to handle these dependencies more compactly. Shieber (1985) presents evidence for weak non-context-freeness from Swiss German cross-serial constructions, under the linguistic assumptions made in that argument. Tree-adjoining grammars are one way to extend context-free generative capacity while retaining polynomial parsing. Generative capacity and parsing complexity must be distinguished: even weakly equivalent formalisms can differ when grammar size is included in the analysis, as Kuhlmann, Satta, and Jonsson (2018) show for one CCG formalism versus TAG. Capacity and complexity claims require a specified formalism and explicit restrictions. The debate is subtle, and the practical lesson is not that CFGs are useless. It is that CFGs are a powerful approximation whose limits depend on which linguistic phenomena matter for the task.

Ambiguity is another fundamental limitation. A grammar can generate many trees for the same sentence, and the grammar alone cannot choose among them. PCFGs add probabilities, but their independence assumptions are strong. Rule probabilities depend only on the left-hand side, not on the surrounding lexical context or the larger discourse. A PCFG with coarse nonterminal categories shares rule probabilities across contexts that its category labels do not distinguish. Adding lexical or structural information to those labels can refine the model. Some neural chart-based constituency parsers use learned structural scores in place of local PCFG rule probabilities while retaining chart decoding. Other parsing approaches use different search procedures, so this architecture should not be attributed to every modern parser. The move from symbolic grammars to statistical and neural parsing is not a rejection of constituency. It is a change in how constituency is scored and induced.

The impact of CFGs is therefore double-edged. They gave NLP a formal backbone for syntax, a family of parsing algorithms, and a way to evaluate structural predictions. They also revealed the limits of pure symbolic description. A grammar can tell you that a sentence has two possible structures. It cannot tell you which one a speaker intended. Inferring an intended reading may require context, world knowledge, or clarification from the speaker. The next chapters take up these threads. Dependency syntax offers a different view of structure, constituency and dependency parsing turn grammars into practical predictors, and structured prediction and parsing evaluation give us the tools to measure whether those predictors are any good.