Montague Semantics (1973) › Compositionality in the Neural Age

The tension between Montague-style symbolic compositionality and neural distributional semantics has sparked interesting recent research. One line of work attempts to inject compositional structure into neural models. Tree-structured neural networks, recursive neural networks, and neural module networks all try to build representations of complex expressions by explicitly combining representations of parts, mimicking the compositional structure of Montague's framework. These models often outperform flat neural models on tasks requiring systematic generalization.understanding novel combinations of known parts.

Another line of work studies whether standard neural models learn compositional representations implicitly. Recent analyses of transformer models suggest they capture compositional structure to some degree: representations of complex phrases reflect systematic combinations of their constituent representations. The attention mechanism in transformers can be viewed as learning soft compositional functions, attending to relevant parts and combining their representations. While this differs from Montague's hard, rule-based composition, it captures a similar intuition: meaning is built from parts, and the building process follows learnable patterns.

The debate over compositional versus holistic representation echoes older debates in cognitive science. Fodor and Pylyshyn's famous argument for compositional mental representations paralleled Montague's linguistic arguments: thought is productive and systematic, suggesting that complex concepts compose from simpler ones. Connectionists responded that neural networks could exhibit productivity through continuous, distributed representations without explicit compositional structure. This debate, initially about mental architecture, now plays out in language AI: do we need explicit symbolic composition (the Montague approach) or can statistical learning over distributed representations suffice (the neural approach)?

The practical answer seems to be: both have value. For tasks requiring precise semantic interpretation.translating natural language to code, answering complex logical questions, interfacing with structured databases.explicit compositional semantics remains valuable. For tasks requiring robustness to variation, learning from data, and handling the statistical regularities of natural language use.machine translation, sentiment analysis, text generation.neural distributional approaches excel. Hybrid systems that combine both, using neural networks to handle surface variation and ambiguity while outputting compositional semantic representations, show particular promise.