Lesk Algorithm (1983) › Legacy in Modern Language AI

The Lesk algorithm occupies an interesting place in the history of language AI: it's simultaneously obsolete and foundational. Few modern systems use the original algorithm; its performance has been surpassed by supervised machine learning approaches, particularly deep learning models trained on sense-annotated corpora. Modern word sense disambiguation systems achieve accuracy rates twenty to thirty percentage points higher than classic Lesk variants by using contextual embeddings, attention mechanisms, and millions of labeled examples. In purely practical terms, Lesk has been superseded.

Yet its conceptual legacy permeates contemporary language AI. The central insight.that context provides evidence for meaning, and that evidence can be captured computationally through comparison.echoes through the field. Distributional semantics, famously summarized by Firth's dictum "you shall know a word by the company it keeps," formalizes and extends the Lesk intuition. Word embeddings like Word2Vec and GloVe learn representations where semantically similar words cluster together, essentially learning to recognize the kind of contextual overlap that Lesk computed explicitly. These embeddings enable semantic comparison without hand-crafted definitions, but they're doing computationally what Lesk did manually: using distributional evidence to disambiguate meaning.

The algorithm also foreshadowed the shift from rule-based to data-driven NLP. While Lesk himself still relied on a hand-crafted resource (the dictionary), his method avoided writing explicit disambiguation rules. The algorithm extracted patterns from data.the data being dictionary definitions and the text itself. This represented an intermediate step between classical rule-based AI and modern statistical learning. Researchers in the 1990s and 2000s built directly on this foundation, asking: if we can use the statistical overlap between definitions and contexts, why not use statistical patterns learned from large corpora? This question led to supervised learning approaches that trained classifiers on sense-annotated examples, learning disambiguation patterns directly from data rather than inferring them from definitions.

Modern transformer models like BERT and GPT can be understood as learning contextual representations that implicitly solve word sense disambiguation. When BERT generates a context-dependent embedding for "bank" in "I sat by the bank fishing," that embedding differs from the embedding of "bank" in "The bank approved my loan," capturing the sense distinction. The model hasn't explicitly chosen between dictionary senses; instead, it's learned representations where different contexts of the same word type push its embedding toward different regions of the semantic space.regions that align with human sense distinctions. This is Lesk's principle scaled to massive data and parameterized through neural networks: context shapes meaning, and that shaping can be captured computationally.