Lexical Semantics and Word Senses › Approaches to Word-Sense Disambiguation

WSD methods fall into three broad families: knowledge-based, supervised, and contextual neural methods. The boundaries are not rigid. A modern system may combine a lexical knowledge base with a pretrained encoder and a small supervised classifier. The families differ mainly in where the evidence comes from. Knowledge-based methods use dictionaries and lexical relations. Supervised methods use sense-tagged training data. Contextual neural methods use pretrained representations that encode context. Each family has strengths and weaknesses, and the best choice depends on the inventory, the domain, the amount of labeled data, and the evaluation goal.

Knowledge-based WSD uses lexical-resource definitions or relations without requiring a task-specific sense-tagged training corpus. For a gloss method, the resource must supply usable candidate definitions or examples in a language the method can compare with the context. For a graph method, relevant relations and coverage are also necessary. Curated lexical resources already embody human semantic decisions; absence of a sense-tagged training corpus does not mean absence of human annotation. A resource's availability alone does not guarantee transfer to an arbitrary language or domain.

The Lesk AlgorithmLink Copied

Michael Lesk's 1986 paper introduced a dictionary-based approach to sense disambiguation. Lesk-style methods use lexical overlap as evidence. Here we study a simplified context-to-gloss variant: compare a candidate's definition with the words surrounding the target and choose the highest score. This implementation should not be read as a reproduction of every step in the original algorithm. Gloss-to-gloss comparisons and related-synset extensions are other variants; the exact text compared must be specified.

The simplified version is easy to describe. The scoring heuristic favors candidates whose signatures share vocabulary with the context. Whether that ranking selects the intended sense depends on the available words and candidate definitions. For a target word (w) in context (c), and a candidate sense (s) with gloss (g_s), compute:

$$\text{score}(s, c) = \sum_{t \in c} \mathbb{1}[t \in g_s],$$

where:

• (\text{score}(s, c)): the overlap score for candidate sense (s) given context (c)
• (c): the set of distinct preprocessed content words in the context window
• (t): one distinct content word in that set
• (g_s): the set of content words in the gloss (definition) of sense (s)
• (\mathbb{1}[\cdot]): the indicator function, equal to 1 when its argument is true and 0 otherwise

In words, count how many context words appear in the gloss. Choose the sense with the highest count. The formula is simple, but it hides several design choices. Which words count as content words? How large is the context window? Are words stemmed or lemmatized? Are stopwords removed? Are multiword expressions treated as units? Each choice can affect the score. With preprocessing and signatures fixed, enlarging the context word set cannot lower any candidate's raw intersection count. It can add matches for incorrect candidates as well as the intended one. Stemming can reconcile inflections and can collapse distinct words. A stopword policy also removes potential matches, so check which cues the task needs. The Lesk algorithm is therefore not a single algorithm. It is a family of algorithms defined by these preprocessing choices.

A version with a larger lexical signature uses examples as well as the definition:

$$\text{score}(s, c) = |c \cap (g_s \cup e_s)|,$$

where:

• (c): the set of content words in the context window
• (g_s): the set of content words in the gloss of sense (s)
• (e_s): the set of content words in the example sentences for sense (s)
• (|\cdot|): the cardinality (number of elements) of a set

You can also include the glosses of related senses, which gives extended Lesk. The extended signature can supply words missing from the target sense's own short gloss. If the gloss for a sense is "a financial institution," the words are few. Adding the glosses of hypernyms or related concepts can supply additional vocabulary and matches. The tradeoff is that related senses may introduce vocabulary that is not specific to the target sense. A larger signature can create extra overlaps for correct and incorrect candidates. Whether those changes improve WSD precision or recall must be measured on a stated task.

Raw overlap counts do not include a penalty for signature length. If one signature contains every type in another and adds more types, its intersection count cannot decrease; that does not guarantee a better sense ranking. Jaccard normalization divides the intersection size by the union size, so adding unmatched types can lower its score:

$$\text{score}(s, c) = \frac{|c \cap g_s|}{|c \cup g_s|},$$

where:

• (c \cap g_s): the set of content words shared by the context and the gloss
• (c \cup g_s): the set of all content words appearing in either the context or the gloss

This is the Jaccard similarity when the union is nonempty. If preprocessing leaves both sets empty, the displayed fraction is undefined; an implementation must state a fallback, such as returning zero evidence. Another option is cosine similarity over TF-IDF vectors, which we implement later. For two nonzero vectors, cosine is their dot product divided by the product of their lengths: it compares direction after length normalization. A zero vector has no direction; the scikit-learn function used below returns zero similarities for a zero-vector row. TF-IDF supplies the term weighting, while cosine supplies the normalized comparison. Their effect on WSD must be measured for the chosen glosses and preprocessing.

For a short illustration, hold the context size fixed and let three candidate glosses grow in length while their intersections grow more slowly. Raw overlap rewards the longest gloss, but the Jaccard score reverses that advantage.

The implemented overlap scorer requires no sense-tagged training corpus. It does require candidate definitions or examples, compatible context/resource text, and preprocessing suited to that text. The code below uses an English resource and ASCII alphabetic token extraction, so transferring it requires checking those assumptions. Its diagnostic output makes the shared words inspectable. A per-candidate classifier could also take the numeric overlap score as one input feature alongside other representations; whether that hybrid improves predictions requires evaluation.

It also has clear weaknesses. Exact word overlap can be sparse. Inflection can hide a match: deposited and deposits are different strings. Synonyms such as car and automobile do not match as strings. A short gloss can omit cues to the intended use. The method is sensitive to window size and stopword handling. It has no way to learn which context words are actually diagnostic. A word like the may overlap with many glosses but provide no evidence. A rare word may be highly diagnostic but appear in only one gloss. Lesk treats all overlaps as equal unless weighting is added. This is why more sophisticated knowledge-based methods use graph structure, selectional preferences, or learned weights.

Graph-Based MethodsLink Copied

Graph-based WSD represents senses and semantic relations as a graph. Candidate senses are nodes; resource relations or other declared associations provide edges. A method can use connections among the candidate senses of different context words as a coherence heuristic. It then ranks candidates with a stated rule, such as a PageRank-like score. The connections are evidence used by the model, rather than a test that identifies every correct sense.

Graph-based methods use relation evidence beyond exact string overlap. Candidate construction, selected relations, and the ranking rule determine what that evidence can contribute. Missing links can remove useful connections, while unrelated links can add competing associations. Inspect those edges and the resulting rankings when diagnosing errors. A performance comparison with Lesk needs a stated inventory, corpus, and preprocessing policy.

Selectional PreferencesLink Copied

Selectional preferences capture the idea that predicates prefer certain kinds of arguments. Eat prefers food objects. In its written-language sense, read prefers text objects. In its theft sense, rob can take the person or place robbed as its object; the stolen property need not be money. If a verb strongly selects for a particular semantic class, that preference can disambiguate its arguments. For example, "The bank was robbed" favors a banking-related reading over the riverbank reading. The cue alone leaves both the institution and its premises as possible readings. "The bank was muddy" strongly suggests the riverbank sense because financial institutions are not usually muddy. Selectional preferences can be learned from corpus data (McCarthy, 2009). They are a useful source of evidence, but they are not sufficient on their own because many predicates are flexible and many nouns are polysemous. The visual and cognitive senses of see allow different kinds of objects, so the sense and construction matter. A noun like bank can be the object of many verbs. Selectional preferences provide soft constraints, not deterministic rules. Selectional preferences can complement local context, lexical relations, and other evidence; evaluate the combination on the intended task.

Supervised WSD treats sense selection as a classification problem. The training examples pair an annotated word in context with its sense label. The system learns a classifier that maps a context representation to a sense label. Supervised WSD can be very effective when enough labeled data is available. It can learn subtle cues that are not present in glosses. It can learn domain-specific patterns and collocations. The classifier can use word-form and syntactic features, or representations learned by a neural encoder, when the relevant cues are present in its training data. The main limitation is the cost of annotation and the sparsity of labeled data for rare senses.

Classical supervised systems have used local words, POS, and document-topical context (McCarthy, 2009). For a particular classifier, select and evaluate input features such as:

• Local context: words within a window around the target, represented as unordered counts or as position-sensitive lexical features.
• Part of speech: the POS of the target and its neighbors.
• Collocations: frequent word pairs or triples that co-occur with a sense.
• Syntax: dependency relations, subject/object positions, and phrase structure.
• Semantic information: selectional-preference scores or domain indicators when available.
• Discourse: the "one sense per discourse" heuristic, which says a word tends to keep the same sense throughout a coherent text.
• Topic: document-level lexical or topical information, such as a bag of words from the wider text.

Classifiers include Naive Bayes, decision trees, support vector machines, logistic regression, and conditional random fields. The 2017 unified evaluation compared conventional local-feature classifiers alongside neural approaches, so this was not a complete replacement of manual features. Both formulations predict a sense label from a context representation. The features are still useful for understanding what evidence matters. Local context captures collocations. Syntax captures argument structure. Semantic information captures selectional preferences. Discourse captures consistency. Topic captures domain effects. A neural model may learn these patterns implicitly, but it still depends on the data containing the relevant signals.

A classifier needs a policy for labels with sparse or absent training examples. Frequency fallback, gloss scoring, and shared representations are alternatives. Whether an all-words test contains those cases depends on its documents, inventory, target exclusions, and training split. Report the relevant training frequencies and evaluate the selected policy on the same target set.

The "one sense per discourse" heuristic is a useful constraint. If a word is used in one sense in a document, it is likely to keep that sense throughout. A model can add this as a preference while allowing multiple senses when the context warrants them. The related "one sense per collocation" heuristic says that a strong collocation tends to select a consistent sense. These heuristics reflect regularities in language use. They are not laws, but they provide useful priors. A supervised system can incorporate them as features or as post-processing constraints.

Performance numbers belong to a specified experiment. In Blevins and Zettlemoyer (2020), the gloss-informed bi-encoder reports 79.0% F1 on the paper's aggregate ALL English all-words set, compared with 65.5% for its training-data MFS baseline. ALL concatenates the listed datasets and includes the SE07 development set, so it is not a single wholly held-out test score. This historical result illustrates the reporting details needed for comparison; it is not a current performance range for all WSD systems.

Static word embeddings assign one vector per word type. Looking up bank returns the same vector in the paycheck sentence and the river sentence. The lookup alone therefore cannot distinguish those occurrences. As discussed in Part IV, the type vector represents distributional information; it is not a specified mathematical mixture of separate sense vectors. A WSD system can still use static embeddings together with surrounding-word features, or learn separate sense representations. The limitation belongs to the context-independent lookup, not to every system that uses it as an input.

ELMo and BERT construct token representations using context, so the two bank sentences can receive different target vectors. ELMo's analysis also found information about syntax and word senses in different internal layers. Such features are not dedicated inventory sense vectors: a WSD system still needs a classifier, prototype comparison, gloss score, or another decision rule. The amount of sense-tagged supervision required depends on that method and its evaluation, rather than following from contextuality alone.

There are several ways to turn contextual vectors into sense decisions:

1. Supervised sense classification: Train a classifier on top of contextual vectors to predict a sense label. The prediction head can be a linear layer or a small MLP. The encoder may also be fine-tuned for WSD.
2. Sense embeddings: Represent each sense by a vector, either learned from sense-tagged data or derived from glosses. At test time, compare the contextual vector to the sense vectors and choose the nearest.
3. Contextual sense prototypes: Represent each sense by the average contextual vector of its training examples. At test time, compare the target token's vector to each sense centroid. This nearest-centroid procedure differs from instance-level nearest neighbors, which retain individual training-example vectors.
4. Gloss-based sentence pair classification: Treat the context and the sense gloss as a sentence pair and train a model to predict whether the gloss fits the context. GlossBERT is a well-known example of this approach.
5. End-to-end sense labeling: Treat WSD as a sequence labeling problem, where each eligible target word receives a sense label constrained to its candidate inventory. Check training coverage for the candidate labels and say how the model scores senses with no labeled training examples.

Layer choice is an experimental decision. In the ELMo study, higher LSTM states captured sense-related information while lower states captured more syntactic information; the model combined internal layers for downstream tasks. This does not imply that a particular middle layer is best for every encoder or WSD dataset. Compare layers or layer combinations on development data and keep the test set separate from that choice.

Word-in-Context predicts whether two usages share a meaning rather than returning a particular sense identifier. A binary comparison can leave the label identity undetermined. For a pair task defined by label equality, two correct labels settle the comparison when each occurrence has one adjudicated label in the same aligned inventory. Different inventories or granularities can break a direct comparison, so WiC and inventory-label WSD should be reported with their respective definitions and protocols.

Large language models offer a new interface to WSD. Instead of training a classifier, you can prompt the model with the target sentence and a list of sense definitions, and ask it to choose the best sense. A typical prompt might look like this:

Target word: bank Sentence: She deposited her paycheck at the bank before noon. Candidate senses:

1. sloping land beside a body of water
2. a financial institution that accepts deposits
3. a flight maneuver in which an aircraft tips laterally

Which sense is intended? Answer with the number.

This approach is flexible and requires no task-specific training. A prompt can request a sense identifier, candidate ranking, or explanation; the chosen output still needs validation. It also has serious limitations. The flexibility is attractive because it allows rapid prototyping and adaptation to new inventories. A prompt can include definitions, examples, and instructions. A model can be asked to choose a label, rank candidates, or explain its reasoning. This makes prompt-based WSD a useful tool for exploration. However, the same flexibility makes evaluation difficult. The prompt is part of the model's input, and small changes to the prompt can change the output. The model's behavior is not fixed by training on a specific inventory. It is conditioned by the prompt and the pretraining data.

First, LLMs may not adhere to the sense inventory. They may invent a sense, merge two senses, or choose a definition that is plausible but not in the inventory. Multiple-choice formatting can present the candidates as options. Constrained decoding can enforce valid identifiers, and a validator can reject answers outside the inventory. Even with constraints, the model may choose a label for the wrong reason. It may rely on superficial cues in the prompt, or it may be influenced by the order of the candidates. The output may be valid but not reliable.

Second, prompted WSD predictions can change with option order or wording. Swapping the order of definitions can change the answer. Changing the candidate set also changes the decision problem, so its size and contents belong in the evaluation protocol. Option-order effects have been demonstrated in particular model and dataset studies, including RoDEval (2025); their magnitude should be measured for the system being tested. If the answer depends on the order of the options, the evaluation is measuring prompt sensitivity as well as semantic ability. A robust evaluation should randomize order, vary wording, and report the variance across prompts. It should also compare against a baseline that uses the same prompt format. A single prompt result is not enough to establish robust WSD performance.

Third, LLMs may have seen the benchmark data during pretraining. WiC and SemEval examples are public and may be memorized. A high public-benchmark score alone does not establish robust WSD, especially when contamination has not been assessed. One approach is to use newly constructed test sets that are not publicly available. Another approach is to filter the pretraining data, if that is possible. Another approach is to test on examples that are paraphrased or perturbed in ways that preserve the sense but change the surface form. A drop under paraphrase is a robustness finding. It is not by itself proof of memorization: paraphrasing can change difficulty, pragmatic cues, or the intended sense unless these are checked.

Fourth, distinguish label selection from the quality of generated explanations. A valid label can still be wrong, and a fluent explanation can still misstate a sense. Empirical capability depends on the model and protocol: Meconi et al. (2025) report that leading evaluated LLMs matched specialized WSD systems, whereas RoDEval (2025) found option biases and overconfidence in its tested settings. Neither result establishes a universal coarse-versus-fine ranking. Measure the distinctions needed by the application, and use expert error analysis when fixed-label scores leave important questions unresolved.

The practical lesson is that LLMs are not automatically WSD systems. The cited studies demonstrate strong WSD performance by some of the tested models. An LLM-based WSD application needs an inventory interface, an output policy, and an evaluation protocol that assesses contamination and prompt sensitivity. A prompt-based system can be useful, but it should be treated as a system with its own design choices. The prompt is a kind of interface specification. Include the definitions, demonstration examples, and requested output format when reporting the prompt. The evaluation should therefore report the prompt, the inventory, the decoding constraints, and the baseline.

Multilingual WSD adds challenges in sense-inventory coverage and translation alignment; cross-lingual evidence can also help disambiguation. A word may have different senses in different languages, and the translations may not align. The same concept may be lexicalized differently, and a single translated word may cover multiple source senses. For example, English bank translates to French banque in the financial sense and rive in the river sense. French banque can also translate bank in the storage sense, as in data bank. And river bank can be translated as rive or berge. One target expression can cover several source senses, and one source sense can have several target expressions. This makes multilingual WSD both a challenge and an opportunity. The challenge is that sense inventories and translations do not align cleanly. The opportunity is that translations provide additional evidence about sense distinctions.

Parallel corpora can supply evidence for translation-based WSD. A reliably aligned translation can help distinguish source-word senses when the target language expresses them differently. For example, English bank translates to French banque in the financial sense and rive in the river sense. A parallel corpus can then provide indirect supervision without explicit sense annotation. Word alignment can fail, translations can be idiomatic, and a single translation may cover multiple senses. Validate automatically projected labels for their sense mappings and quality on the task before using them for training or evaluation.

Cross-lingual projection transfers sense annotations from a resource-rich language to a resource-poor one. The projection is noisy because word alignment is imperfect and senses do not map one-to-one. Multilingual encoders such as mBERT and XLM-R provide contextual representations across languages and have been used for zero-shot WSD. For an illustrative one-shot classifier with a finite, nonempty candidate set, take one labeled support context per sense. Encode the marked target in each support context and in a new occurrence with the same encoder and vector extraction rule. Use finite vectors of equal dimension, Euclidean distance, and a fixed rule for ties; return the sense of the nearest support vector. This construction supplies a prediction, not a measured accuracy or cross-language transfer result. BabelNet provides multilingual sense inventories and translations. These resources make multilingual WSD possible, but they do not make it easy. The XL-WSD experiments demonstrate variation across evaluated languages in a general-domain setting. A good result for one language pair does not establish performance on another, and a general-domain benchmark does not establish performance on clinical text. Domain transfer needs its own evaluation.

Translation can also supply disambiguation evidence. Kang, Blevins, and Zettlemoyer (2024) prompt a model to translate the target word in context, then use the overlap between source-word and translated-word sense sets to form WSD predictions. They evaluate the approach on 18 languages from XL-WSD. The translation language and sense mapping are part of this procedure; report them and examine alignment or mapping errors when interpreting its results.