Morphological Analysis and Generation › Limitations and Impact

Finite-state morphology's deepest limitation is not mathematical but human: the lexicon bottleneck. Irregular forms, exceptional stems, continuation classes, and morphophonological alternations must be represented and tested. The engineering effort grows with lexical breadth and alternation complexity, whether the language has a few broad inflectional classes with many irregular lexemes or richer systems of noun classes and stem alternations. Coverage gaps are silent failures: an analyzer may return no analysis for an out-of-lexicon word, and it cannot by itself decide whether the word is rare, new, a typo, or a proper noun. Weighting ranks existing paths but does not create missing ones. Two-level constraints avoid sequential rule ordering, but complex constraint interactions, lexical exceptions, and incomplete test coverage can still produce subtle bugs. Formal correctness remains conditional on the encoded grammar.

Neural morphology trades that brittleness for different problems. It needs parallel data, which is scarce for many morphologically rich languages. It can generate forms that violate phonotactic or inflectional patterns, particularly for rare feature combinations, and a plain seq2seq architecture does not distinguish licensed from unlicensed outputs. Exact-match accuracy also hides the distribution of errors: a score of 90 percent does not identify which constructions make up the remaining 10 percent. When reading inflection benchmarks, inspect the split design, frequency distribution, per-feature results, and irregular or otherwise difficult subsets rather than relying on one aggregate number.

The impact of this chapter's machinery is broad and often invisible. Spell checkers and hyphenators use several architectures; so do text-to-speech systems, search normalization, and machine-translation pipelines. These systems may be finite-state, statistical, neural, or hybrid, with morphological analysis as one component. For neural language models, morphology is entangled with the tokenization you studied in Part V: a vocabulary poorly matched to a morphologically rich language can split words into more tokens, increasing sequence length and changing the evidence available for learning inflectional patterns. We will revisit that problem for multilingual models in Part XXVI.