Morphological Analysis and Generation

In the previous chapter, you learned how languages build words from morphemes, whether by changing grammatical features or creating new words. This chapter turns that linguistic knowledge into computational machinery. We want systems that can do two things with morphologically complex words. Analysis takes a surface form like the Turkish word evlerinizden ("from your houses") and recovers its internal structure: stem ev, plural, possessive second person plural, ablative case. Generation runs the machine in reverse: given the stem ev and a bundle of grammatical features, produce the correct surface form.

This matters because a huge fraction of the world's languages pack what English does with separate words into single word forms. In Finnish, one inflected noun can mark number and case while also carrying possessive or clitic information that would take several English words to express. Systems that treat every word as an opaque atom, as the bag-of-words models from Part II did, drown in sparsity: thousands of distinct surface forms share a single stem, and a statistical model sees each of those forms too rarely to learn anything reliable about it. Solving morphology means recovering the shared structure underneath, so that evidence about one surface form transfers to the thousands of forms built on the same stem.

We will study this machinery through two dominant paradigms. First, finite-state morphology: hand-engineered transducers that encode lexicons and morphophonological rules as inspectable regular relations. Then neural morphology: character-level encoder-decoder models that learn inflection from examples. The code section implements compact rule-based demonstrations of the finite-state ideas and a small neural model that learns Turkish-style vowel harmony from synthetic data, then tests it on stems excluded from training. This lets us compare their assumptions and failure modes without pretending that one small experiment decides which paradigm wins in general.

Before diving into algorithms, let's be precise about the tasks, because "morphology" in NLP bundles several distinct problems that are easy to conflate.

Analysis (surface to analysis). Given a surface word form, recover one or more candidate analyses. An analysis typically pairs a lemma with a bundle of morphosyntactic features, written here in a Universal Dependencies-style FEATS notation:

• evlerinizden $\to$ (ev, Number=Plur, Person[psor]=2, Number[psor]=Plur, Case=Abl)
• flies $\to$ (fly, Number=Plur) or (fly, Person=3, Tense=Pres, VerbForm=Fin)
• watched $\to$ (watch, Tense=Past, VerbForm=Fin)

Analysis can be ambiguous. The same string may decompose in more than one way, and an analyzer can enumerate those possibilities before a later component selects one. Choosing among them often requires context, which connects directly to the sequence labeling machinery you built in Part VI: hidden Markov models, Viterbi decoding, and conditional random fields. A part-of-speech tagger can help disambiguate whether flies in a particular sentence is the plural noun or the inflected verb.

Generation (analysis to surface). Given a lemma plus features, produce one or more licensed surface strings. This is the direction spell checkers, text-to-speech front ends, and machine translation systems need when they must produce a correctly inflected word in the target language. It is also the direction language models implicitly perform every time they generate an inflected form. Generation is usually less ambiguous than analysis, though a feature bundle can license variants. It is where systematic linguistic knowledge pays off most visibly: get the harmony rule wrong and every affected suffixed form is wrong.

Lemmatization is a frequently used special case of analysis: map a surface form to its dictionary form (its lemma), discarding inflectional detail. It is closely related to the stemming you saw in earlier chapters, but the goals differ in ways that matter for downstream systems.

Why not sidestep all of this with the subword tokenizers from Part V? To a first approximation, they do solve a morphology-shaped problem by fragmenting rare words into reusable pieces. BPE greedily merges frequent symbol pairs, while SentencePiece is a toolkit that supports BPE and unigram language-model tokenization. Neither objective requires the resulting units to be linguistically correct morphemes. A BPE vocabulary trained on English may capture the frequent suffix -ing while slicing cancellation into fragments that do not follow its morphological structure. For some language-tokenizer combinations, morphologically rich words split into substantially more tokens, which stretches context windows and changes how much evidence fits into a fixed-length input. We will return to this tension in the limitations section and when we discuss multilingual models in Part XXVI: Multilingual Language Models and Cross-Lingual Transfer.

Finite-state morphology is the classical answer, developed in the 1980s and 1990s by Kimmo Koskenniemi, Lauri Karttunen, and others. It remains an important approach in morphological analyzers, spell checkers, and hyphenation systems. Its core claim is elegant: many practically useful accounts of word structure can be expressed with finite-state automata extended to emit symbols in two alphabets at once.

Building on your knowledge of regular expressions from Part I, Chapter 3, recall that a finite-state automaton recognizes a regular language: it reads a string and answers yes or no. A finite-state transducer (FST) generalizes this from a recognizer to a translator: it reads a string and produces another string.

The power of this formalism for morphology comes from its closure properties. Regular relations remain regular under composition and union, and they can also be inverted: the inverse of an analyzer relation can serve as its generator relation. Because analysis and generation use the same encoded relation, they cannot drift apart as two separately maintained implementations might. The guarantee is relative to the grammar: the machine is consistent with what its lexicon and constraints encode, not automatically with every fact of the natural language. A lexicon transducer can also feed a cascade of rewrite-rule transducers. If $L$ is the lexicon transducer and $R_1, \ldots, R_k$ are sequential rule transducers, the whole relation is their composition $R_k \circ \cdots \circ R_1 \circ L$.

where:

• $L$: the lexicon transducer, which maps lexical strings (stems plus morpheme labels) to intermediate strings
• $R_1, \ldots, R_k$: the phonological rule transducers, applied in order, with $R_1$ first
• $R_k \circ \cdots \circ R_1 \circ L$: the composed transducer, which maps a lexical string directly to its surface form

Composition is what lets the pipeline behave like one relation: each transducer's output is paired with the next transducer's input, and the composite relation is still regular, so it can be inverted. Practically, this means you can develop the lexicon and each rule as separate, testable artifacts, then compile them into one transducer for efficient lookup.

Koskenniemi's two-level morphology, the foundational design, models a word as a pairing between a lexical string (morphemes with their identity intact) and a surface string (what is written). Between the two levels sit correspondence rules that map pairs of symbols in parallel: rather than rewriting a string sequentially, rule by rule, two-level rules constrain, at each position, which lexical symbol may correspond to which surface symbol.

For the English noun plural, the lexicon would generate the lexical string watch +N +PL, conceptually w a t c h ^ s with a morpheme boundary, and the constraints must license the surface form watches. In two-level notation, one writes rules over symbol pairs, where each symbol in the rule is an upper:lower pair. A schematic sibilant-epenthesis constraint looks like this:

where:

• $\epsilon{:}e$: the correspondence pair, with nothing in the lexical string and a surface e on the other side
• $\textit{Sibilant}$: a symbol class matching one sibilant consonant; in a complete grammar, the environment also identifies the plural morpheme boundary
• $\_$: the position of the $\epsilon{:}e$ correspondence itself
• $s{:}s$: the identity correspondence for the plural s, which passes through unchanged
• $\Leftrightarrow$: the biconditional operator

Read this as: "the pair $\epsilon{:}e$ (nothing in the lexical string, an e on the surface) corresponds exactly in the environment where a sibilant precedes the plural s." The biconditional operator $\Leftrightarrow$ means the rule fires exactly in this environment and nowhere else. Different operators encode different logical strengths, and choosing the right one for each rule is part of the grammarian's job:

• $\to$ surface constraint: the correspondence is permitted only in this environment and is blocked everywhere else (this rules out illicit occurrences).
• $\Leftarrow$ obligatory: whenever the left side occurs in this environment, it must be rewritten.
• $\Leftrightarrow$ if and only if: both of the above.

Two-level constraints are evaluated in parallel over the same lexical–surface correspondence; they do not feed or bleed one another through an ordering. For party $\to$ parties, a complete grammar would constrain the lexical y to correspond to surface i in the plural environment and would license surface e before the plural s. Both constraints describe the same aligned pair rather than successively rewriting party+s. That distinguishes the method from an ordered rewrite cascade.

Sequential rewrite transducers are also useful, and they compose. Because an FST denotes a relation rather than necessarily a function, composition is set-valued:

where:

• $x$: an input string, for example a lexical string produced by the lexicon
• $R_1$: the first rule applied, for example the y-to-i rule
• $R_2$: the second rule applied, for example the epenthesis rule
• $y$: one possible intermediate string licensed by $R_1$
• $R_2(y)$: the set of outputs licensed by $R_2$ for that intermediate string

In other words, the composite transducer retains every path whose intermediate output from $R_1$ matches an input to $R_2$. It produces one combined relation, but that relation need not be deterministic. Building the analyzer means compiling the lexicon (which morphemes exist and in what order, a constraint called morphotactics) and the alternation constraints (how morphemes change in contact, called morphophonology) into a transducer that can be used in both directions.

Why is it legitimate to model many sound alternations with finite-state machines? Contexts like "preceded by a sibilant" or "followed by a suffix starting with a front vowel" are the kinds of finite-context patterns regular expressions capture. Kaplan and Kay's classic 1994 treatment showed that the standard, non-self-applying interpretation of finite-context rewrite rules denotes regular relations; FST closure under composition then preserves regularity for a finite cascade. The restrictions matter: arbitrary cyclic or self-feeding rule application is not covered by the claim.

This gives finite-state morphology a precise, grammar-relative guarantee. An analyzer returns exactly the pairs licensed by its compiled lexicon and constraints. It can still accept a non-word if the grammar overgenerates or miss a real word if the lexicon or rules are incomplete. Neural models offer a different tradeoff: their architecture does not hard-code the same closed set of licensed pairs, so they can produce forms outside the observed language patterns.

Let's walk through a small but real example: the Finnish word taloissa, meaning "in the houses." Finnish nouns inflect for number and case, and the case suffixes obey vowel harmony, so a single noun stem generates dozens of forms whose suffix vowels depend on the stem.

The lexicon says a noun stem like talo ("house") inflects by concatenating a number marker and a case marker. The lexical string contains the stem plus abstract markers, some of which (like the plural i and the archiphoneme A inside the case suffix) are later rewritten by the rules:

Notice that the plural has an i allomorph (Finnish uses i or j before most cases) and that the case suffix ssa contains the back vowel a, because the last vowel of the stem is o, a back vowel. A front-vowel stem selects ssä. The form käsissä from käsi additionally involves plural-stem alternation and vowel deletion; harmony alone selects the suffix vowel but does not derive the complete form. In two-level notation, harmony can be expressed with an archiphoneme, a symbol in the lexical alphabet that has not yet decided which vowel it will become:

• $A$: the archiphoneme in the lexical string, the underspecified vowel of the case suffix
• $A{:}a$ / $A{:}\ddot{a}$: the correspondence pair, mapping the underspecified $A$ to the back surface vowel $a$ or the front surface vowel $\ddot{a}$
• $\textit{BackVowel}$ / $\textit{FrontVowel}$: a symbol class matching one harmony-triggering back vowel ($a$, $o$, $u$) or front vowel ($\ddot{a}$, $\ddot{o}$, $y$)
• $\textit{Neutral}^{*}$: zero or more harmony-neutral vowels ($i$ or $e$), such as the plural allomorph $i$, which neither trigger harmony nor block it
• $C_0$: zero or more intervening consonants between the triggering vowel (after any neutral vowels) and the archiphoneme
• $\_$: the position of the $A{:}a$ or $A{:}\ddot{a}$ correspondence itself
• $\Leftrightarrow$: the biconditional, meaning the pair is rewritten exactly in this environment and nowhere else

The lexical string talo i ssA (the A being the archiphoneme inside the case suffix) passes through the harmony rule, which skips the harmony-neutral plural i and commits the A to a because the nearest preceding stem vowel is back. The composed transducer then aligns the two strings character by character:

Lexical:  t a l o   i   s s A
Surface:  t a l o   i   s s a     →  taloissa

where:

• row 1 (Lexical): the lexical string talo i ssA, with the archiphoneme A standing in for the not-yet-harmonized first vowel of the inessive suffix
• row 2 (Surface): the surface string taloissa, after the harmony rule commits A to a
• characters at the same position are aligned in columns, showing that each lexical symbol corresponds to exactly one surface symbol (the +PL and +INE labels are grammatical features rather than surface characters)

The harmony constraint licenses A:a because the preceding harmony-triggering stem vowel is o, a back vowel. For a front-vowel stem it licenses A:ä; separate alternation constraints account for the stem changes in forms such as käsissä.

Running the same transducer in the analysis direction, the input taloissa is scanned left to right: talo matches a noun stem, i matches the plural allomorph, ssa matches the inessive allomorph, and the end of input is reached in a final state. The machine emits talo +N +PL +INE. If we fed it talon, it would emit talo +N +SG +GEN ("of the house"); with no suffix, talo yields talo +N +SG +NOM. Ambiguity arises when multiple lexical paths accept the same surface string. Resolving among such analyses requires sentence context, which is exactly the role Viterbi-style decoding over a sequence tagger plays, as you saw in Part VI, Chapter 6.

A plain FST answers a yes/no question per string pair: is this analysis licensed? Real systems want the most plausible analysis, because a raw enumeration of licensed analyses is often long and mostly useless without a ranking. Weighted FSTs attach a weight to each arc, typically a negative log probability:

where:

• $\text{path}$: one accepting route through the transducer, from the start state to a final state, consuming the input string and emitting a candidate analysis
• $\text{arc}$: a single weighted transition on that path
• $p(\text{arc})$: the probability associated with taking that arc, so that $-\log p(\text{arc})$ is its cost
• $\sum_{\text{arc} \in \text{path}}$: the sum of the arc costs over the whole path

When a path probability is factorized into arc probabilities, which may be conditional on the state and history encoded by the machine, multiplication becomes addition under the negative logarithm. An arc with probability zero has infinite cost. The best analysis of a surface form is then the minimum-weight accepting path, found by the same dynamic programming idea as Viterbi decoding from Part VI, Chapter 6, just over a transducer instead of an HMM. Weights can come from a lexicon of word frequencies, a corpus, or a character-level language model, and they let the analyzer rank flie+s below fly+s without hard-coding a rejection.

To make the ranking concrete, we first attach toy corpus frequencies to the candidate analyses of flies and convert them to path costs.

Because the correct reading carries the bulk of the corpus mass, its cost is far below the spurious analysis, which is exactly the separation a weighted FST exploits. Note that the spurious path remains available at finite cost: weighting softens the all-or-nothing character of the unweighted machine while preserving the complete enumeration of licensed analyses.

In one downstream design, a disambiguating sequence tagger consumes these costs alongside sentence context and selects among the candidate analyses. Other pipelines expose candidates without their original FST weights or combine scores differently. Either way, keeping a candidate at finite cost leaves it available when context favors it.

Several mature toolchains implement this pipeline. Xerox tools (lexc for lexicons, twolc for two-level rules) are the historical standard. HFST and foma are widely used open-source implementations, and pynini brings weighted FSTs into Python. GiellaLT, developed by the Giellatekno community, supports language-technology resources for Sámi and other languages. UniMorph provides a cross-lingual schema and inflection datasets, while UniDive coordinates work on universal and cross-lingual morphology; they are related resources, not FST analyzer toolchains. A high-quality analyzer for a new language can require substantial expert work: enumerating the lexicon, writing morphotactic continuation classes, and testing morphophonological constraints. That cost helped motivate data-driven approaches, which we examine next.

The neural reframing treats inflection as string-to-string transduction learned from data. Given a training set of triples (lemma, morphosyntactic features, target form), a character-level encoder-decoder model learns the mapping end to end, with no hand-written rules. The landmark evaluations here are the SIGMORPHON shared tasks (2016 onward) and the CoNLL-SIGMORPHON 2017–2018 shared tasks, which established character-level encoder-decoder networks as the standard for what is often called morphological reinflection: given watch and the features V;PST, produce watched.

Why characters rather than words? Because the whole point is to generalize to unseen stems. A character-level model can inflect a stem it has never encountered, by combining what it has learned about the -ed pattern with the phonology of the stem it reads, so screeched is reachable even if screech never appeared in training. This is the same subword insight that powered FastText's character n-gram embeddings in Part IV, Chapter 8, applied to a transduction task instead of a representation task.

A minimal architecture looks like this:

• an encoder: an embedding layer over the character vocabulary feeding a GRU or LSTM, which compresses the input string (say, the lemma plus a tag string like N;PL) into a sequence of hidden states;
• a decoder: a second GRU that generates the output string one character at a time, attending over encoder states;
• an optional alignment or copy mechanism: because the output usually shares most of its characters with the input, models can benefit from tracking input positions or explicitly copying characters. Aharoni and Goldberg (2017) used hard monotonic alignment with action-based character decoding rather than an explicit pointer-copy operation. Makarov and Clematide (2018) combined a related transition-based approach with imitation learning in a strong SIGMORPHON system.

The learned model can capture morphophonological patterns implicitly, without hand-written rules. Given suitable English training data, a character model can learn mappings such as walk $\to$ walked, carry $\to$ carried, and stop $\to$ stopped. Some experimental models overregularize irregular forms under particular data and evaluation conditions, but that behavior is not guaranteed and our Turkish-style experiment below does not test English irregulars.

There is a second neural family worth distinguishing. Instead of one word at a time, contextual lemmatizers and morphological taggers operate on whole sentences. Their architectures vary: contextual encoders can predict part-of-speech and morphological features, while lemma components may use lookup tables, edit rules, sequence models, or other language-specific mechanisms. Context helps distinguish flies the noun from flies the verb even though both share the lemma fly. Stanza and spaCy expose these analyses through a common pipeline interface, but their internal lemmatization architectures are not identical. We will meet transformer encoders in Part XXIV.

Finally, there is unsupervised morphological segmentation: given only raw text, look for recurring pieces shared by forms such as walks, walking, and walked. Morfessor formulates this as a compression problem under the Minimum Description Length principle, balancing the cost of a lexicon of morphs against the cost of encoding the corpus with them. The objective is designed for morphological segmentation, but it does not guarantee that every learned segment is a linguist's morpheme. BPE from Part V, Chapter 2 is a cousin with a different objective: it merges frequent symbol pairs and may cross morpheme boundaries.

Let's make the contrast concrete with a single regular phenomenon. Turkish suffixes harmonize with the last vowel of the stem: after a front vowel (e, i, ö, ü) the plural is -ler, and after a back vowel (a, ı, o, u) it is -lar. The harmony rule produces two surface suffixes for nouns covered by this regular pattern; loans and lexicalized exceptions can require additional handling.

The finite-state solution. We write the plural morpheme lexically as lAr, where A is the harmony archiphoneme, plus two two-level rules:

where:

• $A$: the archiphoneme, the underspecified vowel written lexically inside the plural suffix lAr
• $A{:}e$ / $A{:}a$: the correspondence pair, realized as the front surface vowel e or the back surface vowel a
• $\textit{FrontVowel}$ / $\textit{BackVowel}$: the symbol class of the stem's last vowel (e, i, ö, ü or a, ı, o, u)
• $C_0$: zero or more consonants between the stem's last vowel and the archiphoneme
• $\_$: the position of the $A{:}e$ or $A{:}a$ correspondence itself
• $\Leftrightarrow$: the biconditional, so the rewrite happens exactly in this environment and nowhere else

The lexicon licenses ev +PL, and the constraints select evler. The same regular harmony mapping applies productively to stems in the modeled classes, including many novel regular stems, because it references a phonological class rather than memorizing each word. Loans, proper names, consonant alternations, and lexical exceptions require additional treatment. The machinery also extends to the ablative -den/-dan, the genitive -in/-ın/-ün/-un, and buffer consonants such as the y in araba+y+a.

The neural solution. We assemble a training set: (stem, PL, surface) triples like (ev, PL, evler) and (okul, PL, okullar), plus hundreds of other stems, and train a character-level encoder-decoder. The + delimiter explicitly marks where the stem ends and the feature tag begins. What the model must infer from the examples is how much of the stem to reproduce and which surface suffix follows it. Our attention model learns this behavior without an explicit pointer-copy operation.

The code section below demonstrates both ideas, though its rule-based functions are ordinary Python rather than compiled FSTs. The interesting differences appear at the edges: an explicit rule handles a new regular stem without training examples, while the neural model may fail on an unfamiliar character pattern. Conversely, extending a hand-built grammar requires engineering the new behavior, while a neural model can learn it when enough representative examples are available. We will quantify generalization within our synthetic dataset.

We will build three artifacts: a rule-based English plural analyzer and generator, a rule-based Turkish inflector, and a small neural sequence-to-sequence inflector trained on synthetic Turkish-style data that we then test on held-out stems. The Python rules illustrate finite-state behavior without compiling an FST library, including how reversing local rules can yield multiple candidate analyses.

First, the setup cell. It seeds all randomness and imports what we need for the entire chapter.

The chart makes the pattern concrete: an applicable -ies or -es reversal can coexist with the separate final-s reversal, and neither path knows whether the recovered stem is a real word. Plain -s forms such as cats and days, and the irregular children, which bypasses the rules entirely, produce one analysis in this toy grammar.

Now the rule-based Turkish inflector. We model vowel harmony as an explicit allomorphy table keyed by the stem's final vowel, plus buffer consonants (y after a vowel in the dative, n in the genitive). We deliberately omit Turkish consonant assimilation (real kitap → kitabın, with p becoming b); we will note what that omission costs.

Each row has exactly one marked cell, which is the visual signature of a deterministic rule: no stem-final vowel is left underspecified, and the rows partition into a front block and a back block. This is precisely the structure the archiphoneme rule pair encodes, and the same grid is what the neural model must induce implicitly from the training pairs.

For the neural model, we need training pairs. We generate 400 Turkish-like stems from consonant-vowel syllables constrained to one harmony class, an intentionally simplified approximation to the dominant pattern in many native Turkish roots. Loans and several exceptional patterns are outside this generator. We hold out 80 stems entirely from training. Each stem pairs with four inflection tags, giving 1,280 training pairs and 320 held-out pairs. The input is the stem and tag joined by +, and the target is the surface form.

Each held-out stem was never seen in training, so accuracy on the test pairs measures transfer to unseen stems rather than recall of exact training strings. The test stems still come from the same synthetic generator as the training stems, so this is an in-distribution stem split; it does not measure performance under a different language or stem distribution. Shared-task evaluations such as SIGMORPHON likewise use held-out lemma–feature–form triples, though their datasets and splits are substantially richer than this demonstration.

Next, the character vocabulary and tensor preparation. We pad everything to fixed lengths computed from the training data.

Exact-match accuracy on training stems: 100.0%
Exact-match accuracy on held-out stems: 64.7%

These two numbers are the key empirical result of this chapter. Perfect training accuracy says the model fit the pairs, which is expected but uninformative on its own. The held-out number is the real test: those 80 stems and their 320 inflected forms never appeared in training. An exact-match score of 64.7% is far above what an exact-stem memorizer could achieve, so the model has learned reusable copying and harmony behavior. The 35.3-point gap from training accuracy is equally important: this small attention model has not induced a perfectly systematic rule, and exact copying remains brittle on unfamiliar strings.

All computation is done, so we can render the training curve and the per-tag generalization comparison from precomputed values. First, the loss curve.

rocon+GEN      predicted=roconun      gold=roconun      correct
riri+PL        predicted=ririler      gold=ririler      correct
nükdötöf+PL    predicted=nükdötler    gold=nükdötöfler  WRONG
zirzüc+PL      predicted=zirzücler    gold=zirzücler    correct
toptaco+DAT    predicted=toptacoya    gold=toptacoya    correct
pıla+GEN       predicted=pılanın      gold=pılanın      correct

Five of these six held-out predictions match exactly. The displayed error deletes part of the stem in nükdötöfler while retaining the correct plural suffix; it is a copying failure, not a harmony or buffer-consonant error. Six examples are useful for inspection but too few to characterize a general error profile.

Our neural model works on isolated words, which is enough for many analysis tasks. Sentence context becomes important when several analyses survive and must be disambiguated. Many practical pipelines therefore combine contextual part-of-speech or feature prediction with a lemmatization component, but the component order and algorithms differ across toolkits and languages. The code below shows spaCy's user-facing pattern; we mark it eval: false because it requires a separately downloaded model package.

Running it (after the one-time local install of the model package) shows the two occurrences of flies receive the same lemma but different part-of-speech tags: fly as a plural noun in the first clause, fly as a verb in the second. That distinction comes from sentence context, which an isolated-word analyzer lacks. spaCy's internal pipeline is not the weighted-FST-plus-HMM/CRF architecture from Part VI. Such a combination is one possible design: an FST enumerates lexical analyses, and a contextual sequence model ranks them.

Having built both demonstrations, we can compare the approaches across the dimensions in Table fst-vs-neural.

Arabic and Hebrew illustrate why the last row needs nuance. Root-and-pattern morphology interleaves consonantal roots and vocalic patterns: the Arabic root k-t-b is related to forms such as kataba and kitāb. Finite-state systems can model such patterns with appropriate lexical representations and transducers. Neural character models can learn them from examples, but results depend on the data coverage and representation as well as the evaluation conditions.

The paradigms are complementary. Finite-state machines provide grammar-relative guarantees and can be inverted; their behavior on encoded stems and rules is transparent, but coverage requires expert labor. Neural models learn from data and can generalize to unseen strings, though they may produce illegal-looking or unattested forms. Hybrid systems are an active engineering and research pattern: a neural model can propose an analysis or generation, while an FST supplies lexical candidates or hard constraints.

Finite-state morphology's deepest limitation is not mathematical but human: the lexicon bottleneck. Irregular forms, exceptional stems, continuation classes, and morphophonological alternations must be represented and tested. The engineering effort grows with lexical breadth and alternation complexity, whether the language has a few broad inflectional classes with many irregular lexemes or richer systems of noun classes and stem alternations. Coverage gaps are silent failures: an analyzer may return no analysis for an out-of-lexicon word, and it cannot by itself decide whether the word is rare, new, a typo, or a proper noun. Weighting ranks existing paths but does not create missing ones. Two-level constraints avoid sequential rule ordering, but complex constraint interactions, lexical exceptions, and incomplete test coverage can still produce subtle bugs. Formal correctness remains conditional on the encoded grammar.

Neural morphology trades that brittleness for different problems. It needs parallel data, which is scarce for many morphologically rich languages. It can generate forms that violate phonotactic or inflectional patterns, particularly for rare feature combinations, and a plain seq2seq architecture does not distinguish licensed from unlicensed outputs. Exact-match accuracy also hides the distribution of errors: a score of 90 percent does not identify which constructions make up the remaining 10 percent. When reading inflection benchmarks, inspect the split design, frequency distribution, per-feature results, and irregular or otherwise difficult subsets rather than relying on one aggregate number.

The impact of this chapter's machinery is broad and often invisible. Spell checkers and hyphenators use several architectures; so do text-to-speech systems, search normalization, and machine-translation pipelines. These systems may be finite-state, statistical, neural, or hybrid, with morphological analysis as one component. For neural language models, morphology is entangled with the tokenization you studied in Part V: a vocabulary poorly matched to a morphologically rich language can split words into more tokens, increasing sequence length and changing the evidence available for learning inflectional patterns. We will revisit that problem for multilingual models in Part XXVI.

You now have a working command of both paradigms of morphological computation.

• Morphological analysis maps surface words to candidate (lemma, feature bundle) analyses, and generation maps analyses to licensed forms. Either direction can be relational rather than one-to-one; weights and context help rank alternatives.
• Finite-state transducers model morphotactics and morphophonology as regular relations. They are closed under composition and inversion, so one encoded relation can support analysis and generation. Their guarantees are exact relative to the grammar. Two-level constraints describe lexical–surface correspondences in parallel rather than as an ordered cascade.
• Weighted FSTs rank competing analyses with negative log probabilities and pick the minimum-weight path, a transducer-relative of Viterbi decoding.
• Neural sequence-to-sequence models learn inflection from character-level parallel data. Attention or explicit copy mechanisms can help preserve stems, but generalization must be measured on held-out data.
• The paradigms are complementary: rule systems offer grammar-relative guarantees and interpretability, neural systems learn coverage from data, and hybrid designs can combine candidates, learned scores, and hard constraints.

The key parameters for the neural seq2seq inflector are:

• emb_dim = 32: Dimension of the character embeddings.
• hid_dim = 64: Hidden-state size of both GRUs. The decoder's attention can access the full sequence of encoder states, so this is not a single-vector stem bottleneck.
• tf_ratio = 1.0: Probability of teacher forcing at each training decoder step in this experiment. Lower values expose the decoder to its own earlier predictions, but do not by themselves guarantee better copying.
• lr = 3e-3: Learning rate for the Adam optimizer used for the 60-epoch run.

In the next chapter we climb one level of linguistic structure, from the inside of words to the arrangement of words into phrases, with grammars and constituency structure. When we reach dependency syntax and Universal Dependencies later in this part, you will see these morphological feature bundles attached to the tokens for which the relevant features are annotated; a token's FEATS field may also be empty.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about morphological analysis, finite-state transducers, and neural inflection.