Morphological Analysis and Generation › Neural Morphology as Sequence-to-Sequence Learning

The neural reframing treats inflection as string-to-string transduction learned from data. Given a training set of triples (lemma, morphosyntactic features, target form), a character-level encoder-decoder model learns the mapping end to end, with no hand-written rules. The landmark evaluations here are the SIGMORPHON shared tasks (2016 onward) and the CoNLL-SIGMORPHON 2017–2018 shared tasks, which established character-level encoder-decoder networks as the standard for what is often called morphological reinflection: given watch and the features V;PST, produce watched.

Why characters rather than words? Because the whole point is to generalize to unseen stems. A character-level model can inflect a stem it has never encountered, by combining what it has learned about the -ed pattern with the phonology of the stem it reads, so screeched is reachable even if screech never appeared in training. This is the same subword insight that powered FastText's character n-gram embeddings in Part IV, Chapter 8, applied to a transduction task instead of a representation task.

A minimal architecture looks like this:

• an encoder: an embedding layer over the character vocabulary feeding a GRU or LSTM, which compresses the input string (say, the lemma plus a tag string like N;PL) into a sequence of hidden states;
• a decoder: a second GRU that generates the output string one character at a time, attending over encoder states;
• an optional alignment or copy mechanism: because the output usually shares most of its characters with the input, models can benefit from tracking input positions or explicitly copying characters. Aharoni and Goldberg (2017) used hard monotonic alignment with action-based character decoding rather than an explicit pointer-copy operation. Makarov and Clematide (2018) combined a related transition-based approach with imitation learning in a strong SIGMORPHON system.

The learned model can capture morphophonological patterns implicitly, without hand-written rules. Given suitable English training data, a character model can learn mappings such as walk $\to$ walked, carry $\to$ carried, and stop $\to$ stopped. Some experimental models overregularize irregular forms under particular data and evaluation conditions, but that behavior is not guaranteed and our Turkish-style experiment below does not test English irregulars.

There is a second neural family worth distinguishing. Instead of one word at a time, contextual lemmatizers and morphological taggers operate on whole sentences. Their architectures vary: contextual encoders can predict part-of-speech and morphological features, while lemma components may use lookup tables, edit rules, sequence models, or other language-specific mechanisms. Context helps distinguish flies the noun from flies the verb even though both share the lemma fly. Stanza and spaCy expose these analyses through a common pipeline interface, but their internal lemmatization architectures are not identical. We will meet transformer encoders in Part XXIV.

Finally, there is unsupervised morphological segmentation: given only raw text, look for recurring pieces shared by forms such as walks, walking, and walked. Morfessor formulates this as a compression problem under the Minimum Description Length principle, balancing the cost of a lexicon of morphs against the cost of encoding the corpus with them. The objective is designed for morphological segmentation, but it does not guarantee that every learned segment is a linguist's morpheme. BPE from Part V, Chapter 2 is a cousin with a different objective: it merges frequent symbol pairs and may cross morpheme boundaries.