FastText › FastText for Morphologically Rich Languages
Language AI HandbookAll chapters FastText › FastText for Morphologically Rich LanguagesOne section · Sections in this chapter10 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
English morphology is relatively shallow: a handful of common suffixes cover most of the grammatical variation, and the number of distinct forms per word is modest. The real demonstration of FastText's power is on languages where morphology is far more productive, where the number of distinct word forms per root can run into the thousands.
German builds compound nouns without spaces. "Lebensversicherungsgesellschaft" (life insurance company) joins three meaningful roots: "Leben" (life), "Versicherung" (insurance), and "Gesellschaft" (company). A Word2Vec model will never see this exact compound in most corpora. FastText, however, will share n-grams between the compound and its components, encoding the compositional meaning in the embedding automatically. The n-gram "leben" (from "Leben") will match both the standalone word and the compound prefix. The n-gram "gesellschaft" will match the component and the compound suffix.
This compositional encoding is not perfect. FastText does not know where one root ends and another begins inside the compound. But it does not need to: the n-gram overlap gives a rough approximation of the compositional structure, which is enough to produce useful embeddings even for compounds that appear only once in a billion-word corpus.
german_compounds = [
("Lebensversicherung", "life insurance", ["Leben", "Versicherung"]),
("Krankenhaus", "hospital", ["Krank", "Haus"]),
("Handschuh", "glove", ["Hand", "Schuh"]),
("Kindergarten", "kindergarten", ["Kinder", "Garten"]),
]
def compound_overlap(compound, components, min_n=3, max_n=5):
"""Fraction of compound's n-grams that appear in any component."""
compound_ngs = set(get_ngrams(compound.lower(), min_n, max_n))
component_ngs = set()
for c in components:
component_ngs.update(get_ngrams(c.lower(), min_n, max_n))
shared = compound_ngs & component_ngs
return len(compound_ngs), len(shared), sorted(shared)[:5]
analysis = []
for compound, meaning, components in german_compounds:
total, shared_n, examples = compound_overlap(compound, components)
analysis.append(
{
"compound": compound,
"meaning": meaning,
"components": components,
"total": total,
"shared": shared_n,
"overlap_pct": 100 * shared_n / total,
"examples": examples,
}
)N-gram overlap: German compounds with their components
------------------------------------------------------------
Lebensversicherung (life insurance)
Components: Leben + Versicherung
Compound n-grams: 51
Shared with components: 39 (76%)
Shared examples: ['<le', '<leb', '<lebe', 'ben', 'che']
Krankenhaus (hospital)
Components: Krank + Haus
Compound n-grams: 30
Shared with components: 15 (50%)
Shared examples: ['<kr', '<kra', '<kran', 'ank', 'aus']
Handschuh (glove)
Components: Hand + Schuh
Compound n-grams: 24
Shared with components: 15 (62%)
Shared examples: ['<ha', '<han', '<hand', 'and', 'chu']
Kindergarten (kindergarten)
Components: Kinder + Garten
Compound n-grams: 33
Shared with components: 24 (73%)
Shared examples: ['<ki', '<kin', '<kind', 'art', 'arte']Every German compound shares a substantial fraction of its n-grams with its component roots. "Krankenhaus" (hospital) shares about 60% of its n-grams with "Krank" (sick) and "Haus" (house). A FastText model trained on a German corpus will place "Krankenhaus" in a region of embedding space influenced by both components' embeddings, even if the compound never appears in training. The embedding captures compositional meaning for free, without any explicit morphological analyzer or dictionary lookup.
The same logic applies across a wide range of languages and linguistic phenomena:
-
Turkish and Finnish (agglutinative morphology): These languages stack grammatical suffixes onto roots in long chains. "Evlerinizden" (from your houses) in Turkish shares n-grams with "ev" (house) and related suffixed forms. FastText handles new suffix combinations naturally because it has seen the individual suffix n-grams many times during training.
-
Arabic and Hebrew (templatic morphology): These languages use a three-consonant root system where a single root interleaves with vowel patterns to produce different word classes and tenses. Character n-grams capture the consonant sequences that encode root meaning, even though the vowel patterns change between forms.
-
Any language with productive affixation: Medical and scientific terminology across all languages uses Latin and Greek roots that recombine in predictable ways. "Electroencephalography" shares n-grams with "electro," "encephalo," "graph," and their Latin derivatives, giving FastText a way to anchor even unfamiliar coinages in a meaningful region of embedding space.
A Worked Example: Morphological Generalization
To make the cross-lingual benefit concrete, consider what happens when a FastText model trained on Turkish encounters the word "evlerinizden" (from your houses), which might not appear in the training corpus at all. The word decomposes into the root "ev" (house), followed by the plural suffix "-ler," the possessive suffix "-iniz" (your), and the ablative case suffix "-den" (from). Each suffix appears in hundreds of other Turkish words in training. The n-grams "<evl," "evle," "vler," "leri," "erin," "riniz" and others appear in related words. The embedding FastText assigns to this never-seen word will be a blend of embeddings it has seen from "ev," from other words ending in "-den," from other words containing "iniz," and so on. The result is imperfect but meaningful, far better than the zero-vector fallback that Word2Vec would provide.
This is also why FastText was a significant advance for low-resource language processing. Languages with small Wikipedia corpora but rich morphology, like Swahili or Icelandic, have relatively few unique surface forms in any given training corpus, but those forms cover a much larger space of meanings through morphological combination. FastText exploits this structure to learn embeddings that generalize across the morphological space, making it a practical choice even when data is scarce.