FastText › Key Parameters

The key parameters for FastText training are:

• min_n: Minimum n-gram length. The default value of 3 captures short prefixes and suffixes. Setting it to 2 includes bigrams like "th" and "ed," which helps for very short words but roughly doubles the n-gram count and can add noisy, low-information substrings to many embeddings.

• max_n: Maximum n-gram length. The default value of 6 balances morpheme capture with vocabulary size. For agglutinative languages with longer suffixes (Turkish, Finnish, Hungarian), values of 7 or 8 improve coverage of long morphological patterns.

• bucket: Number of hash buckets for n-gram storage. The default of 2,000,000 keeps collision rates below 5% for typical English vocabularies. Reducing this saves memory but increases collisions; values below 100,000 noticeably degrade embedding quality. For small datasets, 500,000 to 1,000,000 is often sufficient.

• dim: Embedding dimensionality. Values of 100 to 300 are standard. Lower dimensions train faster and generalize better on small datasets; higher dimensions capture more nuance but require more data to fill the representational space without overfitting.

• epoch: Training passes over the corpus. The range of 5 to 25 covers most use cases. More epochs help on smaller datasets; for very large corpora (Common Crawl scale), even 1 epoch provides enough training signal for high-quality embeddings.

• lr: Learning rate. The default of 0.05 works well with the linear decay schedule FastText uses by default, which reduces the learning rate linearly to zero over training. Higher values speed training but risk overshooting on small or noisy corpora.

• neg: Number of negative samples per positive pair. Values of 5 to 10 are standard. Higher values improve training quality for rare words at the cost of slower training. For very large corpora, 5 negative samples is usually sufficient.

• ws: Context window size. The default of 5 captures sentence-level co-occurrence. Smaller windows capture tight syntactic relationships (useful for part-of-speech tagging and named entity recognition); larger windows capture broader topical associations (useful for document classification and information retrieval).