BLEU Score › Key Parameters

The key parameters for BLEU calculation are:

• weights: Weights for each n-gram order (typically unigrams through 4-grams). Uniform weighting (0.25 each) is standard, but adjusting these allows you to emphasize matching at specific granularities. Higher weights on larger n-grams reward fluency and word order accuracy, while higher weights on unigrams prioritize vocabulary coverage. Some implementations use non-uniform weights such as [0.1, 0.2, 0.3, 0.4] to emphasize longer matches.

• smoothing: Small constant (default 1e-10) added to zero precision values to prevent log(0) errors, particularly important for sentence-level BLEU on short sentences where higher-order n-grams may not overlap with references. Different smoothing methods exist, including Laplace smoothing (adding 1 to all counts) and Lin smoothing, which becomes more aggressive for higher-order n-grams.

• max_n: The maximum n-gram order to consider. Standard BLEU uses 4. Reducing to 2 or 3 is sometimes appropriate for tasks where local fluency is expected but longer phrase matches are unreliable due to legitimate translation variation.

• case: Whether to lowercase all text before comparison. Case normalization is standard for most benchmarks but can matter for proper noun translation quality, where capitalizing the correct entity name might be meaningful.

• tokenization: How to split text into tokens. Different tokenizers produce different token sequences, which changes n-gram counts. The sacrebleu library standardizes this with its tok parameter, and using a consistent tokenizer is essential for reproducible comparisons.