Phrase-based SMT & MERT (2003) › Limitations and Challenges

Despite their success, phrase-based translation and MERT faced significant limitations. Phrase-based systems still made largely local decisions, translating phrases independently without strong global coherence mechanisms. A translation might get each phrase right in isolation but produce awkward or incoherent sentences overall. Long-range dependencies remained challenging: a pronoun in one phrase might refer to a noun many phrases earlier, and phrase-based systems struggled with such relationships.

The phrase extraction process, while effective, was somewhat ad hoc. Phrase pairs were extracted based on alignment consistency, but this didn't guarantee that extracted phrases were semantically coherent or linguistically meaningful. Some extracted phrases were useful, while others were arbitrary word sequences that happened to align consistently. The system had no principled way to distinguish meaningful phrases from alignment artifacts, relying on frequency and probability estimates to prioritize useful phrases.

MERT's limitations became more problematic as systems grew more complex. The algorithm scaled poorly beyond about 20-30 features, limiting the sophistication of feature sets. MERT could also be unstable, with different random seeds or slight changes in development data producing very different weight settings. The computational cost of MERT, requiring many decoding passes, made it expensive for large systems or frequent retuning.

Domain adaptation remained challenging. Phrase-based systems trained on newswire text often degraded when applied to other domains like social media, scientific literature, or conversational speech. Domain-specific vocabulary, including characteristic phrases and stylistic patterns, required retraining or domain adaptation techniques. This limited portability across text types and genres.

The phrase-based approach also struggled with certain linguistic phenomena. Morphologically rich languages, where word forms change significantly, were more challenging because phrase extraction depended on word-level alignments that might be less reliable. Languages with free word order required more sophisticated reordering models. Low-resource language pairs, with limited parallel data, couldn't support the phrase extraction and probability estimation that phrase-based systems required.