XLM (2019) › 2019: XLM

In 2019, Facebook AI Research introduced XLM (Cross-lingual Language Model), a breakthrough in multilingual natural language processing that demonstrated how cross-lingual pretraining with translation language modeling could enable strong zero-shot and few-shot transfer across languages. XLM's ability to learn cross-lingual representations that captured semantic similarities across different languages opened up new possibilities for multilingual AI applications and influenced the development of many subsequent multilingual language models. The model's success demonstrated that neural language models could be trained to understand and generate text in multiple languages simultaneously, establishing new standards for multilingual NLP and influencing the development of many subsequent systems that could handle multiple languages with a single model.

XLM appeared as the field was moving from monolingual language models to multilingual systems. Models like BERT and GPT had performed well when trained on large monolingual corpora, but extending those results to multiple languages required new methods. Cross-lingual pretraining provided a bridge by transferring knowledge between languages and improving performance on low-resource languages.