Part-of-Speech Tagging › Cross-Lingual POS Tagging
Language AI HandbookAll chapters Part-of-Speech Tagging › Cross-Lingual POS TaggingOne section · Sections in this chapter14 / 18 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
One of the most active research areas in POS tagging is cross-lingual transfer: training a model on one language and applying it to another, with little or no annotated data in the target language. This problem is important because the world has thousands of languages, and creating annotated training data for all of them is infeasible.
The Universal Dependencies project provides the infrastructure for cross-lingual POS work by standardizing both the tag set and the annotation format across languages. A model trained on English UD data uses the same 17 UPOS tags as a model trained on Spanish, Russian, or Swahili data. This shared label space makes it possible, in principle, to transfer a model trained on a high-resource language to a low-resource language.
The challenge is that even with a shared label space, languages differ in their surface forms, morphology, and word order. A model trained on English encounters word sequences like DET ADJ NOUN. Japanese uses postpositions instead of prepositions, and the verb comes last rather than in the middle. A model that learns English distributional patterns cannot directly apply them to Japanese.
Multilingual pre-trained models like mBERT (multilingual BERT) and XLM-RoBERTa have made cross-lingual transfer much more effective. These models are pre-trained on text in over 100 languages simultaneously, learning representations that are partially language-agnostic. When you fine-tune mBERT on English POS annotations and then evaluate on, say, Bulgarian, the model achieves surprisingly high accuracy despite having seen no Bulgarian POS annotations. The pre-training has learned to align representations across languages for related concepts, and the fine-tuning in English teaches the classification head what the UPOS labels mean.
The degree of transfer depends heavily on linguistic relatedness. Transfer from English to German works well (both are Germanic languages with similar morphological and syntactic patterns). Transfer from English to Turkish is harder because Turkish is an agglutinative language with very different morphological structure and verb-final word order. Transfer from English to Chinese is somewhere in between: Chinese has minimal morphology (which simplifies some problems) but very different syntactic constructions.