IBM Statistical Machine Translation (1991) › The Alignment Problem: Learning Hidden Correspondences
History of Language AIAll chapters IBM Statistical Machine Translation (1991) › The Alignment Problem: Learning Hidden CorrespondencesOne section · Sections in this chapter4 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The IBM approach treated word alignment as a hidden correspondence to be learned from data, a concept that would prove influential far beyond machine translation. When you have a sentence in English and its translation in French, the correspondence between individual words is not explicitly marked in the data. You can observe that "The cat sat on the mat" corresponds to "Le chat s'est assis sur le tapis," but which English words produced which French words remains hidden.
This creates a chicken-and-egg problem. To learn good translation probabilities, you need to know the alignments, which words correspond to which. But to determine good alignments, you need to know the translation probabilities, which words are likely translations of each other. The IBM researchers solved this through the expectation-maximization algorithm, an iterative procedure that alternates between estimating alignments given current translation probabilities, and then updating translation probabilities given the estimated alignments.
Consider the example of "The cat sat on the mat" and "Le chat s'est assis sur le tapis." Through training on thousands of similar sentence pairs, the model learns that "cat" reliably aligns with "chat," that "mat" aligns with "tapis," and that "sat" aligns with "s'est assis," a two-word phrase in French expressing what English captures in a single word. These alignment patterns emerge automatically from the data, without anyone having to explicitly encode rules about French verb formation or article usage.
The alignment concept turned out to be remarkably powerful. It provided a way to automatically discover structural correspondences between languages, capturing not just vocabulary differences but also how grammatical structures map onto each other. This automatic discovery of hidden structure would become a recurring theme in language AI, reappearing in different forms in later developments like attention mechanisms in neural networks.