CBOW Model › CBOW Model
Language AI HandbookAll chapters CBOW Model › CBOW ModelOne section · Sections in this chapter1 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
In the previous chapter, we explored the Skip-gram model, which learns word embeddings by training a neural network to predict surrounding context words from a single center word. The Continuous Bag of Words (CBOW) model reverses this relationship: given several surrounding context words, predict the center word that belongs in the middle. This seemingly small inversion produces a model with different strengths, training dynamics, and practical applications.
Both architectures were introduced by Mikolov et al. in the 2013 Word2Vec paper. While Skip-gram excels at capturing representations for rare words, CBOW trains faster, scales better to large corpora, and produces stronger embeddings for frequent words. Understanding both models gives you a complete picture of the Word2Vec framework and helps you choose the right tool for a given task.
This chapter covers the CBOW architecture from the ground up. We build intuition for why averaging context embeddings works, work through the mathematics of the forward pass and objective function, derive the gradients used in training, implement a working CBOW model from scratch, and compare the two architectures in depth.