Bag of Words › A Worked Example: Spam Detection
Language AI HandbookAll chapters Bag of Words › A Worked Example: Spam DetectionOne section · Sections in this chapter9 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To make the BoW model concrete, let's trace through a complete spam detection pipeline from raw text to a classification decision. Email spam detection was one of the first real-world applications where BoW excelled, and the reason is instructive: spam emails use a distinctive vocabulary (words like "free", "winner", "urgent", "prize") that differs reliably from legitimate email. You don't need to understand the grammatical structure of "CONGRATULATIONS! You have won a FREE prize!" to know it's likely spam. The presence of those words is sufficient.
Imagine you are building a spam filter for a small email service. You have 100 labeled emails: 50 spam and 50 legitimate. The process unfolds in three stages. In the first stage, preprocessing, you tokenize each email, lowercase all words, and strip punctuation. You build a vocabulary from all the words that appear in the training set. In the second stage, vectorization, you transform each email into a count vector using that vocabulary. In the third stage, classification, you train a model (such as Naive Bayes) on the count vectors and their labels.
# Quick demonstration: sentiment classification with BoW
from sklearn.naive_bayes import MultinomialNB
# Simple sentiment dataset
reviews = [
"This movie was fantastic and amazing",
"Absolutely loved this film, brilliant",
"Great movie, highly recommend",
"Terrible waste of time, boring",
"Awful movie, complete disaster",
"Worst film I have ever seen",
]
labels = [1, 1, 1, 0, 0, 0] # 1 = positive, 0 = negative
# Vectorize and train
vec = CountVectorizer()
X = vec.fit_transform(reviews)
clf = MultinomialNB()
clf.fit(X, labels)
# Test on new reviews
test_reviews = [
"This was a brilliant and fantastic experience",
"Complete waste of time, terrible",
]
test_X = vec.transform(test_reviews)
predictions = clf.predict(test_X)
probabilities = clf.predict_proba(test_X)Sentiment Classification with Bag of Words: -------------------------------------------------- Review: 'This was a brilliant and fantastic experience' Prediction: Positive (confidence: 97.7%) Review: 'Complete waste of time, terrible' Prediction: Negative (confidence: 97.3%)
Even this trivial example shows BoW capturing sentiment through word presence. Words like "brilliant", "fantastic", and "terrible" carry strong sentiment signals regardless of context. Notice that the model correctly classifies both test reviews despite having very small training data. The key insight is that sentiment-bearing words cluster reliably: positive reviews use a different vocabulary than negative ones, and that vocabulary-level signal is sufficient for classification on simple examples.
In practice, this pattern extends to many classification tasks. Topic classification (sports vs. politics vs. technology) works well with BoW because different topics use distinctly different vocabularies. Language detection (is this English or French?) works extremely well because the vocabularies have almost no overlap. Even author identification has been done with BoW: different authors have characteristic word preferences that persist across their writings.