Term Frequency › Term Frequency Weighting Schemes

You've seen how the Bag of Words model counts word occurrences and the n-gram model captures local word sequences. But not all word counts are created equal. A word that appears 100 times in a document tells a very different story depending on whether the document is 200 words long or 20,000 words long. And a word appearing twice differs from one appearing once, but is it twice as important? These questions sit at the heart of term frequency weighting.

Term frequency is the simplest and most intuitive way to transform raw word counts into something more analytically useful. The idea is straightforward: words that appear more often in a document are more representative of that document's content. But "more often" hides a surprising amount of nuance. Raw counts, logarithmic scaling, boolean presence, augmented normalization, and L2 normalization all encode different assumptions about what "frequency" means and how it should influence downstream comparisons between documents. Understanding each variant, when to use it, and why it was designed the way it was will equip you to make principled decisions about text representation.

This chapter covers every major variant of term frequency weighting, the mathematical intuitions behind each, and how sparsity shapes the practical efficiency of these representations. It also grounds the discussion in real-world data, so you leave with an understanding of what TF distributions look like in practice and how different weighting choices affect the geometry of the resulting document space.