Chunking › Chunking

Sentences contain meaningful groups of words that function as units. In "The quick brown fox jumps over the lazy dog," you naturally perceive "the quick brown fox" as a noun phrase describing the subject, "jumps" as the action, and "over the lazy dog" as a prepositional phrase describing where the action occurs. Chunking, also called shallow parsing, identifies these non-overlapping segments without building a full syntactic tree.

Chunking occupies a useful middle ground between part-of-speech tagging and full parsing. POS tagging labels individual words. Full parsing constructs hierarchical tree structures showing how phrases nest within phrases. Chunking finds a middle ground: it groups consecutive words into flat, non-recursive chunks without attempting to show how chunks relate to each other or how they nest.

This chapter explores chunking from multiple angles. You'll learn the major chunk types, understand how IOB tagging represents chunk boundaries, implement chunkers using both regular expressions and machine learning, and see how chunking is a preprocessing step for information extraction and other later tasks.