Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Chunking › Chunking with spaCy

Michael BrenndoerferUpdated March 21, 20262 min read

Language AI HandbookAll chapters

Part 6 of 67Sequence Labeling

Chapter 39Chunking › Chunking with spaCy

Reading section 12 of 15 · Chunking with spaCy · 2 minSection 12 of 15 · 2 min

Section 12 of 15 · 2 min read

Chunking › Chunking with spaCyOne section · Sections in this chapter12 / 15 ↓

Sections in this chapter15

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

spaCy provides noun phrase chunking through its noun_chunks property. Unlike NLTK's regex-based approach, spaCy derives chunks from its dependency parse. This gives it access to syntactic relations like subject and object, producing more linguistically accurate chunks.

In[47]:

Code

import spacy

try:
    nlp = spacy.load("en_core_web_sm")
except OSError:
    import subprocess

    subprocess.run(
        ["python", "-m", "spacy", "download", "en_core_web_sm"],
        capture_output=True,
    )
    nlp = spacy.load("en_core_web_sm")

# Example sentences that test different NP structures
spacy_sentences = [
    "The quick brown fox jumped over the lazy dog.",
    "A major breakthrough in artificial intelligence was announced yesterday.",
    "The President of the United States addressed the nation.",
    "Machine learning models trained on large corpora show impressive results.",
]

spacy_results = []
for sent in spacy_sentences:
    doc = nlp(sent)
    chunks = [
        (chunk.text, chunk.root.text, chunk.root.dep_)
        for chunk in doc.noun_chunks
    ]
    spacy_results.append((sent, chunks))

Out[48]:

Console

spaCy Noun Chunk Extraction
=================================================================

Sentence: The quick brown fox jumped over the lazy dog.
  Noun chunks:
    'The quick brown fox'
      Root: 'fox' | Dependency role: nsubj
    'the lazy dog'
      Root: 'dog' | Dependency role: pobj

Sentence: A major breakthrough in artificial intelligence was announced yesterday.
  Noun chunks:
    'A major breakthrough'
      Root: 'breakthrough' | Dependency role: nsubjpass
    'artificial intelligence'
      Root: 'intelligence' | Dependency role: pobj

Sentence: The President of the United States addressed the nation.
  Noun chunks:
    'The President'
      Root: 'President' | Dependency role: nsubj
    'the United States'
      Root: 'States' | Dependency role: pobj
    'the nation'
      Root: 'nation' | Dependency role: dobj

Sentence: Machine learning models trained on large corpora show impressive results.
  Noun chunks:
    'Machine learning models'
      Root: 'models' | Dependency role: nsubj
    'large corpora'
      Root: 'corpora' | Dependency role: pobj
    'impressive results'
      Root: 'results' | Dependency role: dobj

spaCy's chunks carry more information than regex chunks. The root attribute identifies the head noun of the phrase, and dep_ identifies its syntactic role (nsubj for subject, dobj for direct object, pobj for object of a preposition). This makes spaCy's noun chunks particularly useful for information extraction and relation detection.

spaCy does not expose VP or PP chunks through a simple property because its dependency-based analysis handles those structures differently. If you need all three chunk types, NLTK's regex or statistical chunker remains the more direct tool.