Document Chunking: Optimizing RAG Retrieval Pipelines › Recursive Chunking
Language AI HandbookAll chapters Document Chunking: Optimizing RAG Retrieval Pipelines › Recursive ChunkingOne section · Sections in this chapter5 / 16 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Real documents have hierarchical structure: sections, subsections, paragraphs, sentences, and words. Recursive chunking exploits this structure by attempting to split at the most meaningful boundary possible. The strategy embodies a simple but effective principle: always prefer splitting at the highest-level boundary that still produces chunks within the size limit.
Think of recursive chunking as the way you might physically organize a large collection of documents. You first try to separate them by major category (sections). If a category still has too many documents, you separate by subcategory (paragraphs). If a subcategory is still too large, you split by individual pages (sentences). Only as a last resort do you cut mid-page (characters). At each level, you prefer the coarser division, resorting to finer divisions only when necessary.
The algorithm tries a sequence of separators in order of decreasing granularity. If splitting by double newlines (paragraph boundaries) produces chunks that are small enough, it stops there. If not, it falls back to single newlines, then sentences, then words, then characters. This layered approach means the algorithm naturally adapts to different parts of a document. A section with short paragraphs will be split along paragraph boundaries, preserving each paragraph as a self-contained unit. A section containing a single long paragraph will be split at sentence boundaries within that paragraph.
This approach is popularized by LangChain's RecursiveCharacterTextSplitter and is one of the most widely used strategies in practice. Its popularity stems from the fact that it produces reasonable results across a wide variety of document types without requiring any domain-specific configuration beyond the choice of separators. For many teams building their first RAG system, recursive chunking is the right default choice: it respects document structure when that structure exists, and degrades gracefully to simpler splits when it does not.
def recursive_chunk(text, chunk_size=200, overlap=50, separators=None):
"""Recursively split text using a hierarchy of separators."""
if separators is None:
separators = ["\n\n", "\n", ". ", " ", ""]
# Base case: text fits in one chunk
if len(text) <= chunk_size:
return [text]
# Find the best separator that produces a split
chosen_sep = separators[-1] # fallback to character-level
for sep in separators:
if sep in text:
chosen_sep = sep
break
# Split text using the chosen separator
parts = text.split(chosen_sep) if chosen_sep else list(text)
# Merge parts into chunks that respect the size limit
chunks = []
current_chunk = ""
for part in parts:
# Reconstruct with separator
candidate = current_chunk + chosen_sep + part if current_chunk else part
if len(candidate) <= chunk_size:
current_chunk = candidate
else:
if current_chunk:
chunks.append(current_chunk)
# If a single part exceeds chunk_size, recurse with finer separators
if len(part) > chunk_size:
remaining_seps = separators[separators.index(chosen_sep) + 1 :]
if remaining_seps:
sub_chunks = recursive_chunk(
part, chunk_size, overlap, remaining_seps
)
chunks.extend(sub_chunks)
current_chunk = ""
else:
current_chunk = part
else:
current_chunk = part
if current_chunk:
chunks.append(current_chunk)
return chunksLet's test it on a structured document with paragraphs:
structured_text = """Climate Change Overview
Global temperatures have risen by approximately 1.1 degrees Celsius since the pre-industrial era. This warming is primarily driven by greenhouse gas emissions from human activities, including burning fossil fuels and deforestation.
Impact on Ecosystems
Rising temperatures affect biodiversity across the planet. Coral reefs are bleaching at unprecedented rates. Arctic sea ice is declining, threatening polar bear habitats. Migration patterns of birds and marine species are shifting northward.
Mitigation Strategies
Renewable energy sources like solar and wind power are expanding rapidly. Many countries have committed to net-zero emissions targets by 2050. Carbon capture technology is being developed but remains expensive. Individual actions like reducing meat consumption and flying less also contribute to emission reductions."""rec_chunks = recursive_chunk(structured_text, chunk_size=250, overlap=0)Chunk 0: [23 chars] 'Climate Change Overview' Chunk 1: [231 chars] 'Global temperatures have risen by approximately 1.1 degrees Celsius since the pre-industrial era. This warming is primarily driven by greenhouse gas emissions from human activities, including burning fossil fuels and deforestation.' Chunk 2: [20 chars] 'Impact on Ecosystems' Chunk 3: [241 chars] 'Rising temperatures affect biodiversity across the planet. Coral reefs are bleaching at unprecedented rates. Arctic sea ice is declining, threatening polar bear habitats. Migration patterns of birds and marine species are shifting northward.' Chunk 4: [21 chars] 'Mitigation Strategies' Chunk 5: [209 chars] 'Renewable energy sources like solar and wind power are expanding rapidly. Many countries have committed to net-zero emissions targets by 2050. Carbon capture technology is being developed but remains expensive' Chunk 6: [105 chars] 'Individual actions like reducing meat consumption and flying less also contribute to emission reductions.'
The recursive approach respects paragraph boundaries when possible, falling back to finer-grained splits only when a paragraph exceeds the size limit. This preserves the document's logical structure in the chunks. The "Climate Change Overview" heading and its accompanying paragraph naturally form one chunk, while the "Impact on Ecosystems" section forms another.
The algorithm arrives at these natural divisions not because it understands the content, but because the paragraph boundaries encoded by double newlines happen to align with topical boundaries. This is a pattern that holds reliably across well-structured documents, whether those documents are Wikipedia articles, academic papers, or technical documentation. The recursive chunker exploits this alignment silently and automatically, which is why it works so well in practice without domain-specific tuning.
Recursive chunking differs from simple separator-based splitting. A simple paragraph splitter would split every double newline, even when two short consecutive paragraphs together fit within the chunk size limit. Recursive chunking instead merges adjacent short paragraphs into a single chunk, only splitting when the merged size would exceed the limit. This behavior is generally preferable because it reduces the total number of chunks and ensures each chunk carries enough content to be useful.
Key Parameters
The key parameters for recursive chunking are:
- chunk_size: The hard limit on chunk size. The algorithm attempts to keep chunks under this limit while using the largest possible separators. This means the algorithm always prefers a paragraph-level split over a sentence-level split, as long as both produce chunks that fit within this budget.
- overlap: The number of characters to overlap between chunks. In recursive chunking, overlap is applied after the initial splitting pass, duplicating content from the end of one chunk into the beginning of the next.
- separators: An ordered list of strings used to split the text (e.g.,
["\n\n", "\n", " ", ""]). The algorithm tries them in sequence to find the best split point. The ordering encodes your preference for which types of boundaries should be preserved. By placing paragraph separators first, you ensure the algorithm only resorts to sentence or word breaks when paragraph-level splits are insufficient.