Document Chunking: Optimizing RAG Retrieval Pipelines › Sentence-Based Chunking
Language AI HandbookAll chapters Document Chunking: Optimizing RAG Retrieval Pipelines › Sentence-Based ChunkingOne section · Sections in this chapter4 / 16 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
A more linguistically motivated approach uses sentence boundaries as the atomic unit. Rather than slicing text at arbitrary positions, this strategy first identifies where sentences end and then groups consecutive sentences together until a size limit is reached. The result is chunks that always begin and end at natural linguistic boundaries.
As we covered in the Sentence Segmentation chapter, identifying sentence boundaries is itself a non-trivial task that must handle abbreviations, decimal numbers, and other edge cases. A naive approach of splitting on periods fails for "Dr. Smith increased the dosage to 2.5 mg." Here, we use spaCy's sentence segmentation for reliable boundary detection. SpaCy's model combines statistical patterns with rule-based heuristics that handle most edge cases correctly.
The idea is simple but powerful: split the text into individual sentences first, then group consecutive sentences together into chunks until adding the next sentence would exceed the desired size limit. Think of sentences as Lego bricks. Each brick is a complete, self-contained unit. You can stack them together to form larger structures, but you never break a brick in the middle. This guarantees that no chunk will ever contain a partial sentence, and every chunk begins at the start of a sentence and ends at the conclusion of one.
This sentence-preservation property matters because embedding models are trained on complete, well-formed text. Most modern sentence embedding models fine-tune their weights on datasets of complete sentences or paragraphs. When you feed them truncated or fragmented text, the resulting embeddings are noisier and less semantically reliable. Preserving sentence boundaries aligns your chunks with the distribution of text the embedding model was trained on, producing higher-quality representations.
import spacy
nlp = spacy.load("en_core_web_sm")
def chunk_by_sentences(text, max_chunk_size=200, overlap_sentences=0):
"""Group sentences into chunks that respect sentence boundaries."""
doc = nlp(text)
sentences = [sent.text.strip() for sent in doc.sents]
chunks = []
current_chunk = []
current_length = 0
for sent in sentences:
sent_length = len(sent)
# If adding this sentence would exceed the limit, finalize current chunk
if current_chunk and current_length + sent_length + 1 > max_chunk_size:
chunks.append(" ".join(current_chunk))
# Keep the last `overlap_sentences` for context continuity
if (
overlap_sentences > 0
and len(current_chunk) >= overlap_sentences
):
current_chunk = current_chunk[-overlap_sentences:]
current_length = (
sum(len(s) for s in current_chunk) + len(current_chunk) - 1
)
else:
current_chunk = []
current_length = 0
current_chunk.append(sent)
current_length += sent_length + (1 if current_length > 0 else 0)
if current_chunk:
chunks.append(" ".join(current_chunk))
return chunkssent_chunks = chunk_by_sentences(
sample_text, max_chunk_size=200, overlap_sentences=0
)Chunk 0: [191 chars] 'The Amazon rainforest produces about 20 percent of the world's oxygen. It spans across nine countries in South America. The forest is home to approximately 10 percent of all species on Earth.' Chunk 1: [168 chars] 'Deforestation has reduced its area significantly over the past decades. Scientists estimate that about 17 percent of the Amazon has been destroyed in the last 50 years.' Chunk 2: [145 chars] 'Conservation efforts are critical to preserving this vital ecosystem. Many indigenous communities depend on the rainforest for their livelihoods.' Chunk 3: [94 chars] 'The Amazon River, which flows through the forest, is the largest river by volume in the world.'
Each chunk now contains complete sentences. The text within each chunk is coherent, making it much easier for an embedding model to capture its meaning. When the model processes a sentence-aligned chunk, it encounters grammatically well-formed text with clear subjects, verbs, and objects, exactly the kind of input it was trained on. This alignment between the structure of training data and the structure of your chunks tends to produce higher-quality embeddings.
The chunk sizes are no longer perfectly uniform, since they depend on sentence lengths, but this is a worthwhile trade-off. A chunk that is 180 characters of coherent prose will almost always produce a better embedding than a chunk that is exactly 200 characters but begins with the fragment "tion efforts are critical to preserving." The embedding model extracts meaning from the semantics of the text, not from its length. Feeding it coherent input produces a coherent, useful embedding.
One practical consideration with sentence-based chunking is that it can produce chunks of highly variable size when sentence lengths vary dramatically. A document alternating between very short sentences ("DNA mutates.") and very long ones could produce chunks that range from 10 characters to 400 characters for the same max_chunk_size setting. This variability rarely causes problems for retrieval quality, but it is worth monitoring if you care about predictable indexing time or storage usage.
Key Parameters
The key parameters for sentence-based chunking are:
- max_chunk_size: The maximum length (in characters or tokens) allowed for a chunk before a split is forced. Sentences are accumulated into the current chunk as long as this limit is not exceeded, so actual chunk sizes will vary depending on where the sentence boundaries fall relative to this ceiling.
- overlap_sentences: The number of full sentences to repeat at the beginning of the next chunk to preserve context. Unlike character or token overlap, sentence overlap guarantees that the shared content is always a complete, meaningful unit. This is particularly valuable when consecutive sentences contain co-references ("This process..." or "These results...") that would be difficult to interpret without the preceding sentence.