Chunk Overlap

Chunk overlap is the amount of text repeated between one retrieval chunk and the next when a document is split for indexing. Rather than cutting a document into strictly back-to-back, non-overlapping pieces, most fixed-size and recursive splitters let each chunk end by re-including the last portion of the previous chunk, so a fact, sentence, or clause that happens to fall right on a chunk boundary still shows up whole in at least one of the two chunks that straddle it. It’s a deliberately blunt fix for a problem every fixed-length splitter has: cutting at a fixed character or token count has no idea where a sentence, an idea, or a cause-and-effect pair actually ends, so without overlap, the exact information a query is looking for can be split in half between two separately-retrieved chunks, neither of which contains the complete answer.

The Problem Overlap Solves

Picture a 500-character chunk boundary landing in the middle of the sentence “the warranty covers manufacturing defects but not damage caused by water exposure.” If the cut falls right after “not damage caused by,” one chunk ends with an incomplete clause and the next chunk opens mid-thought with “water exposure,” disconnected from what it modifies. A retriever that pulls only the first chunk will surface a sentence that looks like it says the warranty has no exceptions; a retriever that pulls only the second has a dangling fragment with no subject. Chunk overlap addresses this directly: if the splitter carries the last 75 characters of chunk one forward into the start of chunk two, the full sentence appears intact in chunk two even though the boundary technically falls elsewhere. It doesn’t prevent the boundary from happening (that’s what semantic chunking tries to do); it hedges against the boundary landing somewhere important by duplicating the risky zone into both neighbors.

How Overlap Is Set in Practice

Every mainstream fixed-size or recursive splitter exposes overlap as an explicit parameter measured in the same unit as chunk size (characters or tokens).

LangChain’s RecursiveCharacterTextSplitter takes a chunk_overlap argument alongside chunk_size. The splitter tries a list of separators in order (paragraph breaks, then line breaks, then sentence-ending punctuation, then spaces, then raw characters) to keep cuts as clean as possible, and after finding a cut point, it walks backward by chunk_overlap characters to decide where the next chunk should start.

# Scenario: a long onboarding doc where step-by-step instructions
# often span more than one paragraph, so context needs to carry across chunks
from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=120,  # 15% overlap: enough to catch a straddling sentence, not much more
)
chunks = splitter.split_text(onboarding_doc)
# chunks[1] will start by repeating the tail ~120 characters of chunks[0]

LlamaIndex’s SentenceSplitter takes the equivalent chunk_overlap parameter, measured in tokens rather than characters, and is sentence-aware: it tries to keep whole sentences together even inside the target chunk_size, using overlap as a secondary safety net rather than the primary mechanism for preserving context.

# Scenario: indexing a support-ticket knowledge base for a RAG assistant,
# where chunk_size and chunk_overlap are both measured in tokens, not characters
from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=512, chunk_overlap=64)  # 64/512 = 12.5% overlap
nodes = splitter.get_nodes_from_documents(documents)

Both libraries apply the same overlap value uniformly across every chunk boundary in a document; neither adjusts overlap per boundary based on how “risky” that particular cut looks; that kind of content-aware boundary decision is what semantic chunking is for.

The Core Trade-off

Overlap has exactly one dial, and it pulls in two directions at once.

Too little overlap (or none at all) means a fact or sentence that falls on a boundary is genuinely lost: it exists in the source document but never appears complete in any single retrieved chunk, so no amount of retrieval or reranking downstream can recover it. This is the failure mode overlap exists to prevent.

Too much overlap creates a different set of problems:

• Index bloat. If every chunk repeats a large fraction of its neighbor, the total number of chunk-tokens indexed grows well past the size of the source document. At the extreme (overlap approaching chunk size), the same sentence can appear in three, four, or more chunks.
• Redundant embeddings. Every one of those near-duplicate chunks needs its own embedding computed and stored, which is pure wasted spend on both the embedding API calls at ingest time and the storage/memory footprint of the vector index.
• Retrieval noise from near-duplicates. When several nearly-identical chunks compete for the same top-k slots in a similarity search, they crowd out chunks that would have added genuinely different information to the retrieved context, so a query can come back with three redundant restatements of one passage instead of a diverse set of relevant ones.

# Scenario: quantifying index bloat before committing to an overlap setting,
# the kind of check worth running once per corpus rather than guessing
def indexed_chunk_tokens(source_len, chunk_size, chunk_overlap):
    stride = chunk_size - chunk_overlap
    if stride <= 0:
        raise ValueError("chunk_overlap must be smaller than chunk_size")
    n_chunks = max(1, -(-source_len // stride))  # ceiling division
    return n_chunks * chunk_size

for overlap in (0, 50, 100, 200, 400):
    total = indexed_chunk_tokens(source_len=5000, chunk_size=500, chunk_overlap=overlap)
    print(f"overlap={overlap:>4} -> {total} indexed tokens ({total / 5000:.1f}x source length)")

Typical Overlap Ratios

There’s no formula that derives the “correct” overlap from first principles; it’s a heuristic tuned by convention and empirical testing. In practice, teams commonly land on an overlap of roughly 10-20% of chunk size: enough to catch a sentence or clause straddling a boundary without meaningfully inflating the index. A 500-token chunk with 50-100 tokens of overlap, or LlamaIndex’s own commonly-cited defaults in that same range, are typical starting points. Very technical or densely cross-referential text (legal contracts, medical literature, tightly-argued technical documentation) sometimes justifies pushing overlap toward the higher end of that range, since a single sentence there is more likely to be load-bearing and expensive to lose; loosely structured or highly redundant text (FAQs, transcripts with lots of filler) can often get away with less. The LlamaIndex team’s own published chunk-size evaluation work treats chunk size and overlap as a pair to sweep together against a retrieval eval set, rather than picking either in isolation, which is the same advice that applies to any specific overlap ratio quoted here: it is a reasonable starting point, not a substitute for measuring retrieval quality on the actual corpus.

Chunk Overlap vs. Semantic Chunking

graph LR
    classDef default fill:#ffffff,stroke:#4338CA,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef data    fill:#EEF0F7,stroke:#0D9488,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef process fill:#F7F8FC,stroke:#6366F1,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef output  fill:#4338CA,stroke:#4338CA,stroke-width:2px,color:#ffffff,rx:8px,ry:8px;

    DOC([Document]):::data --> CO[Fixed-size split +<br/>duplicate boundary window]:::process --> COUT[Overlapping chunks:<br/>boundary loss hedged by redundancy]:::output
    DOC --> SC[Cut only where topic<br/>similarity drops]:::process --> SCOUT[Non-overlapping chunks:<br/>boundary loss avoided by placement]:::output

Chunk overlap and semantic chunking are two different answers to the same underlying problem: information sitting at a chunk boundary getting lost or fragmented. Overlap is the brute-force answer, spend extra index size to make sure the risky zone around every boundary is covered twice, regardless of whether that particular boundary was actually risky. Semantic chunking is the precision answer: spend extra compute at ingest time to place boundaries only where the content actually shifts topic, so there’s less boundary-straddling information to lose in the first place. They aren’t mutually exclusive. A pipeline can run semantic chunking to get topically coherent chunk boundaries and still apply a small overlap on top as a cheap second line of defense against any boundary the similarity signal missed; in fact, most production semantic-chunking configurations do exactly this rather than relying on the similarity threshold alone.

What’s New (2025-2026)

• Token-based overlap over character-based. As more pipelines standardize on token-counted chunk sizes to match embedding model context limits precisely, chunk_overlap is increasingly specified and reasoned about in tokens (as LlamaIndex’s SentenceSplitter already does) rather than raw characters, since token count is what actually determines embedding cost and context-window fit.
• Overlap as one axis of a broader chunk-size sweep, not a standalone setting. Following the pattern LlamaIndex popularized in its own chunk-size evaluation work, more teams now treat chunk size and chunk overlap as a pair to grid-search together against a retrieval eval set, rather than picking a single “recommended” overlap ratio and moving on.
• Larger embedding-model context limits have reduced, but not eliminated, the pressure on overlap. As embedding models comfortably handle longer inputs, some pipelines use larger base chunk sizes with proportionally smaller overlap ratios, since a boundary inside a longer chunk is statistically less likely to fall on a critical sentence than a boundary inside a short one.

Chunk overlap remains the cheapest, most widely deployed fix for boundary information loss precisely because it requires no extra model calls and works with any fixed-size splitter out of the box. It is a hedge, not a guarantee: a large enough overlap makes boundary loss unlikely rather than impossible, and pushing it too far trades that safety margin for index bloat and retrieval noise, which is why picking a ratio (commonly 10-20% of chunk size) and validating it against real retrieval queries matters more than any single “correct” default.

from langchain_text_splitters import RecursiveCharacterTextSplitter

# A knowledge-base article where key facts often sit right at
# the boundary a fixed-size splitter would otherwise cut through
splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=75,   # 15% of chunk_size: a typical ratio in practice
    separators=["\n\n", "\n", ". ", " ", ""],
)

chunks = splitter.split_text(open("article.md").read())
print(f"{len(chunks)} chunks, avg {sum(len(c) for c in chunks) / len(chunks):.0f} chars")