Chunking

What is Chunking?

Chunking is the preprocessing step that determines RAG quality. Too large and the embedding is diluted; too small and you lose context. Strategies include fixed-size, sentence-boundary, semantic, and hierarchical chunking. The right choice depends on document type, embedding model, and downstream task. See RAG Technology Best Practices.

What is an example of Chunking?

For policy manuals, a pipeline can split by headings and clauses, keep a small overlap, attach the policy version and access group, and preserve a parent-section link so the answer can cite the full context.

How is Chunking different from related concepts?

Chunking divides and labels source content. Tokenization divides text into model-readable units. A chunk contains many tokens and is designed around retrieval meaning, not only model syntax.

What should enterprises evaluate for Chunking?

• Create representative questions and measure whether the correct passage appears in the top retrieval results.
• Compare structure-aware, semantic, and fixed-size strategies on the actual document types in scope.
• Prevent tables, clauses, code blocks, and headings from being separated in ways that destroy meaning.

Related terms

• RAG (Retrieval-Augmented Generation)
• Embeddings