RAG Text Splitter — Split Text by Tokens for RAG, with Overlap
Input
Chunk Settings
Presets
500 tokens · 50 overlap · Recursive — the general-purpose default for a docs chatbot.
~0 chunks at this size
50-token overlap between adjacent chunks
Splits by paragraphs, then sentences, then raw tokens — keeps paragraphs whole unless they alone exceed the chunk size. Best general-purpose default.
Chunks
Paste text, drop a PDF, or scrape a URL on the left to see RAG-ready chunks here.
Your doc is 40,000 tokens → split into 80 chunks of ~500 tokens for Pinecone → cost per query: ~1,500 tokens (3 chunks retrieved), not 40,000 (96% cheaper).
What is RAG chunking?
Retrieval-augmented generation (RAG) needs small, searchable pieces of a document, not the whole thing. You split a 40,000-token document into roughly 80 chunks of 500 tokens each, embed every chunk, and store the vectors. When a user asks a question, you search for the 3 most relevant chunks — about 1,500 tokens — and send only those to the LLM, instead of the full 40,000-token document. That is the entire point of chunking: it turns a document too big for any context window into a document you never have to send in full at all.
Why chunk by tokens, not characters?
Tokens are the LLM's actual currency — 500 tokens is roughly 2,000 characters and roughly 375 words, but that ratio drifts with punctuation, code, and non-English text, so a character-based or word-based limit is only ever an approximation. Splitting by real token count (the same tiktoken encoder GPT-4o uses) guarantees each chunk fits exactly into an embedding model's input limit and a retrieval budget you can actually plan around.
Best chunk size for RAG
| Use case | Chunk size | Overlap | Strategy | Why |
|---|---|---|---|---|
| RAG Chatbot over docs | 500 tokens | 50-80 tokens | Recursive | General best default — enough context per chunk without diluting the embedding. |
| Code RAG | 300 tokens | 30 tokens | Preserve code blocks | Code needs smaller windows, and a split function is worse than a small chunk. |
| Long Doc QA / Book | 800 tokens | 100 tokens | Semantic | Narrative and manuals need more surrounding context per chunk. |
| Fine-tuning | 500 tokens | 0 tokens | Token-based | Training data wants exact, non-overlapping windows, not retrieval overlap. |
How much overlap for RAG?
80 tokens of overlap on a 500-token chunk is 16% — the sweet spot for most documents. It exists to stop an answer from being lost at a chunk boundary: if chunk 1 ends mid-sentence about a refund policy, chunk 2 starts with the last 80 tokens of chunk 1 repeated, so whichever chunk the retriever returns, the full sentence is inside it. Turn on Merge tiny chunks so a stray 40-token sliver at the end of a section gets folded into its neighbour instead of shipping as a nearly useless chunk of its own — content is appended, never dropped.
Chunking strategies explained
- Recursive Character (LangChain-style) — tries to keep paragraphs together, splitting by blank lines first and falling back to sentence-level splits only when a paragraph alone is bigger than the chunk size. The best general default.
- Token-based — a strict, fixed-size window that may split mid-paragraph. Most precise for exact chunk sizes; best for fine-tuning data.
- Paragraph — fits as many whole paragraphs as possible into a chunk and never splits one unless it alone exceeds the chunk size.
- Two-pass hybrid — split by headers first, then run a token-window pass on any section that still exceeds the chunk size. This tool's Recursive and Paragraph strategies already do this for headed documents when Preserve headings is on.
Best practices for RAG chunking
- Target 300-600 tokens with 80-token overlap — the sweet spot most RAG chunking research converges on.
- Merge tiny chunks under 120 tokens rather than shipping a useless sliver.
- Preserve headings in each chunk's metadata (and text) for better search relevance.
- Keep code blocks whole — never split a fenced ```code``` block mid-function.
- Use overlap at 15-20% of chunk size — 80 tokens for a 500-token chunk.
- Use deterministic chunk IDs (
chunk-1,chunk-2…) for idempotent vector-DB upserts. - Check the Cards view before you export — overlap and chunk boundaries are easy to eyeball, hard to guess from numbers alone.
How RAG chunking works, step by step
A 40,000-token research paper is split at 500 tokens per chunk into roughly 80 chunks. Each chunk is embedded and stored in a vector DB. A user asks “what is the refund window?” — the retriever searches for the 3 most relevant chunks (roughly 1,500 tokens), and only those are sent to the LLM along with the question. The LLM answers from 1,500 tokens of relevant context instead of reading the entire document.
Related tools
- File to Tokens — check if a PDF or DOCX fits before you chunk it.
- URL to Tokens — scrape a page and check its token count first.
- Prompt Compressor — still doesn't fit after compression? Chunk it here instead.
- Token Counter — count tokens for pasted text across nine models.
- Smart Token Reducer — strip whitespace and filler before you chunk.
- Code to LLM — pack a codebase into one prompt, then chunk it here for Code RAG.
- Docs to LLM — convert DOCX, PPTX, XLSX, MD, or HTML to Markdown first.