Free playground · no account

Every RAG answer starts with a knife

Before an AI can retrieve knowledge, someone sliced the documents. Slice them yourself — four splitters, live controls — and see why so many RAG systems miss.

More free playgrounds Join the free cohort

What is this, why does it matter, and how do I use this lab?expand

What it is

Chunking is the step where a long document is cut into smaller pieces before being stored in a retrieval system. A language model can only usefully attend to a limited amount of text at once, and a retrieval system can only return whole pieces — so the shape of those pieces decides what any RAG system can ever find. A chunk is the atom of retrieval: if the answer to a question straddles two chunks, no amount of clever search puts it back together.

What this lab makes clear

• Why 'just split every 500 characters' breaks sentences and destroys meaning at the boundaries
• What overlap actually is — the same text deliberately stored in two neighbouring chunks so context survives the cut (the green regions in this lab)
• Why recursive splitting (paragraphs first, then lines, then words) is the industry default: it respects the document's own structure
• Why chunk size is a trade-off: small chunks retrieve precisely but lose context; big chunks carry context but dilute the match and waste the model's window
• Why markdown and code need their own splitters — a function or a section heading is a natural boundary a character counter can't see

Where you meet it in the real world

• Every document Q&A product (Notion AI, Copilot for Docs, support-bot knowledge bases) chunks its corpus exactly like this before embedding
• This platform's own tutor does it: lessons and case studies are split with a recursive splitter (800 chars, 100 overlap) before being embedded into the index you query in the RAG Lab
• Code-search tools chunk by function/class boundaries so a retrieved result is a runnable unit, not half a loop
• Legal/medical AI systems tune chunking per document type — a contract clause and a lab report have very different natural atoms

Why it matters for you as an engineer

Most RAG failures people blame on 'the model' are actually chunking failures: the retriever returned a fragment whose sentence was cut in half, or a chunk so large the relevant line drowned in noise. Being able to look at a document and predict how it will chunk — and what questions will therefore be unanswerable — is a core skill when you build or debug anything with retrieval in it.

How to use this lab

1. Load the Knight Capital case study and keep the default recursive splitter — notice chunks end at paragraph boundaries
2. Switch to the Character splitter and find a sentence sliced mid-word; that fragment is what a retriever would return
3. Drag overlap from 0 upward and watch the green regions appear — that duplicated text is the safety margin across cuts
4. Turn on A/B compare and pit recursive-400/60 against character-400/0 on the same text; compare the histograms
5. Paste one of your own documents (a README, an essay) and find the chunk config that keeps its ideas whole

Retrieval can only ever return whole chunks — so the moment you chunk a document, you've already decided which questions it can answer.

Document

Knight Capital and the $440 million deploy

On 1 August 2012, Knight Capital deployed new trading code to its eight production servers. The deployment was manual. One server was missed.

The new code reused a feature flag that, on the un-updated server, still pointed at an old function called Power Peg — a test routine designed to buy high and sell low to exercise other systems. When the market opened, orders routed to that server triggered the dead code.

In 45 minutes, Knight executed over 4 million unintended trades across 154 stocks, accumulating a position it had to unwind at a loss of about $440 million. The company that processed 10% of US equity volume was insolvent within days and was acquired shortly after.

The engineering lessons are not about trading. There was no automated, repeatable deployment: a human pushed to eight machines and missed one. There was no verification step that all servers ran the same version. A feature flag was recycled, so old code paths could be re-activated silently. And there was no kill switch: the team took 45 minutes to find and stop the behaviour because they had never rehearsed the failure.

Every modern practice this platform teaches — CI/CD pipelines, immutable deployments, config-as-code, canary releases, fast rollback, chaos drills — exists because a company can lose $10 million per minute when releases are manual.

Splitter config

SplitterTries paragraphs, then lines, then words. The sane default.Chunk size400Overlap (max 45%)60

5 chunksavg 325min 187max 400green = overlap

Knight Capital and the $440 million deploy On 1 August 2012, Knight Capital deployed new trading code to its eight production servers. The deployment was manual. One server was missed. The new code reused a feature flag that, on the un-updated server, still pointed at an old function called Power Peg — a test routine designed to buy high and sell low to exercise other systems. When the market opened, orders routed to that server triggered the dead code. In 45 minutes, Knight executed over 4 million unintended trades across 154 stocks, accumulating a position it had to unwind at a loss of about $440 million. The company that processed 10% of US equity volume was insolvent within days and was acquired shortly after. The engineering lessons are not about trading. There was no automated, repeatable deployment: a human pushed to eight machines and missed one. There was no verification step that all servers ran the same version. A feature flag was recycled, so old code paths could be re-activated silently. And there was no kill switch: the team took 45 minutes to find and stop the behaviour because they had never rehearsed the failure. Every modern practice this platform teaches — CI/CD pipelines, immutable deployments, config-as-code, canary releases, fast rollback, chaos drills — exists because a company can lose $10 million per minute when releases are manual.

chunk-size distribution

Why it matters: a RAG system can only retrieve whole chunks. If the answer to a question is split across a chunk boundary with no overlap, retrieval returns a fragment and the model guesses the rest. That's why production pipelines default to recursive splitting with 10–15% overlap — try reproducing that here.

What is chunking in RAG?

Chunking splits documents into retrievable pieces before they are embedded. Retrieval returns chunks, not documents — so chunk boundaries decide what an AI system can find.

Why do RAG systems give incomplete answers?

Often because the fact was split across two chunks, or a chunk lost its surrounding context. Bad chunking looks like a bad model but is really a data problem.

What chunk size should I use?

There is no universal number — it depends on the document structure and the questions asked. That is exactly what this lab lets you feel: drag the size and overlap and watch what breaks.

Deeper dive: RAG, defined properly

Next: dissect a real prompt

Prompt Anatomy walks a production-grade prompt apart with the COSTAR framework — click each piece to see what it does.

Open Prompt Anatomy Join free