System Design Studio

System Design Studio

Wire up an AI request path — cache, embedding, vector store, rerank, generation — then ramp traffic against it and watch it find its breaking point. Instead of a slide that says "add caching," the studio simulates load across the pipeline and lights up the single stage whose throughput ceiling gives out first, naming the exact failure mode: a cache stampede, an embedder 429, a retrieval timeout, or unbounded generation cost. The moat is that it finds and names the breaking node live, on a pipeline you assembled yourself, before any real request exists. Runs entirely in your browser (0 uploads, works offline).

How to use it

1. Toggle stages on or off to shape your pipeline — each node shows its throughput ceiling (req/s) and the latency it adds.
2. Set the incoming traffic with the ramp slider, or load a preset — Steady, Spike, or Adversarial burst.
3. Watch each node's utilisation bar fill; the first stage to cross 100% is flagged as the bottleneck and the verdict tells you whether the design holds or breaks.
4. Open the X-ray panel to read the exact failure mode of the breaking node — then unplug or reinforce it and re-run to see the ceiling move.

What this clears up (the fundamentals)

• Capacity is the weakest link, not the average — a pipeline can only serve as many req/s as its lowest-ceiling stage. The bottleneck is always the enabled stage with the smallest maxRps, no matter how fast everything else is.
• The bottleneck moves when you change the design — unplug the generation node and the ceiling jumps to the next-tightest stage. Optimising anything but the bottleneck buys you nothing.
• Failures have names, not vibes — "it'll fall over under load" becomes a specific mode: cache stampede, quota 429, retrieval timeout, unbounded cost. Naming it turns a worry into a next action.
• Latency and throughput are different axes — the generation node can be your throughput ceiling and your biggest latency add at the same time; the studio shows both so you don't conflate them.

Where it's used

A top-of-funnel AI-engineering architecture check — the fastest way to feel why "it works in the demo" and "it survives 500 req/min" are different claims. It's a miatz build-lab concept playable: play the simulation here, then learn to build the traffic engine and the first-failure detector yourself.

FAQ

How does it decide which stage breaks first?

Every stage has a throughput ceiling (maxRps). The bottleneck is the enabled stage with the lowest ceiling; if your incoming traffic exceeds that number, the pipeline breaks there first. It's a deterministic rule — no model call, no randomness.

Why does removing a node sometimes make the pipeline stronger?

Because capacity is set by the tightest ceiling. If the stage you remove was the bottleneck, the ceiling rises to the next-lowest stage, so the same traffic now holds. The studio recomputes the bottleneck every time you change the wiring.

What do the failure modes mean?

Each node fails in a characteristic way when overrun — a cache stampede when a miss storm floods the backend, a 429 when an endpoint rate-limits, a timeout when retrieval can't answer in budget, unbounded cost when generation has no ceiling. The X-ray names the one that applies to your breaking node.

Is this a real load test against my infrastructure?

No. It's a seeded simulation with fixed per-stage ceilings — a teaching model of how load finds a bottleneck, not a benchmark of any specific deployment. There is no server, no infra, and no cost.

Is anything uploaded?

No. Your pipeline and traffic settings are simulated entirely in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.

Limits

This is a seeded demo: the five stages carry hand-set throughput ceilings and latencies to teach how a bottleneck emerges, not to measure your real system. Actual ceilings depend on your models, hardware, batch sizes, and network, and real pipelines fail in messier, overlapping ways. Live inference against your own architecture — routing a real model call through your key with a cost meter — is a separate BYOK mode, not this teaching simulation. The lesson it teaches (capacity is the weakest link; name the failure before you ship it) is exactly the real practice.

Related

Part of the miatz playgrounds. Explore the rest from the playground catalog.

Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.