Cost & Latency Bench

Cost & Latency Bench

Describe one model call — token shape, per-million prices, per-request latency — then drag a single production-scale slider and watch a demo's pennies turn into a monthly line item. The bench computes cost per request, per day and per month, the throughput ceiling one stream can hold, and the moat: the cliff multiple between a 10-request/day demo and the scale you actually plan to ship. It turns the invisible unit economics of AI into a number you can argue about in a budget review. Runs entirely in your browser (0 uploads, works offline).

How to use it

1. Pick a model tier (Fast / Balanced / Frontier) to set illustrative prices and latency, or load a preset — Cheap demo, Startup, or Production scale.
2. Set the token shape of one request (input and output tokens) and, if you know real numbers, the exact per-million prices and per-request latency.
3. Drag the 1000× scale slider from a 10-req/day demo up toward 1M/day and watch the monthly bill and verdict climb.
4. Open the X-ray panel to see the cliff laid out — demo bill vs. production bill and the exact multiple between them — then copy or share the number.

What this clears up (the fundamentals)

• A cheap demo is not a cheap product. Cost per request is a fraction of a cent; multiply by real volume and 30 days and the same call becomes a four- or five-figure monthly bill. The demo hides the cliff — scale reveals it.
• The bill is tokens × price ÷ 1M. Input and output tokens are priced separately (output usually costs more), so a chatty completion or a bloated prompt moves the number as much as the request count does.
• Latency sets throughput, not price. One stream serves 60000 ÷ latency requests per minute; hitting a target volume is a parallelism problem, and slow-but-cheap can cost more in workers than fast-but-pricey.
• Routing and caching bend both curves. Sending the cheapest model that clears your quality bar, and caching repeat calls, is how the cliff flattens without dropping the feature.

Where it's used

A top-of-funnel AI-engineering cost-literacy check — the fastest way to feel why "it was basically free in testing" stops being true in production. It's a miatz build-lab concept playable: play the cliff here, then learn to build the cost meter, the routing-under-a-quality-floor logic, and the extrapolation reveal yourself in the AI Engineering module.

FAQ

How is the cost calculated?

Per request, cost is (inputTokens × pricePerMInput + outputTokens × pricePerMOutput) ÷ 1,000,000. That is multiplied by your requests/day for a daily cost and by 30 for a monthly cost. It is a deterministic formula — no model call, no randomness, no network.

What is the "cliff multiple"?

Your monthly bill at the chosen scale divided by the monthly bill at a fixed 10-request/day demo. It answers "how many times bigger is my real bill than the demo that convinced everyone this was cheap?" — often hundreds or thousands of times.

Are the model prices real?

No — the three tiers ship illustrative teaching numbers, not a live price sheet, so the tool stays fully offline and never phones a vendor. Type your provider's exact per-million prices into the fields to bench your real workload.

Is anything uploaded?

No. Everything is computed in your browser from the numbers you enter — nothing is transmitted, stored, or logged, and no API key is ever needed. Turn off your Wi-Fi and it still works.

Limits

A teaching bench with flat assumptions: a 30-day month, one price pair, and a single-stream throughput read. Real bills also carry batching discounts, cached-input pricing, rate limits, retries, and mixed traffic. Treat the output as an honest order-of-magnitude — enough to spot a cliff before you ship past it, not a finance-grade forecast.

Related

Part of the miatz playgrounds. Explore the rest from the playground catalog.

Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.