The Guardrail Builder
Everyone agrees AI needs guardrails — but writing a check and writing a check that actually catches things are two different skills. Here you author a real guardrail (block keywords, plus an optional PII switch), fire it at a battery of labelled messages — a prompt injection, a leaked API key, an email, a phone number, an SSN, and a few harmless ones — and watch exactly which it catches, which it lets slip, and which it wrongly flags. A precision / recall / F1 score tells you how good your rule really is. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Write the block keywords — comma or newline separated — that your guardrail should stop on (case-insensitive substrings).
- Optionally tick block PII to also catch any message containing an email, phone number, or SSN.
- Hit Stress-test it and read the per-message verdict: caught the bad one, let it slip through, or wrongly flagged a good one — plus your robustness score.
What this clears up (the fundamentals)
- A keyword list is not a guardrail — it catches the phrasings you thought of and misses the ones you didn't. An attacker only has to word it differently once.
- Precision and recall pull against each other — widen your rule to catch more attacks (higher recall) and you start blocking legitimate requests (lower precision). The score makes that trade-off visible.
- Fluent-but-wrong is the hard case — a message that reads perfectly reasonable but should be blocked is where naive rules fail, and where a real eval earns its keep.
- Guardrails are earned by attacking your own rule — the reveal is the gap between "I wrote a check" and "my check actually catches things." You close it by throwing adversarial samples at your own guardrail until it holds.
Where it's used
A top-of-funnel AI-safety reflex builder: the fastest way to feel why a guardrail needs testing, not just writing. It's a miatz build-lab concept playable — the same mechanic reappears inside the Agent Builder, MCP Playground, and Evals Runner labs, where you learn to build and harden guardrails yourself.
FAQ
What is an AI guardrail?
A check that sits around a model to catch unsafe or unwanted input or output — a prompt-injection filter, an output-schema validator, a PII redactor, a tool-scope restriction, or a pass/fail rubric. This playground models the simplest kind: a keyword and PII block rule you can test against known-good and known-bad messages.
Why isn't a keyword blocklist enough?
Because natural language has endless phrasings. "Ignore previous instructions" is easy to block; a paraphrase you didn't list walks straight through. Blocklists also over-fire — a broad word like "weather" can flag perfectly safe requests. That is exactly the precision/recall tension the score exposes.
What do precision, recall, and F1 mean here?
Recall is the share of should-block messages your rule actually caught. Precision is the share of your blocks that were correct (not false alarms). F1 is their balance — a single number that punishes a rule that only does well on one of the two.
Is anything uploaded?
No. The message battery, your rule, and the scoring all run entirely in your browser — nothing is transmitted, logged, or stored. Turn off your Wi-Fi and it still works.
Limits
A teaching model with a fixed battery of labelled messages and a substring/PII rule engine — real guardrails span schema validation, tool-scope restriction, semantic classifiers, and live adversarial testing against an actual model. The reflex it trains — write a rule, then attack it until it holds — transfers directly to production guardrail work.
Related
Part of the miatz playgrounds. Explore the rest from the playground catalog.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.