Structured Outputs

Structured outputs is a generation mode where the model's decoding process is constrained, token by token, to only ever produce text that matches a specified schema — typically JSON Schema — so the response is guaranteed to parse and conform, not merely likely to.

Asking a model nicely to "respond in this exact format" gets you close most of the time and wrong just often enough to break a parser in production: a missing field, an extra field, a string where a number belonged, a trailing comment the model added out of habit. Structured outputs removes the "most of the time" by making the invalid path mechanically unavailable during generation itself.

How it works

A JSON Schema — field names, types, required fields, enums — is compiled into a grammar or finite-state constraint over the model's vocabulary. At every decoding step, the constraint engine computes which tokens would keep the output on a path that can still validate against the schema, and masks every other token's probability to zero before sampling happens. The model is never asked to be well-behaved; it is architecturally prevented from being anything else, because an invalid next token literally is not in the sampling pool.

This is a meaningfully stronger guarantee than instructing a model to output JSON in a prompt, which relies on the model choosing to comply. It is also more expensive to set up: schemas with enums, nested objects, or unusual type combinations need real compilation into the constraint format, and providers differ in which schema features they support under strict guarantees versus which they only encourage through training.

Why it matters

Structured outputs is what makes model output safe to feed directly into a downstream system — a database insert, a function call, an API request — without a validation-and-retry loop wrapped around every single response. Before it, teams built elaborate defenses: regex extraction, retry-on-parse-failure, a second model call just to "clean up" the first one's JSON. Structured outputs replaces that defensive layer with a guarantee made at generation time.

A worked example

Schema: an object with fields "sentiment" (must be one of positive, neutral, negative) and "confidence" (a number between 0 and 1). Without constraint, a model might return sentiment "Positive" and confidence "high" — wrong casing, wrong type — technically readable by a human, unparseable by strict code expecting the schema. With structured outputs enforcing that schema, "Positive" is never reachable: the moment decoding reaches the sentiment field, only the tokens for positive, neutral, or negative have any probability at all, and the confidence field can only ever decode as a valid number in range. The output sentiment "positive," confidence 0.86 is not the model's best effort — it is the only kind of output the schema permits.

How miatz teaches it

Structured outputs is where the prompting module meets the coding reps: learners define a JSON Schema for a real task — ticket triage, entity extraction — turn on schema-constrained decoding, and compare the failure rate against the same task run in plain prompt-only JSON mode, over a fixed eval set, so the guarantee is measured rather than assumed.