Summary
JSON outputs move the model from returning strings to returning typed data. Constrained decoding — the model is forced to emit valid JSON matching a schema — has made this reliable enough to build systems on top of.
Objectives
- 01State what constrained JSON decoding guarantees.
- 02Design a JSON schema for a prompt.
- 03Distinguish provider approaches (OpenAI, Anthropic, Google, local).
The Lesson
The guarantee
With OpenAI's structured outputs or JSON Schema mode, output is guaranteed to parse and match the provided schema. With Anthropic and Google, output is strongly encouraged but not guaranteed — validation and retry logic is required.
Schema design
Prefer flat schemas over nested. Prefer enums over free-text fields. Include descriptions on every field — they serve as inline instructions. Local models (via Outlines, LM Format Enforcer, llama.cpp grammar mode) can also constrain decoding.
Key Ideas
- Constrained JSON decoding is a hard guarantee, not a hope.
- Design schemas as if the model has never seen your domain.