Structured outputs constrain the model to emit valid JSON matching a schema. OpenAI guarantees schema conformance; Anthropic and Google strongly encourage but do not guarantee; local models require decoder-level constraint (Outlines, grammars). The compatibility matrix matters.
- 01State each provider's structured-output guarantee level.
- 02Design a schema that works cross-provider.
- 03Handle schema-violation retries.
Provider matrix
OpenAI: json_schema mode, hard guarantee. Anthropic: tool use with JSON schema, strong but not guaranteed — validate. Google: response_mime_type = application/json with responseSchema; similar to Anthropic. Local: llama.cpp grammars, Outlines, LM Format Enforcer for hard constraints.
Cross-provider design
Assume best-effort. Validate outputs against your schema. On failure: retry once with the error message in the prompt; then fall back to a repair prompt (asking the model to fix its own output). Ship the validation and retry logic; do not skip.
- Only OpenAI hard-guarantees schema conformance.
- Always validate; always have a retry-and-repair path.