Constrained Decoding › JSON Schema Constraints
Language AI HandbookAll chapters Constrained Decoding › JSON Schema ConstraintsOne section · Sections in this chapter5 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
JSON is the most common structured format for LLM outputs. Schema-constrained generation ensures the model produces valid JSON that conforms to a specified schema.
From Schema to Grammar
A JSON schema can be converted into a grammar that describes all valid JSON documents matching that schema. This grammar then guides generation token by token.
from dataclasses import dataclass
from typing import Any
@dataclass
class JSONSchema:
"""Simplified JSON schema representation."""
type: str
properties: dict[str, "JSONSchema"] = None
required: list[str] = None
items: "JSONSchema" = None
enum: list[Any] = None
def schema_to_valid_prefixes(schema: JSONSchema, current: str) -> set[str]:
"""
Determine valid next characters given schema and current output.
This is a simplified demonstration. Production implementations
use finite state machines compiled from the schema.
"""
current = current.strip()
if schema.type == "object":
if not current:
return {"{"}
if current == "{":
if schema.required:
return {'"'} # Must start a key
return {'"', "}"} # Key or empty object
# ... more cases for object parsing
elif schema.type == "string":
if not current:
return {'"'}
if current.startswith('"') and not current.endswith('"'):
# Inside string - allow any character (simplified)
return set(
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 "'
)
if (
current.startswith('"')
and current.endswith('"')
and len(current) > 1
):
return set() # Complete
elif schema.type == "number":
if not current:
return set("-0123456789")
if all(c in "-0123456789." for c in current):
return set("0123456789.")
elif schema.type == "boolean":
for val in ["true", "false"]:
if val.startswith(current):
if len(current) < len(val):
return {val[len(current)]}
return set()
return set()Valid continuations for a User schema (name: string, age: number):
(empty) → next chars: ['{']
"{" → next chars: ['"']
"{"" → next chars: ['a', 'b', 'c', 'd', 'e']...
"{"name" → next chars: ['"', 'a', 'b', 'c']
"{"name"" → next chars: [':']
"{"name":" → next chars: [' ', '"']The Outlines Library
For production use, the Outlines library provides efficient JSON schema constraints. It compiles schemas into finite state machines that run in parallel with generation.
# Example of how Outlines-style generation works conceptually
# (Actual Outlines usage requires installing the library)
def compile_schema_to_fsm(schema: dict) -> dict:
"""
Compile a JSON schema to a finite state machine.
Returns a mapping from (state, token) -> next_state.
This is a simplified conceptual demonstration.
"""
# In practice, this would build a complete FSM
# tracking position in the JSON structure
states = {
"start": {"valid_tokens": ["{"], "next": "object_open"},
"object_open": {"valid_tokens": ['"'], "next": "key_start"},
"key_start": {
"valid_tokens": list("abcdefghijklmnopqrstuvwxyz"),
"next": "in_key",
},
"in_key": {
"valid_tokens": list('abcdefghijklmnopqrstuvwxyz"'),
"next": "key_end_or_continue",
},
# ... many more states
}
return states
# The FSM approach is much faster than re-parsing at each step
# because state transitions are precomputedFSM-based schema validation (conceptual):
State transitions for JSON object generation:
start --{--> object_open
object_open --"--> key_start
key_start --[a-z]--> in_key
in_key --"--> after_key
after_key --:--> before_value
before_value --{--> nested_object (recursive)
before_value --"--> string_value
before_value --[0-9]--> number_value
Each state knows exactly which tokens are valid next.The FSM approach speeds up this step. Instead of parsing the entire output at each step, we simply look up the current state and get the valid tokens in constant time.