JSON Schema Structured Generation Comparison

Generated on 2026-01-09 with XGrammar 0.1.29, dotjson v0.1.45.

XGrammar: Supported: 27, Unsupported: 1, Unsound: 31
dotjson: Supported: 43, Partial: 1, Unsupported: 14, Unsound: 1

Methodology

This page contrasts JSON schema support for XGrammar, and dotjson. It is auto-generated by a test suite. For each keyword, we have schemas, and tests which are supposed to validate against, or not validate against, those schemas.

We classify support status by a set of rules, interpreted with fallthrough for the tables below, and without for the summary above, to keep the totals consistent.

If no schemas compile => ⛔ Unsupported
If a test case fails => 🔴 Unsound
If less than all schemas compile => 🟡 Partial
If all suites pass => 🟢 Supported

Basic behavior

XGrammardotjson
constraint propagation🔴 Unsound
Details
Schema:
{ "type": "integer", "$defs": { "a": { "maximum": 13 } }, "$ref": "#/$defs/a", "anyOf": [ { "minimum": 10 } ] }

Input:
null
Was expected to be rejected, but wasn't

Input:
9
Was expected to be rejected, but wasn't

Input:
14
Was expected to be rejected, but wasn't
🟢 Supported
unrealizable schema optionality⛔ Unsupported
DetailsCompilation failed.
Schema:
{ "anyOf": [ { "type": "integer", "minimum": 11, "maximum": 10 }, { "type": "boolean" } ] }

Error:
Check failed: (!(start && end) || *start <= *end) is false: Invalid range: minimum greater than maximum
🟢 Supported

    Effectful keywords

    KeywordXGrammardotjson
    $anchor🔴 Unsound
    Details
    Schema:
    { "$defs": { "flag": { "$anchor": "flag", "const": "pass" } }, "$ref": "#flag" }

    Input:
    "fail"
    Was expected to be rejected, but wasn't
    Schema:
    { "$defs": { "a": { "$id": "https://test.com", "$defs": { "flag": { "$anchor": "flag", "const": "pass" } } } }, "$ref": "https://test.com#flag" }

    Input:
    "fail"
    Was expected to be rejected, but wasn't
    		🟢 Supported
    $defs🟢 Supported🟢 Supported
    $id🔴 Unsound
    Details
    Schema:
    { "$id": "https://example.com/my-schema", "$ref": "https://example.com/my-schema#/$defs/helper", "$defs": { "helper": { "type": "null" } } }

    Input:
    1
    Was expected to be rejected, but wasn't
    Schema:
    { "$id": "https://example.com/my-schema", "$ref": "my-helper", "$defs": { "helper": { "$id": "my-helper", "type": "null" } } }

    Input:
    1
    Was expected to be rejected, but wasn't
    Schema:
    { "$id": "https://example.com/my-schema", "$ref": "/my-helper", "$defs": { "helper": { "$id": "my-helper", "type": "null" } } }

    Input:
    1
    Was expected to be rejected, but wasn't
    Schema:
    { "$id": "https://example.com/my-schema", "$ref": "my-schema/../my-helper", "$defs": { "helper": { "$id": "my-helper", "type": "null" } } }

    Input:
    1
    Was expected to be rejected, but wasn't
    Schema:
    { "$id": "https://example.com/my-schema", "$ref": "https://example.com/my-helper", "$defs": { "helper": { "$id": "my-helper", "type": "null" } } }

    Input:
    1
    Was expected to be rejected, but wasn't
    		🟢 Supported
    $ref🟢 Supported🟢 Supported
    $schema🟢 Supported🟢 Supported
    additionalProperties🔴 Unsound
    Details
    Schema:
    { "type": "object", "properties": { "a": { "type": "integer" } }, "additionalProperties": { "type": "boolean" } }

    Input:
    { "a": false }
    Was expected to be rejected, but wasn't
    		🟢 Supported
    allOf🔴 Unsound
    Details
    Schema:
    { "type": "integer", "allOf": [ { "minimum": 3, "maximum": 6 }, { "minimum": 0, "maximum": 4 } ] }

    Input:
    2
    Was expected to be rejected, but wasn't

    Input:
    5
    Was expected to be rejected, but wasn't
    		🟢 Supported
    anyOf🟢 Supported🟢 Supported
    const🔴 Unsound
    Details
    Schema:
    { "const": { "version": 1 } }

    Input:
    { "version": 1 }
    Was expected to be accepted, but wasn't
    		🟢 Supported
    contentEncoding1🟢 Supported1🟢 Supported1
    contentMediaType2🟢 Supported2🟢 Supported2
    contentSchema3🟢 Supported3🟢 Supported3
    enum🔴 Unsound
    Details
    Schema:
    { "enum": [ [ 1 ], { "a": null } ] }

    Input:
    [ 1 ]
    Was expected to be accepted, but wasn't

    Input:
    { "a": null }
    Was expected to be accepted, but wasn't
    		🟢 Supported
    exclusiveMaximum🔴 Unsound / 🟡 Partial
    Details
    Schema:
    { "type": "number", "exclusiveMaximum": 5.3 }

    Input:
    5.3
    Was expected to be rejected, but wasn't

    Input:
    -0.3
    Was expected to be accepted, but wasn't
    Compilation failed.
    Schema:
    { "type": "integer", "exclusiveMaximum": 5.3 }

    Error:
    Check failed: (val == std::floor(val)) is false: Integer constraint must be a whole number
    		🟢 Supported
    exclusiveMinimum🔴 Unsound / 🟡 Partial
    Details
    Schema:
    { "type": "integer", "exclusiveMinimum": -13 }

    Input:
    -12
    Was expected to be accepted, but wasn't
    Schema:
    { "type": "number", "exclusiveMinimum": 5.3 }

    Input:
    5.29
    Was expected to be rejected, but wasn't

    Input:
    5.3
    Was expected to be rejected, but wasn't
    Compilation failed.
    Schema:
    { "type": "integer", "exclusiveMinimum": 5.3 }

    Error:
    Check failed: (val == std::floor(val)) is false: Integer constraint must be a whole number
    		🟢 Supported
    format🟢 Supported🟢 Supported
    items🟢 Supported🟢 Supported
    maxItems🟢 Supported🟢 Supported
    maxLength🟢 Supported🟢 Supported
    maxProperties🟢 Supported🟢 Supported
    maximum🔴 Unsound / 🟡 Partial
    Details
    Schema:
    { "type": "number", "maximum": 5.3 }

    Input:
    -0.3
    Was expected to be accepted, but wasn't
    Compilation failed.
    Schema:
    { "type": "integer", "maximum": 5.3 }

    Error:
    Check failed: (val == std::floor(val)) is false: Integer constraint must be a whole number
    		🟢 Supported
    minItems🟢 Supported🟢 Supported
    minLength🟢 Supported🟢 Supported
    minProperties🟢 Supported🟢 Supported
    minimum🔴 Unsound / 🟡 Partial
    Details
    Schema:
    { "type": "integer", "minimum": -13 }

    Input:
    -12
    Was expected to be accepted, but wasn't

    Input:
    -13
    Was expected to be accepted, but wasn't
    Schema:
    { "type": "number", "minimum": 5.3 }

    Input:
    5.29
    Was expected to be rejected, but wasn't
    Compilation failed.
    Schema:
    { "type": "integer", "minimum": 5.3 }

    Error:
    Check failed: (val == std::floor(val)) is false: Integer constraint must be a whole number
    		🟢 Supported
    pattern🔴 Unsound
    Details
    Schema:
    { "type": "string", "pattern": "^\n$" }

    Input:
    "\n"
    Was expected to be accepted, but wasn't
    Schema:
    { "type": "array", "prefixItems": [ { "type": "string", "pattern": "^\"$" } ] }

    Input:
    [ "\"" ]
    Was expected to be accepted, but wasn't
    		🟢 Supported
    patternProperties🔴 Unsound
    Details
    Schema:
    { "type": "object", "patternProperties": { "^[ab]$": { "type": [ "integer", "null" ] }, "^b|c$": { "type": [ "null", "boolean" ] } } }

    Input:
    { "b": 1 }
    Was expected to be rejected, but wasn't

    Input:
    { "c": null }
    Was expected to be accepted, but wasn't

    Input:
    { "c": true }
    Was expected to be accepted, but wasn't
    		🟢 Supported
    prefixItems🔴 Unsound
    Details
    Schema:
    { "type": "array", "prefixItems": [ { "type": "string" }, { "type": "integer" } ], "items": false }

    Input:
    []
    Was expected to be accepted, but wasn't

    Input:
    [ "id" ]
    Was expected to be accepted, but wasn't
    		🟢 Supported
    properties🟢 Supported🟢 Supported
    propertyNames🔴 Unsound
    Details
    Schema:
    { "type": "object", "propertyNames": { "pattern": "a" } }

    Input:
    { "contains_an_a": 1 }
    Was expected to be accepted, but wasn't
    Schema:
    { "type": "object", "properties": { "a": { "const": "a" }, "c": { "const": "c" } }, "propertyNames": { "pattern": "^[ab]$" }, "additionalProperties": false }

    Input:
    { "a": 3 }
    Was expected to be rejected, but wasn't
    		🟢 Supported
    required🟢 Supported🟢 Supported
    type🟢 Supported🟢 Supported
    multipleOf🔴 Unsound
    Details
    Schema:
    { "type": "integer", "multipleOf": 5 }

    Input:
    11
    Was expected to be rejected, but wasn't
    Schema:
    { "type": "integer", "multipleOf": 3 }

    Input:
    2
    Was expected to be rejected, but wasn't
    Schema:
    { "type": "number", "multipleOf": 2.2 }

    Input:
    3.3
    Was expected to be rejected, but wasn't
    		🟡 Partial
    DetailsCompilation failed.
    Schema:
    { "type": "number", "multipleOf": 2.2 }

    Error:
    Parsing JSON schema failed Caused by: 0: In 'multipleOf' 1: Expected a positive int
    $dynamicAnchor🔴 Unsound
    Details
    Schema:
    { "$defs": { "node": { "$dynamicAnchor": "node", "type": "string", "const": "green" } }, "type": "object", "properties": { "value": { "$dynamicRef": "#node" } }, "required": [ "value" ] }

    Input:
    { "value": "blue" }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "$defs": { "node": { "$dynamicAnchor": "node", "type": "string", "const": "green" } }, "type": "object", "properties": { "value": { "$dynamicRef": "#node" } }, "required": [ "value" ] }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: In 'properties' 2: In key 'value' 3: Unsupported feature: 'dynamicRef'
    $dynamicRef🔴 Unsound
    Details
    Schema:
    { "$defs": { "node": { "$dynamicAnchor": "node", "type": "string", "const": "green" } }, "$dynamicRef": "#node" }

    Input:
    "red"
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "$defs": { "node": { "$dynamicAnchor": "node", "type": "string", "const": "green" } }, "$dynamicRef": "#node" }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'dynamicRef'
    contains🔴 Unsound
    Details
    Schema:
    { "type": "array", "contains": { "type": "integer", "minimum": 2 } }

    Input:
    [ 0, 1 ]
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "array", "contains": { "type": "integer", "minimum": 2 } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'contains'
    dependentRequired🔴 Unsound
    Details
    Schema:
    { "type": "object", "properties": { "a": { "type": "number" }, "b": { "type": "number" } }, "dependentRequired": { "a": [ "b" ] } }

    Input:
    { "a": 1 }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "object", "properties": { "a": { "type": "number" }, "b": { "type": "number" } }, "dependentRequired": { "a": [ "b" ] } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'dependentRequired'
    dependentSchemas🔴 Unsound
    Details
    Schema:
    { "type": "object", "properties": { "a": { "type": "boolean" } }, "dependentSchemas": { "a": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] } } }

    Input:
    { "a": true }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "object", "properties": { "a": { "type": "boolean" } }, "dependentSchemas": { "a": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] } } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'dependentSchemas'
    else🔴 Unsound
    Details
    Schema:
    { "type": "object", "if": { "properties": { "mode": { "const": "strict" } }, "required": [ "mode" ] }, "then": { "required": [ "limit" ], "properties": { "limit": { "minimum": 10 } } }, "else": { "required": [ "fallback" ], "properties": { "fallback": { "const": true } } } }

    Input:
    { "mode": "strict", "limit": 5 }
    Was expected to be rejected, but wasn't

    Input:
    { "mode": "relaxed" }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "object", "if": { "properties": { "mode": { "const": "strict" } }, "required": [ "mode" ] }, "then": { "required": [ "limit" ], "properties": { "limit": { "minimum": 10 } } }, "else": { "required": [ "fallback" ], "properties": { "fallback": { "const": true } } } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'if'
    if🔴 Unsound
    Details
    Schema:
    { "if": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] }, "then": { "required": [ "detail" ] } }

    Input:
    { "flag": true }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "if": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] }, "then": { "required": [ "detail" ] } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'if'
    maxContains🔴 Unsound
    Details
    Schema:
    { "type": "array", "contains": { "type": "number", "minimum": 0 }, "maxContains": 1 }

    Input:
    [ 1, 2 ]
    Was expected to be rejected, but wasn't
    Schema:
    { "type": "array", "prefixItems": [ { "const": "x" }, { "const": "y" } ], "contains": { "const": "x" }, "maxContains": 1 }

    Input:
    []
    Was expected to be accepted, but wasn't

    Input:
    [ "x" ]
    Was expected to be accepted, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "array", "contains": { "type": "number", "minimum": 0 }, "maxContains": 1 }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'contains'
    Compilation failed.
    Schema:
    { "type": "array", "prefixItems": [ { "const": "x" }, { "const": "y" } ], "contains": { "const": "x" }, "maxContains": 1 }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'contains'
    minContains🔴 Unsound
    Details
    Schema:
    { "type": "array", "contains": { "const": "x" }, "minContains": 2 }

    Input:
    [ "x", "y" ]
    Was expected to be rejected, but wasn't
    Schema:
    { "type": "array", "prefixItems": [ { "const": "x" }, { "const": "y" } ], "contains": { "const": "x" }, "minContains": 2 }

    Input:
    [ "x", "y" ]
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "array", "contains": { "const": "x" }, "minContains": 2 }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'contains'
    Compilation failed.
    Schema:
    { "type": "array", "prefixItems": [ { "const": "x" }, { "const": "y" } ], "contains": { "const": "x" }, "minContains": 2 }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'contains'
    not🔴 Unsound
    Details
    Schema:
    { "not": { "type": "string" } }

    Input:
    "bad"
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "not": { "type": "string" } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'not'
    then🔴 Unsound
    Details
    Schema:
    { "type": "object", "if": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] }, "then": { "required": [ "detail" ] } }

    Input:
    { "flag": true }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "object", "if": { "properties": { "flag": { "const": true } }, "required": [ "flag" ] }, "then": { "required": [ "detail" ] } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'if'
    unevaluatedItems🔴 Unsound
    Details
    Schema:
    { "$schema": "https://json-schema.org/draft/2019-09/schema", "type": "array", "unevaluatedItems": { "type": "boolean" }, "anyOf": [ { "type": "array", "items": { "type": "string" } } ] }

    Input:
    [ true, false ]
    Was expected to be accepted, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "array", "additionalItems": { "type": "number" }, "unevaluatedItems": { "type": "string" } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'unevaluatedItems'
    Compilation failed.
    Schema:
    { "type": "array", "prefixItems": [ { "type": "string" } ], "unevaluatedItems": { "type": "integer" } }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'unevaluatedItems'
    Compilation failed.
    Schema:
    { "$schema": "https://json-schema.org/draft/2019-09/schema", "type": "array", "unevaluatedItems": { "type": "boolean" }, "anyOf": [ { "type": "array", "items": { "type": "string" } } ] }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'unevaluatedItems'
    unevaluatedProperties🔴 Unsound
    Details
    Schema:
    { "anyOf": [ { "type": "object", "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": [ "street_address", "city", "state" ] } ], "properties": { "type": { "enum": [ "residential", "business" ] } }, "required": [ "type" ], "unevaluatedProperties": false }

    Input:
    { "street_address": "1600 Pennsylvania Avenue NW", "city": "Washington", "state": "DC", "type": "business", "something that doesn't belong": "hi!" }
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "anyOf": [ { "type": "object", "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": [ "street_address", "city", "state" ] } ], "properties": { "type": { "enum": [ "residential", "business" ] } }, "required": [ "type" ], "unevaluatedProperties": false }

    Error:
    Parsing JSON schema failed Caused by: 0: In 'unevaluatedProperties' 1: 'false' schemas are only supported as part of additionalProperties/items/additionalItems
    uniqueItems🔴 Unsound
    Details
    Schema:
    { "type": "array", "uniqueItems": true }

    Input:
    [ 1, 1 ]
    Was expected to be rejected, but wasn't
    		⛔ Unsupported
    DetailsCompilation failed.
    Schema:
    { "type": "array", "uniqueItems": true }

    Error:
    Failed lowering in JSON Schema Frontend Caused by: 0: Failed processing schema 1: Unsupported feature: 'uniqueItems'
    oneOf4🔴 Unsound4
    Details
    Schema:
    { "oneOf": [ { "type": "integer" }, { "type": "number", "minimum": 0 } ] }

    Input:
    2
    Was expected to be rejected, but wasn't
    		🔴 Unsound4
    Details
    Schema:
    { "oneOf": [ { "type": "integer" }, { "type": "number", "minimum": 0 } ] }

    Input:
    2
    Was expected to be rejected, but wasn't
    1. Content encoding doesn't affect JSON schema validation: source
    2. Content media type doesn't affect JSON schema validation: source
    3. Content schema doesn't affect JSON schema validation: source
    4. Usage of this feature is usually interchangeable with 'anyOf', and while this is often what users want in practice, it's not strictly always correct. We provide an option to disable this behaviour.

    Annotation-only keywords

    KeywordXGrammardotjson
    $comment🟢 Supported🟢 Supported
    $vocabulary🟢 Supported🟢 Supported
    default🟢 Supported🟢 Supported
    deprecated🟢 Supported🟢 Supported
    description🟢 Supported🟢 Supported
    examples🟢 Supported🟢 Supported
    readOnly🟢 Supported🟢 Supported
    title🟢 Supported🟢 Supported
    writeOnly🟢 Supported🟢 Supported