schemagen: $defs and definitions are merged into one namespace, so a $ref can silently resolve to the wrong definition #75

New Issue

2026-11-07T16:57:23Z

weaselbot commented

2026-11-07 16:57:23 +00:00

Summary

contrib/schemagen/weaseljson_schemagen.py copies both root containers of
named definitions ($defs and definitions) into a single flat dict and then
looks definitions up only by their last path segment. The container named
in the $ref is validated and then thrown away, so #/$defs/X and
#/definitions/X are treated as the same definition.

That is not what JSON Schema says: $defs and definitions are two
independent locations. When both containers exist and contain an entry with
the same name, one silently shadows the other, and the generator emits the
wrong C++ type / wrong validation for the property. The reverse case is
equally wrong in the other direction: a $ref to a definition that does not
exist in the container it names is silently accepted instead of being reported
as an unknown $ref target.

schemagen exits 0 with no diagnostic in every case below; the mistake is only
visible in the generated header's types and in the parser's accept/reject
behaviour.

Root cause

contrib/schemagen/weaseljson_schemagen.py, Builder.__init__ (lines 241-245):

class Builder:
    def __init__(self, root_schema):
        self.root_schema = root_schema
        self.defs = {}
        for key in ("$defs", "definitions"):
            self.defs.update(root_schema.get(key, {}))   # <-- merged, `definitions` wins

Builder.ref_name() (lines 293-299) parses the container out of the pointer but
returns only the last segment:

    def ref_name(self, ref):
        ...
        parts = ref[2:].split("/")
        if len(parts) != 2 or parts[0] not in ("$defs", "definitions"):
            raise GenError(f"unsupported $ref target: {ref}")
        return parts[1]                                  # <-- container is dropped

and Builder.build_def() (lines 301-308) resolves the bare name against the
merged dict:

        if defname not in self.defs:
            raise GenError(f"$ref to unknown def: {defname}")
        node = self.defs[defname]

self._building (the $ref-cycle cache, line 248) is keyed the same way, so it
has the same aliasing problem.

Reproduction 1 — the generated type is the other container's type

schema.json:

{
  "$defs":       { "X": { "type": "string" } },
  "definitions": { "X": { "type": "integer" } },
  "type": "object",
  "additionalProperties": false,
  "required": ["a"],
  "properties": { "a": { "$ref": "#/$defs/X" } }
}

python3 contrib/schemagen/weaseljson_schemagen.py schema.json -o gen.h --namespace ns
echo $?                       # 0, no diagnostic
sed -n '/^struct Root1 {/,/^};/p' gen.h

Actual (the definitions entry is used, not the $defs one that the $ref
names):

struct Root1 {
  int64_t a = 0;  // "a"
};

Expected: std::optional<std::string> a; ($defs/X is {"type":"string"}).

Because the member is an int64_t, the generated parser now enforces the wrong
schema (RootBuilder::cbNumberData / cbStringData in the generated builder):

$ printf '{"a":"hi"}' | ./demo   # "$defs/X" says string -> should be OK
2                                # WeaselJson_REJECT  (schema-valid document rejected)
$ printf '{"a":1}' | ./demo      # integer is not allowed by "$defs/X"
0                                # WeaselJson_OK      (schema-invalid document accepted)

A document that is valid against the schema is rejected, and one that is
invalid is accepted — with no warning at generation time.

Reproduction 2 — an unresolvable `$ref` is silently accepted

schema2.json (Z exists only under definitions, but the property points at
$defs):

{
  "definitions": { "Z": { "type": "integer" } },
  "type": "object",
  "additionalProperties": false,
  "required": ["b"],
  "properties": { "b": { "$ref": "#/$defs/Z" } }
}

$ python3 contrib/schemagen/weaseljson_schemagen.py schema2.json -o gen2.h --namespace ns
$ echo $?
0
$ sed -n '/^struct Root1 {/,/^};/p' gen2.h
struct Root1 {
  int64_t b = 0;  // "b"
};

#/$defs/Z does not exist, so this should be rejected the same way a genuinely
missing name is:

$ printf '%s' '{"type":"object","additionalProperties":false,"properties":{"c":{"$ref":"#/$defs/Nope"}}}' > schema3.json
$ python3 contrib/schemagen/weaseljson_schemagen.py schema3.json -o gen3.h
error: $ref to unknown def: Nope
$ echo $?
1

The same silent aliasing happens for definitions-only schemas referenced
through #/$defs/...; $defs-only schemas referenced through
#/definitions/... resolve too.

Impact

The generated header has the wrong member type, so downstream code silently
gets a value of the wrong type (or does not compile, if the two definitions
map to unrelated structs/enums).
The generated parser validates documents against a different definition than
the one the schema names — schema-valid documents are rejected
(WeaselJson_REJECT) and schema-invalid ones accepted (WeaselJson_OK),
which defeats the point of generating a validating parser.
A $ref typo/dangling reference (#/definitions/... left over from a
draft-07 schema while the definition lives in $defs) is not diagnosed even
though schemagen otherwise rejects unknown $ref targets.
Only schemas that keep definitions in both containers (e.g. a schema migrated
from draft-07 to 2020-12 that gained a $defs) are affected, which is
exactly when the aliasing is hardest to notice.

Suggested fix

Keep the two containers separate (or key everything by
(container, name)): have ref_name() return the container together with the
name, and use it for both the lookup in build_def() and the self._building
cycle cache, so #/$defs/X can only ever resolve $defs/X and
#/definitions/X can only resolve definitions/X. A $ref to a name that is
absent from the container it names should then fall through to the existing
$ref to unknown def GenError.

Test gap

contrib/schemagen/test_schemagen.py (and the *.schema.json files) only ever
use one of the two containers, so nothing exercises a schema that has both, or
a $ref that names the container the definition is not in.

## Summary `contrib/schemagen/weaseljson_schemagen.py` copies **both** root containers of named definitions (`$defs` and `definitions`) into a single flat dict and then looks definitions up **only by their last path segment**. The container named in the `$ref` is validated and then thrown away, so `#/$defs/X` and `#/definitions/X` are treated as the same definition. That is not what JSON Schema says: `$defs` and `definitions` are two independent locations. When both containers exist and contain an entry with the same name, one silently shadows the other, and the generator emits the **wrong C++ type / wrong validation** for the property. The reverse case is equally wrong in the other direction: a `$ref` to a definition that does not exist in the container it names is silently accepted instead of being reported as an unknown `$ref` target. schemagen exits 0 with no diagnostic in every case below; the mistake is only visible in the generated header's types and in the parser's accept/reject behaviour. ## Root cause `contrib/schemagen/weaseljson_schemagen.py`, `Builder.__init__` (lines 241-245): ```python class Builder: def __init__(self, root_schema): self.root_schema = root_schema self.defs = {} for key in ("$defs", "definitions"): self.defs.update(root_schema.get(key, {})) # <-- merged, `definitions` wins ``` `Builder.ref_name()` (lines 293-299) parses the container out of the pointer but returns only the last segment: ```python def ref_name(self, ref): ... parts = ref[2:].split("/") if len(parts) != 2 or parts[0] not in ("$defs", "definitions"): raise GenError(f"unsupported $ref target: {ref}") return parts[1] # <-- container is dropped ``` and `Builder.build_def()` (lines 301-308) resolves the bare name against the merged dict: ```python if defname not in self.defs: raise GenError(f"$ref to unknown def: {defname}") node = self.defs[defname] ``` `self._building` (the `$ref`-cycle cache, line 248) is keyed the same way, so it has the same aliasing problem. ## Reproduction 1 — the generated type is the *other* container's type `schema.json`: ```json { "$defs": { "X": { "type": "string" } }, "definitions": { "X": { "type": "integer" } }, "type": "object", "additionalProperties": false, "required": ["a"], "properties": { "a": { "$ref": "#/$defs/X" } } } ``` ```sh python3 contrib/schemagen/weaseljson_schemagen.py schema.json -o gen.h --namespace ns echo $? # 0, no diagnostic sed -n '/^struct Root1 {/,/^};/p' gen.h ``` Actual (the `definitions` entry is used, not the `$defs` one that the `$ref` names): ```cpp struct Root1 { int64_t a = 0; // "a" }; ``` Expected: `std::optional<std::string> a;` (`$defs/X` is `{"type":"string"}`). Because the member is an `int64_t`, the generated parser now enforces the wrong schema (`RootBuilder::cbNumberData` / `cbStringData` in the generated builder): ```console $ printf '{"a":"hi"}' | ./demo # "$defs/X" says string -> should be OK 2 # WeaselJson_REJECT (schema-valid document rejected) $ printf '{"a":1}' | ./demo # integer is not allowed by "$defs/X" 0 # WeaselJson_OK (schema-invalid document accepted) ``` A document that is valid against the schema is rejected, and one that is invalid is accepted — with no warning at generation time. ## Reproduction 2 — an unresolvable `$ref` is silently accepted `schema2.json` (`Z` exists only under `definitions`, but the property points at `$defs`): ```json { "definitions": { "Z": { "type": "integer" } }, "type": "object", "additionalProperties": false, "required": ["b"], "properties": { "b": { "$ref": "#/$defs/Z" } } } ``` ```console $ python3 contrib/schemagen/weaseljson_schemagen.py schema2.json -o gen2.h --namespace ns $ echo $? 0 $ sed -n '/^struct Root1 {/,/^};/p' gen2.h struct Root1 { int64_t b = 0; // "b" }; ``` `#/$defs/Z` does not exist, so this should be rejected the same way a genuinely missing name is: ```console $ printf '%s' '{"type":"object","additionalProperties":false,"properties":{"c":{"$ref":"#/$defs/Nope"}}}' > schema3.json $ python3 contrib/schemagen/weaseljson_schemagen.py schema3.json -o gen3.h error: $ref to unknown def: Nope $ echo $? 1 ``` The same silent aliasing happens for `definitions`-only schemas referenced through `#/$defs/...`; `$defs`-only schemas referenced through `#/definitions/...` resolve too. ## Impact * The generated header has the wrong member type, so downstream code silently gets a value of the wrong type (or does not compile, if the two definitions map to unrelated structs/enums). * The generated parser validates documents against a different definition than the one the schema names — schema-valid documents are rejected (`WeaselJson_REJECT`) and schema-invalid ones accepted (`WeaselJson_OK`), which defeats the point of generating a validating parser. * A `$ref` typo/dangling reference (`#/definitions/...` left over from a draft-07 schema while the definition lives in `$defs`) is not diagnosed even though `schemagen` otherwise rejects unknown `$ref` targets. * Only schemas that keep definitions in both containers (e.g. a schema migrated from draft-07 to 2020-12 that gained a `$defs`) are affected, which is exactly when the aliasing is hardest to notice. ## Suggested fix Keep the two containers separate (or key everything by `(container, name)`): have `ref_name()` return the container together with the name, and use it for both the lookup in `build_def()` and the `self._building` cycle cache, so `#/$defs/X` can only ever resolve `$defs/X` and `#/definitions/X` can only resolve `definitions/X`. A `$ref` to a name that is absent from the container it names should then fall through to the existing `$ref to unknown def` `GenError`. ## Test gap `contrib/schemagen/test_schemagen.py` (and the `*.schema.json` files) only ever use one of the two containers, so nothing exercises a schema that has both, or a `$ref` that names the container the definition is not in.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: weaselab/weaseljson#75

schemagen: $defs and definitions are merged into one namespace, so a $ref can silently resolve to the wrong definition #75