11 Commits
Author SHA1 Message Date
weaselbot df693ef4c9 schemagen: reject scalar root values when root schema is object/array
The generated RootBuilder crashed (undefined behavior on std::vector::back())
when a JSON document's root value was a scalar or null while the schema
declared a non-nullable object or array root. The stack starts empty for
object/array roots, but cbStringData, cbNumberData, and cbBool called
stack_.back() without checking for an empty stack.

Add an empty-stack guard to the three scalar callbacks so they reject
instead of crashing. cbNull already handles the empty-stack case.

Regression tests added for:
- non-nullable object root rejecting null, boolean, number, and string roots
- nullable object root rejecting scalar roots
- nullable array root rejecting scalar roots

Closes #16
2026-06-24 14:30:36 -04:00
andrew e5cbaac401 Merge pull request 'schemagen: cache non-object $defs entries to avoid duplicate types' (#31) from weaselbot/weaseljson:weaselbot/issue-17 into main
Reviewed-on: weaselab/weaseljson#31
2026-06-24 17:02:38 +00:00
weaselbot e155e0bbf4 schemagen: restore nullable tuple return for object schemas
The previous change to cache object definitions started returning the
bare TObj from build_type for object schemas, discarding the nullable
flag. This caused nullable root objects to be emitted as plain structs
instead of std::optional<RootInner>, breaking the nullable root tests.

Return the (TObj, nullable) tuple so callers (including the root
emitter) see the correct nullability again.
2026-06-24 09:51:19 -04:00
weaselbot 717f30099f schemagen: cache non-object $defs entries to avoid duplicate types
Extend the existing per-definition cache (`self._building`) to enum,
array, and scalar $defs, not just object definitions. This ensures
that multiple $refs to the same non-object definition reuse the same
C++ type instead of generating Role, Role2, Role3, etc.

- Cache the built (type, nullable) tuple under defname for enum,
  scalar, and array definitions.
- Pre-register array definitions before recursing into items so $ref
  cycles resolve to the same TArr instance.
- Store object definitions as (TObj, nullable) tuples so nullable object
  $defs also preserve their nullability when referenced.

Add a regression test for issue #17 covering reused enum and array-of-enum
$defs.
2026-06-23 22:16:34 -04:00
andrew 1404ebdfbd Merge pull request 'fix schemagen integer parsing boundary bugs' (#30) from weaselbot/weaseljson:weaselbot/issue-19 into main
Reviewed-on: weaselab/weaseljson#30
2026-06-24 00:40:09 +00:00
weaselbot 38079cc278 schemagen: parse integer fallback exactly for decimal/exponent forms
Replace the double-based fallback in generated integer slots with a
string-to-int64 parser that handles decimal points and exponents
without losing precision near the int64 boundaries.

The old path used std::from_chars<double> and compared against
±9223372036854775808.0, which rounds the int64 max and min so that
valid values are rejected and out-of-range negatives are accepted.

The new helper:
- Parses sign, integer part, optional fraction, and optional exponent.
- Strips trailing zeros to cancel fractional places.
- Rejects non-integral values and overflow using exact uint64_t
  arithmetic.

Adds regression tests covering root integer and object-field integer
boundary values, including the cases from issue #19.
2026-06-23 18:36:53 -04:00
andrew ceb4bc1041 Merge pull request 'schemagen: drop support for additionalProperties: true' (#28) from weaselbot/weaseljson:weaselbot/issue-20 into main
Reviewed-on: weaselab/weaseljson#28
2026-06-23 20:56:58 +00:00
weaselbot 16f13c241c schemagen: drop support for additionalProperties: true
Following review feedback, the generator no longer supports permissive
objects. Changes:

- Reject `additionalProperties: true` at generation time.
- Treat an absent `additionalProperties` as `false`, so every object is
  strict by default and unknown keys are rejected during parsing.
- Remove the now-dead permissive-object infrastructure: `Kind::Skip`,
  `Cat::Skip`, `kSkip`, the per-frame `unknown` key set, and `isStrict()`.
- Update the README feature/rejection tables accordingly.
- Remove the permissive "loose" object from example.schema.json and the
  associated tests from test_gen.cpp.
- Add Python unit tests verifying the new `additionalProperties` behavior.

All tests pass (`ctest --output-on-failure`).
2026-06-23 15:11:48 -04:00
weaselbot ab95fefb09 schemagen: reject duplicate unknown keys in non-strict objects
Track unknown keys in a per-object unordered_set so that permissive
objects (additionalProperties absent/true) still reject duplicate keys,
matching the README guarantee.

- Add std::unordered_set<std::string> to Frame.
- Insert unknown keys in cbKeyData and reject duplicates before skipping.
- Add a permissive "loose" subobject to example.schema.json.
- Test single unknown key accepted and duplicate unknown/known keys rejected.
2026-06-23 15:08:48 -04:00
andrew e8830e27e9 Merge pull request 'schemagen: reserve generated Root/Skip/RootScalar and avoid Kind/ArrN collisions' (#27) from weaselbot/weaseljson:weaselbot/issue-21 into main
Reviewed-on: weaselab/weaseljson#27
2026-06-23 17:48:57 +00:00
weaselbot 46ff8e2164 schemagen: reserve generated Root/Skip/RootScalar and avoid Kind/ArrN collisions
Make the type-name allocator aware of the identifiers the generator emits
itself (`Root` alias, `Skip`/`RootScalar` Kind enumerators) so user `$defs`
names can no longer collide with them.  Array-kind names (`Arr0`, `Arr1`, ...)
are now allocated only after checking for object/enum names, preventing
duplicate `Kind` enumerators when a schema defines e.g. `Arr0`.

Add Python regression tests that also syntax-check the generated headers
with a C++ compiler.

Closes #21
2026-06-23 12:23:09 -04:00
5 changed files with 583 additions and 76 deletions
+5 -5
View File
@@ -45,7 +45,7 @@ into the result, so it is non-movable.
| `$ref` to `$defs`/`definitions` | the referenced named struct | | `$ref` to `$defs`/`definitions` | the referenced named struct |
| recursive `$ref` | `std::unique_ptr<T>` (cycle broken) | | recursive `$ref` | `std::unique_ptr<T>` (cycle broken) |
| `additionalProperties: false` | unknown keys rejected | | `additionalProperties: false` | unknown keys rejected |
| `additionalProperties` absent / `true` | unknown keys' values skipped | | `additionalProperties` absent / `true` | not supported (rejected at generation) |
## Schema violations (rejected at parse time) ## Schema violations (rejected at parse time)
@@ -55,15 +55,15 @@ into the result, so it is non-movable.
- value not in a string `enum` - value not in a string `enum`
- a number not representable in the target type (e.g. `1.5` for an `integer`) - a number not representable in the target type (e.g. `1.5` for an `integer`)
- duplicate object keys - duplicate object keys
- unknown key under `additionalProperties: false` - unknown key in any object
## Not supported (rejected at generation time, no fallback) ## Not supported (rejected at generation time, no fallback)
`oneOf` / `anyOf` / `allOf` / `not` / `if`-`then`-`else`, `oneOf` / `anyOf` / `allOf` / `not` / `if`-`then`-`else`,
`patternProperties`, `additionalProperties` with a schema (typed map), `patternProperties`, `additionalProperties` with a schema (typed map),
`prefixItems` (tuples), `const`, `dependentSchemas`/`dependentRequired`, `additionalProperties: true`, `prefixItems` (tuples), `const`,
union `type` lists other than `["T", "null"]`, non-string enums, and remote `dependentSchemas`/`dependentRequired`, union `type` lists other than
(`$ref` to other documents). `["T", "null"]`, non-string enums, and remote (`$ref` to other documents).
## Notes ## Notes
+4
View File
@@ -184,6 +184,10 @@ int main() {
expectReject(R"({"name":"x","age":1,"address":{"zip":5}})", expectReject(R"({"name":"x","age":1,"address":{"zip":5}})",
"missing required nested 'city'"); "missing required nested 'city'");
expectReject(R"([1,2,3])", "array where object expected (root)"); expectReject(R"([1,2,3])", "array where object expected (root)");
expectReject("null", "null where object expected (root)");
expectReject("true", "boolean where object expected (root)");
expectReject("123", "number where object expected (root)");
expectReject(R"("hi")", "string where object expected (root)");
expectReject(R"({"name":"x","age":1,)", "truncated / invalid json"); expectReject(R"({"name":"x","age":1,)", "truncated / invalid json");
if (failures == 0) { if (failures == 0) {
+40
View File
@@ -61,6 +61,18 @@ static void expectReject(nullable_object::RootBuilder &b, std::string in,
} }
} }
static void expectReject(nullable_array::RootBuilder &b, std::string in,
const char *what) {
WeaselJsonStatus s = parseStrided(b, in);
if (s == WeaselJson_REJECT) {
printf("ok reject: %s\n", what);
} else {
printf("FAIL expected reject (%s) got status %d for: %s\n", what, s,
in.c_str());
++failures;
}
}
int main() { int main() {
// ---- nullable root object: valid document ---- // ---- nullable root object: valid document ----
{ {
@@ -149,6 +161,34 @@ int main() {
} }
} }
// ---- nullable root object: scalar values rejected ----
{
nullable_object::RootBuilder b;
expectReject(b, "true", "boolean where nullable object expected (root)");
}
{
nullable_object::RootBuilder b;
expectReject(b, "123", "number where nullable object expected (root)");
}
{
nullable_object::RootBuilder b;
expectReject(b, R"("hi")", "string where nullable object expected (root)");
}
// ---- nullable root array: scalar values rejected ----
{
nullable_array::RootBuilder b;
expectReject(b, "true", "boolean where nullable array expected (root)");
}
{
nullable_array::RootBuilder b;
expectReject(b, "123", "number where nullable array expected (root)");
}
{
nullable_array::RootBuilder b;
expectReject(b, R"("hi")", "string where nullable array expected (root)");
}
if (failures == 0) { if (failures == 0) {
printf("\nALL TESTS PASSED\n"); printf("\nALL TESTS PASSED\n");
return 0; return 0;
+382
View File
@@ -3,9 +3,12 @@
import json import json
import os import os
import re
import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import textwrap
import unittest import unittest
SCRIPT = os.path.join(os.path.dirname(__file__), "weaseljson_schemagen.py") SCRIPT = os.path.join(os.path.dirname(__file__), "weaseljson_schemagen.py")
@@ -87,5 +90,384 @@ class SchemagenKeywordTest(unittest.TestCase):
self.assertNotIn(f"std::optional<std::string> {kw};", stdout) self.assertNotIn(f"std::optional<std::string> {kw};", stdout)
class SchemagenCollisionTest(unittest.TestCase):
"""Regression tests for issue #21: generated Root alias / Kind enum collisions."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.compiler = shutil.which("c++")
def generate_and_compile(self, schema):
"""Run schemagen on schema and syntax-check the resulting header."""
with tempfile.TemporaryDirectory() as tmpdir:
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
cmd = [
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
]
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
self.assertEqual(result.returncode, 0, msg=result.stderr)
if self.compiler:
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(
'#include "gen.h"\n'
"int main() {\n"
" test_schema::RootBuilder b;\n"
" test_schema::Root r = b.take();\n"
" (void)r;\n"
"}\n"
)
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-fsyntax-only",
"-I",
self.include_dir,
"-I",
tmpdir,
cpp_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
with open(header_path) as fp:
return fp.read()
def test_root_alias_does_not_collide_with_user_type(self):
schema = {
"type": "array",
"items": {"$ref": "#/$defs/Root"},
"$defs": {"Root": {"enum": ["a", "b"]}},
}
out = self.generate_and_compile(schema)
self.assertIn("enum class Root1 : int { a, b };", out)
self.assertIn("using Root = std::vector<Root1>;", out)
self.assertNotIn("using Root = std::vector<Root>;", out)
def test_kind_enum_does_not_duplicate_arr0(self):
schema = {
"type": "object",
"properties": {
"arr": {"type": "array", "items": {"type": "string"}},
"obj": {"$ref": "#/$defs/Arr0"},
},
"$defs": {"Arr0": {"type": "object", "properties": {}}},
}
out = self.generate_and_compile(schema)
self.assertIn("struct Arr0", out)
m = re.search(r"enum class Kind : uint8_t \{([^}]+)\}", out)
self.assertIsNotNone(m)
enumerators = [e.strip() for e in m.group(1).split(",")]
self.assertIn("Arr0", enumerators)
self.assertIn("Arr1", enumerators)
self.assertEqual(len(enumerators), len(set(enumerators)))
def test_skip_user_type_is_allowed(self):
schema = {
"type": "array",
"items": {"$ref": "#/$defs/Skip"},
"$defs": {"Skip": {"type": "object", "properties": {}}},
}
out = self.generate_and_compile(schema)
self.assertIn("struct Skip", out)
self.assertNotIn("struct Skip1", out)
m = re.search(r"enum class Kind : uint8_t \{([^}]+)\}", out)
self.assertIsNotNone(m)
enumerators = [e.strip() for e in m.group(1).split(",")]
self.assertIn("Skip", enumerators)
self.assertIn("Arr0", enumerators)
self.assertEqual(len(enumerators), len(set(enumerators)))
def test_non_object_defs_reused_across_refs(self):
"""Regression test for issue #17: enum and array $defs must be reused."""
schema = {
"type": "object",
"properties": {
"role1": {"$ref": "#/$defs/Role"},
"role2": {"$ref": "#/$defs/Role"},
"roles1": {"$ref": "#/$defs/Roles"},
"roles2": {"$ref": "#/$defs/Roles"},
},
"$defs": {
"Role": {"enum": ["admin", "user"]},
"Roles": {"type": "array", "items": {"$ref": "#/$defs/Role"}},
},
}
out = self.generate_and_compile(schema)
# Exactly one Role enum is generated.
self.assertIn("enum class Role : int { admin, user };", out)
self.assertNotIn("enum class Role1 : int", out)
self.assertNotIn("enum class Role2 : int", out)
# The array of enum is represented by a single kind.
m = re.search(r"enum class Kind : uint8_t \{([^}]+)\}", out)
self.assertIsNotNone(m)
enumerators = [e.strip() for e in m.group(1).split(",")]
self.assertEqual(enumerators.count("Arr0"), 1)
# All four fields use the same C++ types.
self.assertIn("std::optional<Role> role1;", out)
self.assertIn("std::optional<Role> role2;", out)
self.assertIn("std::optional<std::vector<Role>> roles1;", out)
self.assertIn("std::optional<std::vector<Role>> roles2;", out)
class SchemagenAdditionalPropertiesTest(unittest.TestCase):
def run_schemagen(self, schema, args=None):
"""Run schemagen on a schema dict. Returns (returncode, stdout, stderr)."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fp:
json.dump(schema, fp)
schema_path = fp.name
try:
cmd = [sys.executable, SCRIPT, schema_path]
if args:
cmd.extend(args)
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
return result.returncode, result.stdout, result.stderr
finally:
os.unlink(schema_path)
def test_additional_properties_true_rejected(self):
schema = {
"type": "object",
"additionalProperties": True,
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertNotEqual(rc, 0)
self.assertIn("additionalProperties: true is not supported", stderr)
def test_additional_properties_absent_defaults_to_strict(self):
schema = {
"type": "object",
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertEqual(rc, 0, msg=stderr)
# The generated parser should reject unknown keys. Verify the key-matching
# helper returns -1 for an unknown key and cbKeyData rejects it.
self.assertIn("int matchKey(Kind k, std::string_view key) const {", stdout)
self.assertNotIn("bool isStrict(Kind k) const", stdout)
def test_additional_properties_false_accepted(self):
schema = {
"type": "object",
"additionalProperties": False,
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertEqual(rc, 0, msg=stderr)
class SchemagenIntegerBoundaryTest(unittest.TestCase):
"""Regression tests for issue #19: integer slot parsing near int64 boundaries."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.lib_src = os.path.join(self.repo_root, "src", "lib.cpp")
self.compiler = shutil.which("c++")
def _compile_harness(self, tmpdir, schema, harness):
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
result = subprocess.run(
[
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(result.returncode, 0, msg=result.stderr)
lib_obj = os.path.join(tmpdir, "lib.o")
comp_lib = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
os.path.join(self.repo_root, "third_party", "include"),
"-I",
os.path.join(self.repo_root, "third_party", "valgrind"),
"-c",
self.lib_src,
"-o",
lib_obj,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp_lib.returncode, 0, msg=comp_lib.stderr)
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(harness)
exe_path = os.path.join(tmpdir, "test")
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
tmpdir,
lib_obj,
cpp_path,
"-o",
exe_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
run = subprocess.run([exe_path], capture_output=True, text=True, check=False)
self.assertEqual(run.returncode, 0, msg=run.stdout + run.stderr)
def _build_cases_array(self, name, entries):
lines = [
f" struct {name}Case {{ const char *s; WeaselJsonStatus expected; long long v; }};"
]
lines.append(f" {name}Case {name}_cases[] = {{")
for s, exp, v in entries:
val = f"{v}LL" if isinstance(v, int) else v
lines.append(f' {{ R"({s})", {exp}, {val} }},')
lines.append(" };")
return "\n".join(lines)
def test_integer_boundary_root(self):
if not self.compiler:
self.skipTest("C++ compiler not available")
cases = [
("9223372036854775807.0", "WeaselJson_OK", 9223372036854775807),
("9223372036854775807", "WeaselJson_OK", 9223372036854775807),
("9223372036854775806.0", "WeaselJson_OK", 9223372036854775806),
("9223372036854775808", "WeaselJson_REJECT", 0),
("-9223372036854775808", "WeaselJson_OK", "-9223372036854775807LL - 1"),
("-9223372036854775808.0", "WeaselJson_OK", "-9223372036854775807LL - 1"),
("-9223372036854775809", "WeaselJson_REJECT", 0),
("1e3", "WeaselJson_OK", 1000),
("2.0", "WeaselJson_OK", 2),
("0.001", "WeaselJson_REJECT", 0),
("1e-3", "WeaselJson_REJECT", 0),
("1000e-3", "WeaselJson_OK", 1),
("100.0e-2", "WeaselJson_OK", 1),
("123.0", "WeaselJson_OK", 123),
("9e18", "WeaselJson_OK", 9000000000000000000),
("10e18", "WeaselJson_REJECT", 0),
]
cases_src = self._build_cases_array("root", cases)
harness = textwrap.dedent(
f"""
#include "gen.h"
#include <cstdio>
#include <cstring>
{cases_src}
int main() {{
for (const auto \u0026c : root_cases) {{
test_schema::RootBuilder b;
char buf[512];
std::strncpy(buf, c.s, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\\0';
WeaselJsonStatus st = b.feed(buf, std::strlen(buf));
st = b.finish();
if (st != c.expected) {{
std::printf("root case %s expected %d got %d\\n", c.s, c.expected, st);
return 1;
}}
if (st == WeaselJson_OK \u0026\u0026 b.take() != c.v) {{
std::printf("root case %s value mismatch\\n", c.s);
return 2;
}}
}}
return 0;
}}
"""
)
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, {"type": "integer"}, harness)
def test_integer_boundary_object_field(self):
if not self.compiler:
self.skipTest("C++ compiler not available")
cases = [
('{"age":9223372036854775807.0}', "WeaselJson_OK", 9223372036854775807),
('{"age":-9223372036854775809}', "WeaselJson_REJECT", 0),
('{"age":1e3}', "WeaselJson_OK", 1000),
('{"age":2.0}', "WeaselJson_OK", 2),
('{"age":0.001}', "WeaselJson_REJECT", 0),
(
'{"age":-9223372036854775808.0}',
"WeaselJson_OK",
"-9223372036854775807LL - 1",
),
]
cases_src = self._build_cases_array("object", cases)
harness = textwrap.dedent(
f"""
#include "gen.h"
#include <cstdio>
#include <cstring>
{cases_src}
int main() {{
for (const auto \u0026c : object_cases) {{
test_schema::RootBuilder b;
char buf[512];
std::strncpy(buf, c.s, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\\0';
WeaselJsonStatus st = b.feed(buf, std::strlen(buf));
st = b.finish();
if (st != c.expected) {{
std::printf("object case %s expected %d got %d\\n", c.s, c.expected, st);
return 3;
}}
if (st == WeaselJson_OK \u0026\u0026 b.take().age != c.v) {{
std::printf("object case %s value mismatch\\n", c.s);
return 4;
}}
}}
return 0;
}}
"""
)
schema = {
"type": "object",
"properties": {"age": {"type": "integer"}},
"required": ["age"],
}
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, schema, harness)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+152 -71
View File
@@ -60,7 +60,6 @@ class ObjectType:
def __init__(self, name): def __init__(self, name):
self.name = name self.name = name
self.fields = [] # list[Field] self.fields = [] # list[Field]
self.strict = False # additionalProperties: false
class EnumType: class EnumType:
@@ -216,12 +215,26 @@ class Builder:
base = camel(hint) base = camel(hint)
name = base name = base
i = 1 i = 1
while name in self._used_names: while self._name_taken(name):
candidate = f"{base}{i}"
# A numeric suffix can itself land on a reserved generated name
# (e.g. hint "Arr0" -> "Arr01"). Use an underscore separator so
# we never loop through the reserved block.
if self._is_reserved(candidate):
candidate = f"{base}_{i}"
name = candidate
i += 1 i += 1
name = f"{base}{i}"
self._used_names.add(name) self._used_names.add(name)
return name return name
def _name_taken(self, name):
return name in self._used_names or self._is_reserved(name)
@staticmethod
def _is_reserved(name):
"""Names generated internally that must not collide with user types."""
return name in ("Root", "RootScalar")
def ref_name(self, ref): def ref_name(self, ref):
if not ref.startswith("#/"): if not ref.startswith("#/"):
raise GenError(f"only local $ref supported, got: {ref}") raise GenError(f"only local $ref supported, got: {ref}")
@@ -237,7 +250,6 @@ class Builder:
if defname not in self.defs: if defname not in self.defs:
raise GenError(f"$ref to unknown def: {defname}") raise GenError(f"$ref to unknown def: {defname}")
node = self.defs[defname] node = self.defs[defname]
# For objects we must register the name before recursing into fields.
return self.build_type(node, defname, defname=defname) return self.build_type(node, defname, defname=defname)
def build_type(self, node, hint, defname=None): def build_type(self, node, hint, defname=None):
@@ -266,6 +278,10 @@ class Builder:
if "$ref" in node: if "$ref" in node:
return self.build_def(self.ref_name(node["$ref"])) return self.build_def(self.ref_name(node["$ref"]))
# Non-object $defs entries must reuse the same type for every $ref.
if defname is not None and defname in self._building:
return self._building[defname]
# nullability via type lists: ["string", "null"] # nullability via type lists: ["string", "null"]
nullable = False nullable = False
typ = node.get("type") typ = node.get("type")
@@ -284,18 +300,29 @@ class Builder:
raise GenError("only non-empty string enums are supported") raise GenError("only non-empty string enums are supported")
name = defname and self.unique_name(defname) or self.unique_name(hint) name = defname and self.unique_name(defname) or self.unique_name(hint)
self.enums[name] = EnumType(name, list(vals)) self.enums[name] = EnumType(name, list(vals))
return (TEnum(name), nullable) result = (TEnum(name), nullable)
if defname is not None:
self._building[defname] = result
return result
if typ == "object" or (typ is None and "properties" in node): if typ == "object" or (typ is None and "properties" in node):
return (self._build_object(node, hint, defname), nullable) return (self._build_object(node, hint, defname, nullable), nullable)
if typ == "array": if typ == "array":
if "items" not in node or not isinstance(node["items"], dict): if "items" not in node or not isinstance(node["items"], dict):
raise GenError("arrays require a single 'items' schema") raise GenError("arrays require a single 'items' schema")
# Register the array before building its items so $ref cycles back
# to this definition resolve to the same TArr instance.
t = TArr(None, False)
result = (t, nullable)
if defname is not None:
self._building[defname] = result
elem, elem_nullable = self._unpack( elem, elem_nullable = self._unpack(
self.build_type(node["items"], hint + "Item") self.build_type(node["items"], hint + "Item")
) )
return (TArr(elem, elem_nullable), nullable) t.elem = elem
t.elem_nullable = elem_nullable
return result
scalar = { scalar = {
"string": "str", "string": "str",
@@ -304,7 +331,10 @@ class Builder:
"boolean": "bool", "boolean": "bool",
}.get(typ) }.get(typ)
if scalar: if scalar:
return (TScalar(scalar), nullable) result = (TScalar(scalar), nullable)
if defname is not None:
self._building[defname] = result
return result
if typ == "null": if typ == "null":
raise GenError("'null'-only types are not supported") raise GenError("'null'-only types are not supported")
@@ -318,19 +348,21 @@ class Builder:
return result return result
return (result, False) return (result, False)
def _build_object(self, node, hint, defname): def _build_object(self, node, hint, defname, nullable=False):
name = self.unique_name(defname or hint) name = self.unique_name(defname or hint)
obj = ObjectType(name) obj = ObjectType(name)
self.objects[name] = obj self.objects[name] = obj
# register for $ref cycles before building fields # register for $ref cycles before building fields
tobj = TObj(name)
if defname is not None: if defname is not None:
self._building[defname] = TObj(name) self._building[defname] = (tobj, nullable)
ap = node.get("additionalProperties", True) ap = node.get("additionalProperties", False)
if ap is True:
raise GenError("additionalProperties: true is not supported")
if isinstance(ap, dict): if isinstance(ap, dict):
raise GenError( raise GenError(
"additionalProperties with a schema (typed map) is not " "supported yet" "additionalProperties with a schema (typed map) is not " "supported yet"
) )
obj.strict = ap is False
required = set(node.get("required", [])) required = set(node.get("required", []))
props = node.get("properties", {}) props = node.get("properties", {})
seen_cpp = set() seen_cpp = set()
@@ -344,7 +376,7 @@ class Builder:
cpp = f"{base}{i}" cpp = f"{base}{i}"
seen_cpp.add(cpp) seen_cpp.add(cpp)
obj.fields.append(Field(key, cpp, ty, key in required, nullable)) obj.fields.append(Field(key, cpp, ty, key in required, nullable))
return TObj(name) return tobj
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -435,6 +467,7 @@ class Emitter:
self.kind_order = [] # all Kind enumerators in declaration order self.kind_order = [] # all Kind enumerators in declaration order
self.root_ty = None self.root_ty = None
self.root_nullable = False self.root_nullable = False
self._arr_counter = 0
# -- type strings ------------------------------------------------------- # -- type strings -------------------------------------------------------
def base_cpp(self, ty): def base_cpp(self, ty):
@@ -465,11 +498,19 @@ class Emitter:
def arr_kind(self, tarr): def arr_kind(self, tarr):
sig = self.base_cpp(tarr) sig = self.base_cpp(tarr)
if sig not in self.arr_kinds: if sig not in self.arr_kinds:
name = f"Arr{len(self.arr_kinds)}" name = self._fresh_arr_kind_name()
self.arr_kinds[sig] = name self.arr_kinds[sig] = name
self.arr_types.append((name, tarr)) self.arr_types.append((name, tarr))
self.b._used_names.add(name)
return self.arr_kinds[sig] return self.arr_kinds[sig]
def _fresh_arr_kind_name(self):
while True:
name = f"Arr{self._arr_counter}"
self._arr_counter += 1
if not self.b._name_taken(name):
return name
def cat(self, ty): def cat(self, ty):
if isinstance(ty, TScalar): if isinstance(ty, TScalar):
return {"str": "Str", "int": "Int", "dbl": "Dbl", "bool": "Bool"}[ty.kind] return {"str": "Str", "int": "Int", "dbl": "Dbl", "bool": "Bool"}[ty.kind]
@@ -597,7 +638,6 @@ namespace {ns} {{"""
root_is_container = isinstance(self.root_ty, (TObj, TArr)) root_is_container = isinstance(self.root_ty, (TObj, TArr))
if not root_is_container: if not root_is_container:
kinds.append("RootScalar") kinds.append("RootScalar")
kinds.append("Skip")
self.kind_order = kinds self.kind_order = kinds
return " enum class Kind : uint8_t { " + ", ".join(kinds) + " };" return " enum class Kind : uint8_t { " + ", ".join(kinds) + " };"
@@ -786,17 +826,6 @@ namespace {ns} {{"""
lines.append(" }") lines.append(" }")
return "\n".join(lines) return "\n".join(lines)
def _is_strict(self):
strict = [n for n, o in self.b.objects.items() if o.strict]
if not strict:
return " bool isStrict(Kind) const { return false; }"
cases = " ".join(f"case Kind::{n}:" for n in strict)
return (
" bool isStrict(Kind k) const {\n"
f" switch (k) {{ {cases} return true; default: return false; }}\n"
" }"
)
def _enum_name_arrays(self): def _enum_name_arrays(self):
out = [] out = []
for e in self.b.enums.values(): for e in self.b.enums.values():
@@ -838,13 +867,7 @@ namespace {ns} {{"""
else: else:
lines.append(" if (stack_.empty()) { reject(); return; }") lines.append(" if (stack_.empty()) { reject(); return; }")
lines.append(" Frame &f = stack_.back();") lines.append(" Frame &f = stack_.back();")
lines.append(
" if (f.kind == Kind::Skip) { stack_.push_back(Frame{Kind::Skip, nullptr}); return; }"
)
lines.append(" SlotInfo si = slotInfoG(f);") lines.append(" SlotInfo si = slotInfoG(f);")
lines.append(
" if (si.cat == Cat::Skip) { stack_.push_back(Frame{Kind::Skip, nullptr}); return; }"
)
lines.append(f" if (si.cat != Cat::{event_cat}) {{ reject(); return; }}") lines.append(f" if (si.cat != Cat::{event_cat}) {{ reject(); return; }}")
lines.append(" void *p = engage(f, true);") lines.append(" void *p = engage(f, true);")
lines.append(" stack_.push_back(Frame{si.child, p});") lines.append(" stack_.push_back(Frame{si.child, p});")
@@ -895,7 +918,7 @@ public:
private: private:
{kind_enum} {kind_enum}
enum class Cat {{ Reject, Str, Int, Dbl, Bool, Enum, Obj, Arr, Skip }}; enum class Cat {{ Reject, Str, Int, Dbl, Bool, Enum, Obj, Arr }};
struct SlotInfo {{ struct SlotInfo {{
Cat cat; Cat cat;
Kind child{{}}; Kind child{{}};
@@ -906,11 +929,10 @@ private:
struct Frame {{ struct Frame {{
Kind kind; Kind kind;
void *dest; void *dest;
int field = -1; // object: selected field (-1 want key, -2 skip) int field = -1; // object: selected field (-1 means waiting for key)
std::vector<uint64_t> seen; // populated field bitset (object frames) std::vector<uint64_t> seen; // populated field bitset (object frames)
}}; }};
static constexpr int kWantKey = -1; static constexpr int kWantKey = -1;
static constexpr int kSkip = -2;
{self._enum_name_arrays()} {self._enum_name_arrays()}
@@ -925,10 +947,93 @@ private:
void reject() {{ error_ = true; }} void reject() {{ error_ = true; }}
// Wrap the generated slotInfo() with the generic key/skip states. // Parse a JSON number text as int64_t. Accepts optional decimal point and
// exponent only when the mathematical value is an integer that fits in a
// signed 64-bit range.
static bool parseJsonInt64(const char *b, const char *e, int64_t &out) {{
const char *p = b;
bool neg = false;
if (p < e) {{
if (*p == '-') {{ neg = true; ++p; }}
else if (*p == '+') return false;
}}
const char *intStart = p;
while (p < e && *p >= '0' && *p <= '9') ++p;
const char *intEnd = p;
int fracDigits = 0;
const char *fracStart = p;
if (p < e && *p == '.') {{
++p;
fracStart = p;
while (p < e && *p >= '0' && *p <= '9') {{ ++p; ++fracDigits; }}
if (fracStart == p) return false;
}}
int64_t exp = 0;
bool expNeg = false;
if (p < e && (*p == 'e' || *p == 'E')) {{
++p;
if (p < e && (*p == '-' || *p == '+')) {{ expNeg = (*p == '-'); ++p; }}
if (p == e || *p < '0' || *p > '9') return false;
while (p < e && *p >= '0' && *p <= '9') {{
int digit = *p - '0';
if (exp <= (INT64_MAX - digit) / 10) exp = exp * 10 + digit;
else exp = INT64_MAX;
++p;
}}
if (expNeg) exp = -exp;
}}
if (p != e) return false;
if (intStart == intEnd) return false;
std::string digits;
digits.reserve((intEnd - intStart) + fracDigits);
for (const char *q = intStart; q < intEnd; ++q) digits.push_back(*q);
for (int i = 0; i < fracDigits; ++i) digits.push_back(fracStart[i]);
int64_t trim = 0;
while (!digits.empty() && digits.back() == '0') {{
digits.pop_back();
++trim;
}}
int64_t finalExp = exp - fracDigits + trim;
if (finalExp < 0) return false;
while (!digits.empty() && digits.front() == '0') digits.erase(digits.begin());
if (digits.empty()) {{ out = 0; return true; }}
constexpr uint64_t kMaxNeg = 9223372036854775808ULL;
constexpr uint64_t kMaxPos = 9223372036854775807ULL;
const uint64_t limit = neg ? kMaxNeg : kMaxPos;
if (finalExp > 19) return false;
int64_t maxSig = 19 - finalExp;
uint64_t mag = 0;
int64_t sigDigits = 0;
for (char ch : digits) {{
uint64_t d = static_cast<uint64_t>(ch - '0');
if (sigDigits >= maxSig) return false;
if (mag > (limit - d) / 10) return false;
mag = mag * 10 + d;
++sigDigits;
}}
for (int64_t i = 0; i < finalExp; ++i) {{
if (mag > limit / 10) return false;
mag *= 10;
}}
if (mag > limit) return false;
if (neg) {{
if (mag == kMaxNeg) {{
out = INT64_MIN;
}} else {{
out = -static_cast<int64_t>(mag);
}}
}} else {{
out = static_cast<int64_t>(mag);
}}
return true;
}}
// Wrap the generated slotInfo() with the generic key state.
SlotInfo slotInfoG(const Frame &f) {{ SlotInfo slotInfoG(const Frame &f) {{
if (isObjectKind(f.kind)) {{ if (isObjectKind(f.kind)) {{
if (f.field == kSkip) return SlotInfo{{Cat::Skip}};
if (f.field < 0) return SlotInfo{{Cat::Reject}}; if (f.field < 0) return SlotInfo{{Cat::Reject}};
}} }}
return slotInfo(f); return slotInfo(f);
@@ -953,11 +1058,6 @@ private:
void cbEndObject() {{ void cbEndObject() {{
if (error_) return; if (error_) return;
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) {{
stack_.pop_back();
if (stack_.empty() || stack_.back().kind != Kind::Skip) valueComplete();
return;
}}
if (isObjectKind(f.kind)) {{ if (isObjectKind(f.kind)) {{
const auto &req = requiredMask(f.kind); const auto &req = requiredMask(f.kind);
bool missing = false; bool missing = false;
@@ -975,11 +1075,6 @@ private:
void cbEndArray() {{ void cbEndArray() {{
if (error_) return; if (error_) return;
Frame f = stack_.back(); Frame f = stack_.back();
if (f.kind == Kind::Skip) {{
stack_.pop_back();
if (stack_.empty() || stack_.back().kind != Kind::Skip) valueComplete();
return;
}}
stack_.pop_back(); stack_.pop_back();
valueComplete(); valueComplete();
}} }}
@@ -991,12 +1086,10 @@ private:
scratch_.append(buf, len); scratch_.append(buf, len);
if (!done) return; if (!done) return;
int idx = matchKey(f.kind, scratch_); int idx = matchKey(f.kind, scratch_);
scratch_.clear();
if (idx < 0) {{ if (idx < 0) {{
if (isStrict(f.kind)) {{ reject(); return; }} reject(); return; // unknown key
f.field = kSkip;
return;
}} }}
scratch_.clear();
if (f.seen[idx >> 6] & (1ull << (idx & 63))) {{ if (f.seen[idx >> 6] & (1ull << (idx & 63))) {{
reject(); return; // duplicate key reject(); return; // duplicate key
}} }}
@@ -1005,10 +1098,9 @@ private:
void cbStringData(const char *buf, int len, int done) {{ void cbStringData(const char *buf, int len, int done) {{
if (error_) return; if (error_) return;
if (stack_.empty()) {{ reject(); return; }}
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ if (done) valueComplete(); return; }}
if (si.cat == Cat::Str) {{ if (si.cat == Cat::Str) {{
auto *s = (std::string *)engage(f, !started_); auto *s = (std::string *)engage(f, !started_);
s->append(buf, len); s->append(buf, len);
@@ -1033,10 +1125,9 @@ private:
void cbNumberData(const char *buf, int len, int done) {{ void cbNumberData(const char *buf, int len, int done) {{
if (error_) return; if (error_) return;
if (stack_.empty()) {{ reject(); return; }}
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ if (done) valueComplete(); return; }}
if (si.cat != Cat::Int && si.cat != Cat::Dbl) {{ reject(); return; }} if (si.cat != Cat::Int && si.cat != Cat::Dbl) {{ reject(); return; }}
scratch_.append(buf, len); scratch_.append(buf, len);
started_ = true; started_ = true;
@@ -1051,16 +1142,11 @@ private:
*(int64_t *)p = v; // plain integer literal: parsed exactly *(int64_t *)p = v; // plain integer literal: parsed exactly
}} else {{ }} else {{
// JSON Schema "integer" accepts any number with no fractional part, // JSON Schema "integer" accepts any number with no fractional part,
// including exponent/decimal forms like 1e3 or 2.0. Parse those as a // including exponent/decimal forms like 1e3 or 2.0. Parse those
// double and require an integral value within int64 range. // exactly as int64 when the value is integral and in range.
double d = 0; int64_t v2 = 0;
auto rd = std::from_chars(b, e, d); if (!parseJsonInt64(b, e, v2)) {{ reject(); return; }}
if (rd.ec != std::errc() || rd.ptr != e || d != std::trunc(d) || *(int64_t *)p = v2;
d < -9223372036854775808.0 || d >= 9223372036854775808.0) {{
reject();
return;
}}
*(int64_t *)p = (int64_t)d;
}} }}
}} else {{ }} else {{
double v = 0; double v = 0;
@@ -1073,10 +1159,9 @@ private:
void cbBool(bool value) {{ void cbBool(bool value) {{
if (error_) return; if (error_) return;
if (stack_.empty()) {{ reject(); return; }}
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ valueComplete(); return; }}
if (si.cat != Cat::Bool) {{ reject(); return; }} if (si.cat != Cat::Bool) {{ reject(); return; }}
*(bool *)engage(f, true) = value; *(bool *)engage(f, true) = value;
valueComplete(); valueComplete();
@@ -1092,9 +1177,7 @@ private:
{null_at_root} {null_at_root}
}} }}
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ valueComplete(); return; }}
if (!si.nullable) {{ reject(); return; }} if (!si.nullable) {{ reject(); return; }}
if (isArrayKind(f.kind)) appendNull(f); // keep null array elements if (isArrayKind(f.kind)) appendNull(f); // keep null array elements
valueComplete(); // leave optional empty / pointer null valueComplete(); // leave optional empty / pointer null
@@ -1132,8 +1215,6 @@ private:
{self._reqmask()} {self._reqmask()}
{self._is_object_kind()} {self._is_object_kind()}
{self._is_strict()}
}}; }};
""" """