12 Commits
Author SHA1 Message Date
weaselbot 6fa4a6dcc8 schemagen: parse integer fallback exactly for decimal/exponent forms
Replace the double-based fallback in generated integer slots with a
string-to-int64 parser that handles decimal points and exponents
without losing precision near the int64 boundaries.

The old path used std::from_chars<double> and compared against
±9223372036854775808.0, which rounds the int64 max and min so that
valid values are rejected and out-of-range negatives are accepted.

The new helper:
- Parses sign, integer part, optional fraction, and optional exponent.
- Strips trailing zeros to cancel fractional places.
- Rejects non-integral values and overflow using exact uint64_t
  arithmetic.

Adds regression tests covering root integer and object-field integer
boundary values, including the cases from issue #19.
2026-06-23 18:35:05 -04:00
andrew ceb4bc1041 Merge pull request 'schemagen: drop support for additionalProperties: true' (#28) from weaselbot/weaseljson:weaselbot/issue-20 into main
Reviewed-on: weaselab/weaseljson#28
2026-06-23 20:56:58 +00:00
weaselbot 16f13c241c schemagen: drop support for additionalProperties: true
Following review feedback, the generator no longer supports permissive
objects. Changes:

- Reject `additionalProperties: true` at generation time.
- Treat an absent `additionalProperties` as `false`, so every object is
  strict by default and unknown keys are rejected during parsing.
- Remove the now-dead permissive-object infrastructure: `Kind::Skip`,
  `Cat::Skip`, `kSkip`, the per-frame `unknown` key set, and `isStrict()`.
- Update the README feature/rejection tables accordingly.
- Remove the permissive "loose" object from example.schema.json and the
  associated tests from test_gen.cpp.
- Add Python unit tests verifying the new `additionalProperties` behavior.

All tests pass (`ctest --output-on-failure`).
2026-06-23 15:11:48 -04:00
weaselbot ab95fefb09 schemagen: reject duplicate unknown keys in non-strict objects
Track unknown keys in a per-object unordered_set so that permissive
objects (additionalProperties absent/true) still reject duplicate keys,
matching the README guarantee.

- Add std::unordered_set<std::string> to Frame.
- Insert unknown keys in cbKeyData and reject duplicates before skipping.
- Add a permissive "loose" subobject to example.schema.json.
- Test single unknown key accepted and duplicate unknown/known keys rejected.
2026-06-23 15:08:48 -04:00
andrew e8830e27e9 Merge pull request 'schemagen: reserve generated Root/Skip/RootScalar and avoid Kind/ArrN collisions' (#27) from weaselbot/weaseljson:weaselbot/issue-21 into main
Reviewed-on: weaselab/weaseljson#27
2026-06-23 17:48:57 +00:00
andrew 5427db4b3d Merge pull request 'schemagen: fix nullable root types (#13)' (#29) from weaselbot/weaseljson:weaselbot/issue-13 into main
Reviewed-on: weaselab/weaseljson#29
2026-06-23 17:43:19 +00:00
weaselbot b5491afb38 schemagen: fix nullable root types (#13)
The C++ code generator now handles schemas where the top-level type is
nullable ("type": ["object", "null"], ["string", "null"], or
["array", "null"]).

Changes to weaseljson_schemagen.py:
- Rename the inner object struct when the root is a nullable object, so
  the `using Root = std::optional<...>` alias no longer conflicts with
  `struct Root`.
- Treat nullable root objects and arrays as container roots, emplacing
  the inner value before pushing the root frame and pointing the frame at
  the contained value.
- For nullable root scalars/enums, engage() now returns a pointer to the
  value inside the optional rather than to the optional wrapper itself.
- cbNull() now safely accepts a top-level null when the root is nullable
  and rejects it otherwise.

Regression tests added:
- nullable_object.schema.json + test_nullable_root.cpp
- nullable_string.schema.json
- nullable_array.schema.json

Closes #13
2026-06-23 12:58:28 -04:00
weaselbot 46ff8e2164 schemagen: reserve generated Root/Skip/RootScalar and avoid Kind/ArrN collisions
Make the type-name allocator aware of the identifiers the generator emits
itself (`Root` alias, `Skip`/`RootScalar` Kind enumerators) so user `$defs`
names can no longer collide with them.  Array-kind names (`Arr0`, `Arr1`, ...)
are now allocated only after checking for object/enum names, preventing
duplicate `Kind` enumerators when a schema defines e.g. `Arr0`.

Add Python regression tests that also syntax-check the generated headers
with a C++ compiler.

Closes #21
2026-06-23 12:23:09 -04:00
andrew 8d37b9b602 Merge pull request 'Reject negative lengths in WeaselJsonParser_parse' (#25) from weaselbot/weaseljson:weaselbot/issue-24 into main
Reviewed-on: weaselab/weaseljson#25
2026-06-22 18:53:20 +00:00
andrew 3d7dc97471 Merge pull request 'check WeaselJsonParser_create return value in Python bindings' (#26) from weaselbot/weaseljson:weaselbot/issue-23 into main
Reviewed-on: weaselab/weaseljson#26
2026-06-22 18:52:46 +00:00
weaselbot a26e101191 check WeaselJsonParser_create return value in Python bindings
Raise ValueError from __init__ when the C constructor returns NULL,
instead of storing a NULL pointer that segfaults on parse()/reset().
Add defensive RuntimeError checks in parse() and reset() for closed
or failed parsers.

Add a test covering stackSize values that the C API rejects (-1, 0, 1, 2).

Fixes #23
2026-06-22 02:29:13 -04:00
weaselbot 43e3c9904f Reject negative lengths in WeaselJsonParser_parse
`Parser3::parse` previously formed `buf + len` immediately, so passing a
negative `len` from the C API caused undefined pointer arithmetic. Add an
explicit `len < 0` check that returns `WeaselJson_REJECT` (and makes the
rejected state sticky) before any `buf + len` computation.

Also document the non-negative length precondition in the public header
and add a regression test.

Closes #24
2026-06-22 02:26:30 -04:00
13 changed files with 781 additions and 76 deletions
+48
View File
@@ -46,6 +46,49 @@ target_link_libraries(schemagen_big PRIVATE ${PROJECT_NAME})
target_compile_options(schemagen_big PRIVATE -Wno-switch-enum) target_compile_options(schemagen_big PRIVATE -Wno-switch-enum)
add_dependencies(schemagen_big schemagen_big_h) add_dependencies(schemagen_big schemagen_big_h)
set(NULLABLE_OBJECT_SCHEMA
${CMAKE_CURRENT_SOURCE_DIR}/nullable_object.schema.json)
set(NULLABLE_STRING_SCHEMA
${CMAKE_CURRENT_SOURCE_DIR}/nullable_string.schema.json)
set(NULLABLE_ARRAY_SCHEMA
${CMAKE_CURRENT_SOURCE_DIR}/nullable_array.schema.json)
set(NULLABLE_OBJECT_H ${CMAKE_CURRENT_BINARY_DIR}/nullable_object.h)
set(NULLABLE_STRING_H ${CMAKE_CURRENT_BINARY_DIR}/nullable_string.h)
set(NULLABLE_ARRAY_H ${CMAKE_CURRENT_BINARY_DIR}/nullable_array.h)
add_custom_command(
OUTPUT ${NULLABLE_OBJECT_H}
COMMAND ${Python3_EXECUTABLE} ${SCHEMAGEN_SCRIPT} ${NULLABLE_OBJECT_SCHEMA} -o
${NULLABLE_OBJECT_H} --namespace nullable_object
DEPENDS ${SCHEMAGEN_SCRIPT} ${NULLABLE_OBJECT_SCHEMA}
COMMENT "Generating nullable_object.h")
add_custom_command(
OUTPUT ${NULLABLE_STRING_H}
COMMAND ${Python3_EXECUTABLE} ${SCHEMAGEN_SCRIPT} ${NULLABLE_STRING_SCHEMA} -o
${NULLABLE_STRING_H} --namespace nullable_string
DEPENDS ${SCHEMAGEN_SCRIPT} ${NULLABLE_STRING_SCHEMA}
COMMENT "Generating nullable_string.h")
add_custom_command(
OUTPUT ${NULLABLE_ARRAY_H}
COMMAND ${Python3_EXECUTABLE} ${SCHEMAGEN_SCRIPT} ${NULLABLE_ARRAY_SCHEMA} -o
${NULLABLE_ARRAY_H} --namespace nullable_array
DEPENDS ${SCHEMAGEN_SCRIPT} ${NULLABLE_ARRAY_SCHEMA}
COMMENT "Generating nullable_array.h")
add_custom_target(
schemagen_nullable_h DEPENDS ${NULLABLE_OBJECT_H} ${NULLABLE_STRING_H}
${NULLABLE_ARRAY_H})
add_executable(schemagen_nullable_root
${CMAKE_CURRENT_SOURCE_DIR}/test_nullable_root.cpp)
target_include_directories(schemagen_nullable_root
PRIVATE include ${CMAKE_CURRENT_BINARY_DIR})
target_link_libraries(schemagen_nullable_root PRIVATE ${PROJECT_NAME})
target_compile_options(schemagen_nullable_root PRIVATE -Wno-switch-enum)
add_dependencies(schemagen_nullable_root schemagen_nullable_h)
add_test( add_test(
NAME schemagen_example NAME schemagen_example
COMMAND schemagen_example COMMAND schemagen_example
@@ -55,3 +98,8 @@ add_test(
NAME schemagen_big NAME schemagen_big
COMMAND schemagen_big COMMAND schemagen_big
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}) WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
add_test(
NAME schemagen_nullable_root
COMMAND schemagen_nullable_root
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
+5 -5
View File
@@ -45,7 +45,7 @@ into the result, so it is non-movable.
| `$ref` to `$defs`/`definitions` | the referenced named struct | | `$ref` to `$defs`/`definitions` | the referenced named struct |
| recursive `$ref` | `std::unique_ptr<T>` (cycle broken) | | recursive `$ref` | `std::unique_ptr<T>` (cycle broken) |
| `additionalProperties: false` | unknown keys rejected | | `additionalProperties: false` | unknown keys rejected |
| `additionalProperties` absent / `true` | unknown keys' values skipped | | `additionalProperties` absent / `true` | not supported (rejected at generation) |
## Schema violations (rejected at parse time) ## Schema violations (rejected at parse time)
@@ -55,15 +55,15 @@ into the result, so it is non-movable.
- value not in a string `enum` - value not in a string `enum`
- a number not representable in the target type (e.g. `1.5` for an `integer`) - a number not representable in the target type (e.g. `1.5` for an `integer`)
- duplicate object keys - duplicate object keys
- unknown key under `additionalProperties: false` - unknown key in any object
## Not supported (rejected at generation time, no fallback) ## Not supported (rejected at generation time, no fallback)
`oneOf` / `anyOf` / `allOf` / `not` / `if`-`then`-`else`, `oneOf` / `anyOf` / `allOf` / `not` / `if`-`then`-`else`,
`patternProperties`, `additionalProperties` with a schema (typed map), `patternProperties`, `additionalProperties` with a schema (typed map),
`prefixItems` (tuples), `const`, `dependentSchemas`/`dependentRequired`, `additionalProperties: true`, `prefixItems` (tuples), `const`,
union `type` lists other than `["T", "null"]`, non-string enums, and remote `dependentSchemas`/`dependentRequired`, union `type` lists other than
(`$ref` to other documents). `["T", "null"]`, non-string enums, and remote (`$ref` to other documents).
## Notes ## Notes
@@ -0,0 +1,9 @@
{
"type": [
"array",
"null"
],
"items": {
"type": "integer"
}
}
@@ -0,0 +1,15 @@
{
"type": [
"object",
"null"
],
"additionalProperties": false,
"required": [
"x"
],
"properties": {
"x": {
"type": "string"
}
}
}
@@ -0,0 +1,6 @@
{
"type": [
"string",
"null"
]
}
+158
View File
@@ -0,0 +1,158 @@
// Regression test for issue #13: nullable root types.
#include <cassert>
#include <cstdio>
#include <string>
#include "nullable_array.h"
#include "nullable_object.h"
#include "nullable_string.h"
static int failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); \
++failures; \
} \
} while (0)
static WeaselJsonStatus parseStrided(nullable_object::RootBuilder &b,
std::string in) {
for (size_t i = 0; i < in.size(); ++i) {
char c = in[i];
WeaselJsonStatus s = b.feed(&c, 1);
if (s != WeaselJson_AGAIN)
return s;
}
return b.finish();
}
static WeaselJsonStatus parseStrided(nullable_string::RootBuilder &b,
std::string in) {
for (size_t i = 0; i < in.size(); ++i) {
char c = in[i];
WeaselJsonStatus s = b.feed(&c, 1);
if (s != WeaselJson_AGAIN)
return s;
}
return b.finish();
}
static WeaselJsonStatus parseStrided(nullable_array::RootBuilder &b,
std::string in) {
for (size_t i = 0; i < in.size(); ++i) {
char c = in[i];
WeaselJsonStatus s = b.feed(&c, 1);
if (s != WeaselJson_AGAIN)
return s;
}
return b.finish();
}
static void expectReject(nullable_object::RootBuilder &b, std::string in,
const char *what) {
WeaselJsonStatus s = parseStrided(b, in);
if (s == WeaselJson_REJECT) {
printf("ok reject: %s\n", what);
} else {
printf("FAIL expected reject (%s) got status %d for: %s\n", what, s,
in.c_str());
++failures;
}
}
int main() {
// ---- nullable root object: valid document ----
{
nullable_object::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, R"({"x":"hello"})");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_object::Root r = b.take();
CHECK(r.has_value());
CHECK(r->x == "hello");
printf("ok nullable root object accepts object\n");
}
}
// ---- nullable root object: null document ----
{
nullable_object::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, "null");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_object::Root r = b.take();
CHECK(!r.has_value());
printf("ok nullable root object accepts null\n");
}
}
// ---- nullable root object: schema checks still run ----
{
nullable_object::RootBuilder b;
expectReject(b, R"({"x":"hello","extra":1})",
"unknown key in strict nullable root object");
}
{
nullable_object::RootBuilder b;
expectReject(b, R"({})", "missing required field in nullable root object");
}
// ---- nullable root string: valid value ----
{
nullable_string::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, R"("hello")");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_string::Root r = b.take();
CHECK(r.has_value());
CHECK(*r == "hello");
printf("ok nullable root string accepts string\n");
}
}
// ---- nullable root string: null value ----
{
nullable_string::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, "null");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_string::Root r = b.take();
CHECK(!r.has_value());
printf("ok nullable root string accepts null\n");
}
}
// ---- nullable root array: valid value ----
{
nullable_array::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, "[1,2,3]");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_array::Root r = b.take();
CHECK(r.has_value());
CHECK(r->size() == 3);
CHECK((*r)[0] == 1 && (*r)[1] == 2 && (*r)[2] == 3);
printf("ok nullable root array accepts array\n");
}
}
// ---- nullable root array: null value ----
{
nullable_array::RootBuilder b;
WeaselJsonStatus s = parseStrided(b, "null");
CHECK(s == WeaselJson_OK);
if (s == WeaselJson_OK) {
nullable_array::Root r = b.take();
CHECK(!r.has_value());
printf("ok nullable root array accepts null\n");
}
}
if (failures == 0) {
printf("\nALL TESTS PASSED\n");
return 0;
}
printf("\n%d FAILURE(S)\n", failures);
return 1;
}
+345
View File
@@ -3,9 +3,12 @@
import json import json
import os import os
import re
import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import textwrap
import unittest import unittest
SCRIPT = os.path.join(os.path.dirname(__file__), "weaseljson_schemagen.py") SCRIPT = os.path.join(os.path.dirname(__file__), "weaseljson_schemagen.py")
@@ -87,5 +90,347 @@ class SchemagenKeywordTest(unittest.TestCase):
self.assertNotIn(f"std::optional<std::string> {kw};", stdout) self.assertNotIn(f"std::optional<std::string> {kw};", stdout)
class SchemagenCollisionTest(unittest.TestCase):
"""Regression tests for issue #21: generated Root alias / Kind enum collisions."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.compiler = shutil.which("c++")
def generate_and_compile(self, schema):
"""Run schemagen on schema and syntax-check the resulting header."""
with tempfile.TemporaryDirectory() as tmpdir:
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
cmd = [
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
]
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
self.assertEqual(result.returncode, 0, msg=result.stderr)
if self.compiler:
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(
'#include "gen.h"\n'
"int main() {\n"
" test_schema::RootBuilder b;\n"
" test_schema::Root r = b.take();\n"
" (void)r;\n"
"}\n"
)
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-fsyntax-only",
"-I",
self.include_dir,
"-I",
tmpdir,
cpp_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
with open(header_path) as fp:
return fp.read()
def test_root_alias_does_not_collide_with_user_type(self):
schema = {
"type": "array",
"items": {"$ref": "#/$defs/Root"},
"$defs": {"Root": {"enum": ["a", "b"]}},
}
out = self.generate_and_compile(schema)
self.assertIn("enum class Root1 : int { a, b };", out)
self.assertIn("using Root = std::vector<Root1>;", out)
self.assertNotIn("using Root = std::vector<Root>;", out)
def test_kind_enum_does_not_duplicate_arr0(self):
schema = {
"type": "object",
"properties": {
"arr": {"type": "array", "items": {"type": "string"}},
"obj": {"$ref": "#/$defs/Arr0"},
},
"$defs": {"Arr0": {"type": "object", "properties": {}}},
}
out = self.generate_and_compile(schema)
self.assertIn("struct Arr0", out)
m = re.search(r"enum class Kind : uint8_t \{([^}]+)\}", out)
self.assertIsNotNone(m)
enumerators = [e.strip() for e in m.group(1).split(",")]
self.assertIn("Arr0", enumerators)
self.assertIn("Arr1", enumerators)
self.assertEqual(len(enumerators), len(set(enumerators)))
def test_skip_user_type_is_allowed(self):
schema = {
"type": "array",
"items": {"$ref": "#/$defs/Skip"},
"$defs": {"Skip": {"type": "object", "properties": {}}},
}
out = self.generate_and_compile(schema)
self.assertIn("struct Skip", out)
self.assertNotIn("struct Skip1", out)
m = re.search(r"enum class Kind : uint8_t \{([^}]+)\}", out)
self.assertIsNotNone(m)
enumerators = [e.strip() for e in m.group(1).split(",")]
self.assertIn("Skip", enumerators)
self.assertIn("Arr0", enumerators)
self.assertEqual(len(enumerators), len(set(enumerators)))
class SchemagenAdditionalPropertiesTest(unittest.TestCase):
def run_schemagen(self, schema, args=None):
"""Run schemagen on a schema dict. Returns (returncode, stdout, stderr)."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fp:
json.dump(schema, fp)
schema_path = fp.name
try:
cmd = [sys.executable, SCRIPT, schema_path]
if args:
cmd.extend(args)
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
return result.returncode, result.stdout, result.stderr
finally:
os.unlink(schema_path)
def test_additional_properties_true_rejected(self):
schema = {
"type": "object",
"additionalProperties": True,
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertNotEqual(rc, 0)
self.assertIn("additionalProperties: true is not supported", stderr)
def test_additional_properties_absent_defaults_to_strict(self):
schema = {
"type": "object",
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertEqual(rc, 0, msg=stderr)
# The generated parser should reject unknown keys. Verify the key-matching
# helper returns -1 for an unknown key and cbKeyData rejects it.
self.assertIn("int matchKey(Kind k, std::string_view key) const {", stdout)
self.assertNotIn("bool isStrict(Kind k) const", stdout)
def test_additional_properties_false_accepted(self):
schema = {
"type": "object",
"additionalProperties": False,
"properties": {"name": {"type": "string"}},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertEqual(rc, 0, msg=stderr)
class SchemagenIntegerBoundaryTest(unittest.TestCase):
"""Regression tests for issue #19: integer slot parsing near int64 boundaries."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.lib_src = os.path.join(self.repo_root, "src", "lib.cpp")
self.compiler = shutil.which("c++")
def _compile_harness(self, tmpdir, schema, harness):
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
result = subprocess.run(
[
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(result.returncode, 0, msg=result.stderr)
lib_obj = os.path.join(tmpdir, "lib.o")
comp_lib = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
os.path.join(self.repo_root, "third_party", "include"),
"-I",
os.path.join(self.repo_root, "third_party", "valgrind"),
"-c",
self.lib_src,
"-o",
lib_obj,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp_lib.returncode, 0, msg=comp_lib.stderr)
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(harness)
exe_path = os.path.join(tmpdir, "test")
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
tmpdir,
lib_obj,
cpp_path,
"-o",
exe_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
run = subprocess.run([exe_path], capture_output=True, text=True, check=False)
self.assertEqual(run.returncode, 0, msg=run.stdout + run.stderr)
def _build_cases_array(self, name, entries):
lines = [f" struct {name}Case {{ const char *s; WeaselJsonStatus expected; long long v; }};"]
lines.append(f" {name}Case {name}_cases[] = {{")
for s, exp, v in entries:
val = f"{v}LL" if isinstance(v, int) else v
lines.append(f' {{ R"({s})", {exp}, {val} }},')
lines.append(" };")
return "\n".join(lines)
def test_integer_boundary_root(self):
if not self.compiler:
self.skipTest("C++ compiler not available")
cases = [
("9223372036854775807.0", "WeaselJson_OK", 9223372036854775807),
("9223372036854775807", "WeaselJson_OK", 9223372036854775807),
("9223372036854775806.0", "WeaselJson_OK", 9223372036854775806),
("9223372036854775808", "WeaselJson_REJECT", 0),
("-9223372036854775808", "WeaselJson_OK", "-9223372036854775807LL - 1"),
("-9223372036854775808.0", "WeaselJson_OK", "-9223372036854775807LL - 1"),
("-9223372036854775809", "WeaselJson_REJECT", 0),
("1e3", "WeaselJson_OK", 1000),
("2.0", "WeaselJson_OK", 2),
("0.001", "WeaselJson_REJECT", 0),
("1e-3", "WeaselJson_REJECT", 0),
("1000e-3", "WeaselJson_OK", 1),
("100.0e-2", "WeaselJson_OK", 1),
("123.0", "WeaselJson_OK", 123),
("9e18", "WeaselJson_OK", 9000000000000000000),
("10e18", "WeaselJson_REJECT", 0),
]
cases_src = self._build_cases_array("root", cases)
harness = textwrap.dedent(
f"""
#include "gen.h"
#include <cstdio>
#include <cstring>
{cases_src}
int main() {{
for (const auto \u0026c : root_cases) {{
test_schema::RootBuilder b;
char buf[512];
std::strncpy(buf, c.s, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\\0';
WeaselJsonStatus st = b.feed(buf, std::strlen(buf));
st = b.finish();
if (st != c.expected) {{
std::printf("root case %s expected %d got %d\\n", c.s, c.expected, st);
return 1;
}}
if (st == WeaselJson_OK \u0026\u0026 b.take() != c.v) {{
std::printf("root case %s value mismatch\\n", c.s);
return 2;
}}
}}
return 0;
}}
"""
)
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, {"type": "integer"}, harness)
def test_integer_boundary_object_field(self):
if not self.compiler:
self.skipTest("C++ compiler not available")
cases = [
('{"age":9223372036854775807.0}', "WeaselJson_OK", 9223372036854775807),
('{"age":-9223372036854775809}', "WeaselJson_REJECT", 0),
('{"age":1e3}', "WeaselJson_OK", 1000),
('{"age":2.0}', "WeaselJson_OK", 2),
('{"age":0.001}', "WeaselJson_REJECT", 0),
('{"age":-9223372036854775808.0}', "WeaselJson_OK", "-9223372036854775807LL - 1"),
]
cases_src = self._build_cases_array("object", cases)
harness = textwrap.dedent(
f"""
#include "gen.h"
#include <cstdio>
#include <cstring>
{cases_src}
int main() {{
for (const auto \u0026c : object_cases) {{
test_schema::RootBuilder b;
char buf[512];
std::strncpy(buf, c.s, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\\0';
WeaselJsonStatus st = b.feed(buf, std::strlen(buf));
st = b.finish();
if (st != c.expected) {{
std::printf("object case %s expected %d got %d\\n", c.s, c.expected, st);
return 3;
}}
if (st == WeaselJson_OK \u0026\u0026 b.take().age != c.v) {{
std::printf("object case %s value mismatch\\n", c.s);
return 4;
}}
}}
return 0;
}}
"""
)
schema = {
"type": "object",
"properties": {"age": {"type": "integer"}},
"required": ["age"],
}
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, schema, harness)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+149 -69
View File
@@ -60,7 +60,6 @@ class ObjectType:
def __init__(self, name): def __init__(self, name):
self.name = name self.name = name
self.fields = [] # list[Field] self.fields = [] # list[Field]
self.strict = False # additionalProperties: false
class EnumType: class EnumType:
@@ -216,12 +215,26 @@ class Builder:
base = camel(hint) base = camel(hint)
name = base name = base
i = 1 i = 1
while name in self._used_names: while self._name_taken(name):
candidate = f"{base}{i}"
# A numeric suffix can itself land on a reserved generated name
# (e.g. hint "Arr0" -> "Arr01"). Use an underscore separator so
# we never loop through the reserved block.
if self._is_reserved(candidate):
candidate = f"{base}_{i}"
name = candidate
i += 1 i += 1
name = f"{base}{i}"
self._used_names.add(name) self._used_names.add(name)
return name return name
def _name_taken(self, name):
return name in self._used_names or self._is_reserved(name)
@staticmethod
def _is_reserved(name):
"""Names generated internally that must not collide with user types."""
return name in ("Root", "RootScalar")
def ref_name(self, ref): def ref_name(self, ref):
if not ref.startswith("#/"): if not ref.startswith("#/"):
raise GenError(f"only local $ref supported, got: {ref}") raise GenError(f"only local $ref supported, got: {ref}")
@@ -325,12 +338,13 @@ class Builder:
# register for $ref cycles before building fields # register for $ref cycles before building fields
if defname is not None: if defname is not None:
self._building[defname] = TObj(name) self._building[defname] = TObj(name)
ap = node.get("additionalProperties", True) ap = node.get("additionalProperties", False)
if ap is True:
raise GenError("additionalProperties: true is not supported")
if isinstance(ap, dict): if isinstance(ap, dict):
raise GenError( raise GenError(
"additionalProperties with a schema (typed map) is not " "supported yet" "additionalProperties with a schema (typed map) is not " "supported yet"
) )
obj.strict = ap is False
required = set(node.get("required", [])) required = set(node.get("required", []))
props = node.get("properties", {}) props = node.get("properties", {})
seen_cpp = set() seen_cpp = set()
@@ -435,6 +449,7 @@ class Emitter:
self.kind_order = [] # all Kind enumerators in declaration order self.kind_order = [] # all Kind enumerators in declaration order
self.root_ty = None self.root_ty = None
self.root_nullable = False self.root_nullable = False
self._arr_counter = 0
# -- type strings ------------------------------------------------------- # -- type strings -------------------------------------------------------
def base_cpp(self, ty): def base_cpp(self, ty):
@@ -465,11 +480,19 @@ class Emitter:
def arr_kind(self, tarr): def arr_kind(self, tarr):
sig = self.base_cpp(tarr) sig = self.base_cpp(tarr)
if sig not in self.arr_kinds: if sig not in self.arr_kinds:
name = f"Arr{len(self.arr_kinds)}" name = self._fresh_arr_kind_name()
self.arr_kinds[sig] = name self.arr_kinds[sig] = name
self.arr_types.append((name, tarr)) self.arr_types.append((name, tarr))
self.b._used_names.add(name)
return self.arr_kinds[sig] return self.arr_kinds[sig]
def _fresh_arr_kind_name(self):
while True:
name = f"Arr{self._arr_counter}"
self._arr_counter += 1
if not self.b._name_taken(name):
return name
def cat(self, ty): def cat(self, ty):
if isinstance(ty, TScalar): if isinstance(ty, TScalar):
return {"str": "Str", "int": "Int", "dbl": "Dbl", "bool": "Bool"}[ty.kind] return {"str": "Str", "int": "Int", "dbl": "Dbl", "bool": "Bool"}[ty.kind]
@@ -493,6 +516,18 @@ class Emitter:
self.root_ty, self.root_nullable = Builder._unpack( self.root_ty, self.root_nullable = Builder._unpack(
self.b.build_type(self.b.root_schema, "Root") self.b.build_type(self.b.root_schema, "Root")
) )
# A nullable root object would otherwise produce
# using Root = std::optional<Root>;
# which conflicts with the struct named Root. Rename the inner struct.
if isinstance(self.root_ty, TObj) and self.root_nullable:
old_name = self.root_ty.name
new_name = self.b.unique_name("RootInner")
obj = self.b.objects.pop(old_name)
obj.name = new_name
self.b.objects[new_name] = obj
self.root_ty = TObj(new_name)
break_cycles(self.b.objects) break_cycles(self.b.objects)
# register all array kinds (walk every field + root) # register all array kinds (walk every field + root)
@@ -582,20 +617,17 @@ namespace {ns} {{"""
def _kind_enum(self): def _kind_enum(self):
kinds = list(self.b.objects.keys()) kinds = list(self.b.objects.keys())
kinds += [n for n, _ in self.arr_types] kinds += [n for n, _ in self.arr_types]
root_is_container = ( root_is_container = isinstance(self.root_ty, (TObj, TArr))
isinstance(self.root_ty, (TObj, TArr)) and not self.root_nullable
)
if not root_is_container: if not root_is_container:
kinds.append("RootScalar") kinds.append("RootScalar")
kinds.append("Skip")
self.kind_order = kinds self.kind_order = kinds
return " enum class Kind : uint8_t { " + ", ".join(kinds) + " };" return " enum class Kind : uint8_t { " + ", ".join(kinds) + " };"
def _root_info(self): def _root_info(self):
"""Return (root_cat, root_container_kind_or_None).""" """Return (root_cat, root_container_kind_or_None)."""
if isinstance(self.root_ty, TObj) and not self.root_nullable: if isinstance(self.root_ty, TObj):
return ("Obj", f"Kind::{self.root_ty.name}") return ("Obj", f"Kind::{self.root_ty.name}")
if isinstance(self.root_ty, TArr) and not self.root_nullable: if isinstance(self.root_ty, TArr):
return ("Arr", f"Kind::{self.arr_kind(self.root_ty)}") return ("Arr", f"Kind::{self.arr_kind(self.root_ty)}")
return (self.cat(self.root_ty), None) return (self.cat(self.root_ty), None)
@@ -665,6 +697,11 @@ namespace {ns} {{"""
lines.append(" }") lines.append(" }")
root_cat, root_kind = self._root_info() root_cat, root_kind = self._root_info()
if root_kind is None: if root_kind is None:
if self.root_nullable:
lines.append(
" case Kind::RootScalar: { if (!result_) result_.emplace(); return &*result_; }"
)
else:
lines.append(" case Kind::RootScalar: return &result_;") lines.append(" case Kind::RootScalar: return &result_;")
lines.append(" default: return nullptr;") lines.append(" default: return nullptr;")
lines.append(" }") lines.append(" }")
@@ -771,17 +808,6 @@ namespace {ns} {{"""
lines.append(" }") lines.append(" }")
return "\n".join(lines) return "\n".join(lines)
def _is_strict(self):
strict = [n for n, o in self.b.objects.items() if o.strict]
if not strict:
return " bool isStrict(Kind) const { return false; }"
cases = " ".join(f"case Kind::{n}:" for n in strict)
return (
" bool isStrict(Kind k) const {\n"
f" switch (k) {{ {cases} return true; default: return false; }}\n"
" }"
)
def _enum_name_arrays(self): def _enum_name_arrays(self):
out = [] out = []
for e in self.b.enums.values(): for e in self.b.enums.values():
@@ -807,7 +833,12 @@ namespace {ns} {{"""
lines = [" if (error_) return;"] lines = [" if (error_) return;"]
if root_kind is not None and root_cat == event_cat: if root_kind is not None and root_cat == event_cat:
lines.append(" if (stack_.empty()) {") lines.append(" if (stack_.empty()) {")
lines.append(f" stack_.push_back(Frame{{{root_kind}, &result_}});") if self.root_nullable:
lines.append(" result_.emplace();")
dest = "&*result_"
else:
dest = "&result_"
lines.append(f" stack_.push_back(Frame{{{root_kind}, {dest}}});")
if event_cat == "Obj": if event_cat == "Obj":
lines.append(" {") lines.append(" {")
lines.append(f" int n = fieldCount({root_kind});") lines.append(f" int n = fieldCount({root_kind});")
@@ -818,13 +849,7 @@ namespace {ns} {{"""
else: else:
lines.append(" if (stack_.empty()) { reject(); return; }") lines.append(" if (stack_.empty()) { reject(); return; }")
lines.append(" Frame &f = stack_.back();") lines.append(" Frame &f = stack_.back();")
lines.append(
" if (f.kind == Kind::Skip) { stack_.push_back(Frame{Kind::Skip, nullptr}); return; }"
)
lines.append(" SlotInfo si = slotInfoG(f);") lines.append(" SlotInfo si = slotInfoG(f);")
lines.append(
" if (si.cat == Cat::Skip) { stack_.push_back(Frame{Kind::Skip, nullptr}); return; }"
)
lines.append(f" if (si.cat != Cat::{event_cat}) {{ reject(); return; }}") lines.append(f" if (si.cat != Cat::{event_cat}) {{ reject(); return; }}")
lines.append(" void *p = engage(f, true);") lines.append(" void *p = engage(f, true);")
lines.append(" stack_.push_back(Frame{si.child, p});") lines.append(" stack_.push_back(Frame{si.child, p});")
@@ -839,6 +864,9 @@ namespace {ns} {{"""
root_cat, root_kind = self._root_info() root_cat, root_kind = self._root_info()
begin_obj = self._begin_container("Obj", root_kind) begin_obj = self._begin_container("Obj", root_kind)
begin_arr = self._begin_container("Arr", root_kind) begin_arr = self._begin_container("Arr", root_kind)
null_at_root = (
"done_ = true; return;" if self.root_nullable else "reject(); return;"
)
return f""" return f"""
class RootBuilder {{ class RootBuilder {{
@@ -872,7 +900,7 @@ public:
private: private:
{kind_enum} {kind_enum}
enum class Cat {{ Reject, Str, Int, Dbl, Bool, Enum, Obj, Arr, Skip }}; enum class Cat {{ Reject, Str, Int, Dbl, Bool, Enum, Obj, Arr }};
struct SlotInfo {{ struct SlotInfo {{
Cat cat; Cat cat;
Kind child{{}}; Kind child{{}};
@@ -883,11 +911,10 @@ private:
struct Frame {{ struct Frame {{
Kind kind; Kind kind;
void *dest; void *dest;
int field = -1; // object: selected field (-1 want key, -2 skip) int field = -1; // object: selected field (-1 means waiting for key)
std::vector<uint64_t> seen; // populated field bitset (object frames) std::vector<uint64_t> seen; // populated field bitset (object frames)
}}; }};
static constexpr int kWantKey = -1; static constexpr int kWantKey = -1;
static constexpr int kSkip = -2;
{self._enum_name_arrays()} {self._enum_name_arrays()}
@@ -902,10 +929,87 @@ private:
void reject() {{ error_ = true; }} void reject() {{ error_ = true; }}
// Wrap the generated slotInfo() with the generic key/skip states. // Parse a JSON number text as int64_t. Accepts optional decimal point and
// exponent only when the mathematical value is an integer that fits in a
// signed 64-bit range.
static bool parseJsonInt64(const char *b, const char *e, int64_t &out) {{
const char *p = b;
bool neg = false;
if (p < e) {{
if (*p == '-') {{ neg = true; ++p; }}
else if (*p == '+') return false;
}}
const char *intStart = p;
while (p < e && *p >= '0' && *p <= '9') ++p;
const char *intEnd = p;
int fracDigits = 0;
const char *fracStart = p;
if (p < e && *p == '.') {{
++p;
fracStart = p;
while (p < e && *p >= '0' && *p <= '9') {{ ++p; ++fracDigits; }}
if (fracStart == p) return false;
}}
int64_t exp = 0;
bool expNeg = false;
if (p < e && (*p == 'e' || *p == 'E')) {{
++p;
if (p < e && (*p == '-' || *p == '+')) {{ expNeg = (*p == '-'); ++p; }}
if (p == e || *p < '0' || *p > '9') return false;
while (p < e && *p >= '0' && *p <= '9') {{
int digit = *p - '0';
if (exp <= (INT64_MAX - digit) / 10) exp = exp * 10 + digit;
else exp = INT64_MAX;
++p;
}}
if (expNeg) exp = -exp;
}}
if (p != e) return false;
if (intStart == intEnd) return false;
std::string digits;
digits.reserve((intEnd - intStart) + fracDigits);
for (const char *q = intStart; q < intEnd; ++q) digits.push_back(*q);
for (int i = 0; i < fracDigits; ++i) digits.push_back(fracStart[i]);
int64_t trim = 0;
while (!digits.empty() && digits.back() == '0') {{
digits.pop_back();
++trim;
}}
int64_t finalExp = exp - fracDigits + trim;
if (finalExp < 0) return false;
while (!digits.empty() && digits.front() == '0') digits.erase(digits.begin());
if (digits.empty()) {{ out = 0; return true; }}
constexpr uint64_t kMaxNeg = 9223372036854775808ULL;
constexpr uint64_t kMaxPos = 9223372036854775807ULL;
const uint64_t limit = neg ? kMaxNeg : kMaxPos;
if (finalExp > 19) return false;
int64_t maxSig = 19 - finalExp;
uint64_t mag = 0;
int64_t sigDigits = 0;
for (char ch : digits) {{
uint64_t d = static_cast<uint64_t>(ch - '0');
if (sigDigits >= maxSig) return false;
if (mag > (limit - d) / 10) return false;
mag = mag * 10 + d;
++sigDigits;
}}
for (int64_t i = 0; i < finalExp; ++i) {{
if (mag > limit / 10) return false;
mag *= 10;
}}
if (mag > limit) return false;
__int128 signedMag = static_cast<__int128>(mag);
if (neg) signedMag = -signedMag;
out = static_cast<int64_t>(signedMag);
return true;
}}
// Wrap the generated slotInfo() with the generic key state.
SlotInfo slotInfoG(const Frame &f) {{ SlotInfo slotInfoG(const Frame &f) {{
if (isObjectKind(f.kind)) {{ if (isObjectKind(f.kind)) {{
if (f.field == kSkip) return SlotInfo{{Cat::Skip}};
if (f.field < 0) return SlotInfo{{Cat::Reject}}; if (f.field < 0) return SlotInfo{{Cat::Reject}};
}} }}
return slotInfo(f); return slotInfo(f);
@@ -930,11 +1034,6 @@ private:
void cbEndObject() {{ void cbEndObject() {{
if (error_) return; if (error_) return;
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) {{
stack_.pop_back();
if (stack_.empty() || stack_.back().kind != Kind::Skip) valueComplete();
return;
}}
if (isObjectKind(f.kind)) {{ if (isObjectKind(f.kind)) {{
const auto &req = requiredMask(f.kind); const auto &req = requiredMask(f.kind);
bool missing = false; bool missing = false;
@@ -952,11 +1051,6 @@ private:
void cbEndArray() {{ void cbEndArray() {{
if (error_) return; if (error_) return;
Frame f = stack_.back(); Frame f = stack_.back();
if (f.kind == Kind::Skip) {{
stack_.pop_back();
if (stack_.empty() || stack_.back().kind != Kind::Skip) valueComplete();
return;
}}
stack_.pop_back(); stack_.pop_back();
valueComplete(); valueComplete();
}} }}
@@ -968,12 +1062,10 @@ private:
scratch_.append(buf, len); scratch_.append(buf, len);
if (!done) return; if (!done) return;
int idx = matchKey(f.kind, scratch_); int idx = matchKey(f.kind, scratch_);
scratch_.clear();
if (idx < 0) {{ if (idx < 0) {{
if (isStrict(f.kind)) {{ reject(); return; }} reject(); return; // unknown key
f.field = kSkip;
return;
}} }}
scratch_.clear();
if (f.seen[idx >> 6] & (1ull << (idx & 63))) {{ if (f.seen[idx >> 6] & (1ull << (idx & 63))) {{
reject(); return; // duplicate key reject(); return; // duplicate key
}} }}
@@ -983,9 +1075,7 @@ private:
void cbStringData(const char *buf, int len, int done) {{ void cbStringData(const char *buf, int len, int done) {{
if (error_) return; if (error_) return;
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ if (done) valueComplete(); return; }}
if (si.cat == Cat::Str) {{ if (si.cat == Cat::Str) {{
auto *s = (std::string *)engage(f, !started_); auto *s = (std::string *)engage(f, !started_);
s->append(buf, len); s->append(buf, len);
@@ -1011,9 +1101,7 @@ private:
void cbNumberData(const char *buf, int len, int done) {{ void cbNumberData(const char *buf, int len, int done) {{
if (error_) return; if (error_) return;
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ if (done) valueComplete(); return; }}
if (si.cat != Cat::Int && si.cat != Cat::Dbl) {{ reject(); return; }} if (si.cat != Cat::Int && si.cat != Cat::Dbl) {{ reject(); return; }}
scratch_.append(buf, len); scratch_.append(buf, len);
started_ = true; started_ = true;
@@ -1028,16 +1116,11 @@ private:
*(int64_t *)p = v; // plain integer literal: parsed exactly *(int64_t *)p = v; // plain integer literal: parsed exactly
}} else {{ }} else {{
// JSON Schema "integer" accepts any number with no fractional part, // JSON Schema "integer" accepts any number with no fractional part,
// including exponent/decimal forms like 1e3 or 2.0. Parse those as a // including exponent/decimal forms like 1e3 or 2.0. Parse those
// double and require an integral value within int64 range. // exactly as int64 when the value is integral and in range.
double d = 0; int64_t v2 = 0;
auto rd = std::from_chars(b, e, d); if (!parseJsonInt64(b, e, v2)) {{ reject(); return; }}
if (rd.ec != std::errc() || rd.ptr != e || d != std::trunc(d) || *(int64_t *)p = v2;
d < -9223372036854775808.0 || d >= 9223372036854775808.0) {{
reject();
return;
}}
*(int64_t *)p = (int64_t)d;
}} }}
}} else {{ }} else {{
double v = 0; double v = 0;
@@ -1051,9 +1134,7 @@ private:
void cbBool(bool value) {{ void cbBool(bool value) {{
if (error_) return; if (error_) return;
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ valueComplete(); return; }}
if (si.cat != Cat::Bool) {{ reject(); return; }} if (si.cat != Cat::Bool) {{ reject(); return; }}
*(bool *)engage(f, true) = value; *(bool *)engage(f, true) = value;
valueComplete(); valueComplete();
@@ -1065,10 +1146,11 @@ private:
void cbNull() {{ void cbNull() {{
if (error_) return; if (error_) return;
if (stack_.empty()) {{
{null_at_root}
}}
Frame &f = stack_.back(); Frame &f = stack_.back();
if (f.kind == Kind::Skip) return;
SlotInfo si = slotInfoG(f); SlotInfo si = slotInfoG(f);
if (si.cat == Cat::Skip) {{ valueComplete(); return; }}
if (!si.nullable) {{ reject(); return; }} if (!si.nullable) {{ reject(); return; }}
if (isArrayKind(f.kind)) appendNull(f); // keep null array elements if (isArrayKind(f.kind)) appendNull(f); // keep null array elements
valueComplete(); // leave optional empty / pointer null valueComplete(); // leave optional empty / pointer null
@@ -1106,8 +1188,6 @@ private:
{self._reqmask()} {self._reqmask()}
{self._is_object_kind()} {self._is_object_kind()}
{self._is_strict()}
}}; }};
""" """
+2 -1
View File
@@ -64,7 +64,8 @@ void WeaselJsonParser_destroy(WeaselJsonParser *parser);
/** Incrementally parse `len` more bytes starting at `buf`. `buf` may be /** Incrementally parse `len` more bytes starting at `buf`. `buf` may be
* modified. Call with `len` 0 to indicate end of data. `buf` may be null if * modified. Call with `len` 0 to indicate end of data. `buf` may be null if
* `len` is 0 */ * `len` is 0. `len` must not be negative; a negative length is treated as a
* rejected input. */
WeaselJsonStatus WeaselJsonParser_parse(WeaselJsonParser *parser, char *buf, WeaselJsonStatus WeaselJsonParser_parse(WeaselJsonParser *parser, char *buf,
int len); int len);
+5
View File
@@ -1068,6 +1068,11 @@ inline WeaselJsonStatus Parser3::parse(char *buf, int len) {
return WeaselJson_REJECT; return WeaselJson_REJECT;
} }
if (len < 0) [[unlikely]] {
this->rejected = true;
return WeaselJson_REJECT;
}
#ifdef HAS_MUSTTAIL #ifdef HAS_MUSTTAIL
// The continuation returns a value in 0..3 here (kBounce is only used by the // The continuation returns a value in 0..3 here (kBounce is only used by the
// no-musttail trampoline below), so the conversion back to the enum is in // no-musttail trampoline below), so the conversion back to the enum is in
+14
View File
@@ -246,6 +246,20 @@ TEST_CASE("create rejects too-small stack") {
WeaselJsonParser_destroy(parser); WeaselJsonParser_destroy(parser);
} }
TEST_CASE("parse rejects negative length") {
auto c = noopCallbacks();
auto *parser = WeaselJsonParser_create(1024, &c, nullptr, 0);
REQUIRE(parser != nullptr);
// A negative length must not cause pointer arithmetic UB. It should be
// rejected, and the rejected state should remain sticky.
char buf[10] = "hello";
REQUIRE(WeaselJsonParser_parse(parser, buf, -1) == WeaselJson_REJECT);
REQUIRE(WeaselJsonParser_parse(parser, nullptr, 0) == WeaselJson_REJECT);
WeaselJsonParser_destroy(parser);
}
TEST_CASE("streaming") { testStreaming(json); } TEST_CASE("streaming") { testStreaming(json); }
TEST_CASE("reset clears inKey and transient state") { TEST_CASE("reset clears inKey and transient state") {
+12
View File
@@ -84,7 +84,19 @@ def test_mixed_values():
assert recorder.events.count("null") == 1 assert recorder.events.count("null") == 1
def test_create_rejects_too_small_stack():
for stack_size in (-1, 0, 1, 2):
try:
parser = weaseljson.WeaselJsonParser(Recorder(), stackSize=stack_size)
except ValueError:
continue
# If creation unexpectedly succeeds, close it cleanly and fail the test.
parser.close()
raise AssertionError(f"expected ValueError for stackSize={stack_size}")
if __name__ == "__main__": if __name__ == "__main__":
test_object_keys_routed_correctly() test_object_keys_routed_correctly()
test_mixed_values() test_mixed_values()
test_create_rejects_too_small_stack()
print("python bindings ok") print("python bindings ok")
+12
View File
@@ -117,12 +117,24 @@ class WeaselJsonParser:
self.voidp_callbacks, self.voidp_callbacks,
0, 0,
) )
if self.p is None:
raise ValueError(
"WeaselJsonParser_create returned NULL; "
"check stackSize (must be positive and large enough) "
"and available memory"
)
def _check_open(self):
if self.p is None:
raise RuntimeError("parser has been closed or creation failed")
def parse(self, data: bytes) -> WeaselJsonStatus: def parse(self, data: bytes) -> WeaselJsonStatus:
self._check_open()
buf = (ctypes.c_ubyte * len(data)).from_buffer(bytearray(data)) buf = (ctypes.c_ubyte * len(data)).from_buffer(bytearray(data))
return self._lib.WeaselJsonParser_parse(self.p, buf, len(data)) return self._lib.WeaselJsonParser_parse(self.p, buf, len(data))
def reset(self): def reset(self):
self._check_open()
self._lib.WeaselJsonParser_reset(self.p) self._lib.WeaselJsonParser_reset(self.p)
def __enter__(self): def __enter__(self):