11 Commits
Author SHA1 Message Date
weaselbot f0a338f164 make WeaselJson_OVERFLOW a terminal state like WeaselJson_REJECT
When a parse step returned WeaselJson_OVERFLOW, the pushdown stack was
left corrupted (frames had been popped before the failing push), but the
overflow status was not made sticky the way WeaselJson_REJECT is. A
subsequent end-of-data call WeaselJsonParser_parse(parser, nullptr, 0)
could then dispatch through the corrupted stack and return WeaselJson_OK,
accepting an incomplete, invalid too-deeply-nested document as valid JSON.

Add an `overflowed` flag, set it whenever a continuation returns
WeaselJson_OVERFLOW, and short-circuit parse() to return
WeaselJson_OVERFLOW on every subsequent call. reset() clears the flag so a
reused parser can accept a different document. This mirrors the existing
stickiness handling for WeaselJson_REJECT.

Closes #52
2026-07-13 11:45:58 -04:00
andrew f5ceb392c2 Merge pull request 'schemagen: add regression test for nullable cyclic $ref targets' (#50) from weaselbot/weaseljson:weaselbot/issue-14 into main
Reviewed-on: weaselab/weaseljson#50
2026-06-30 16:50:07 +00:00
weaselbot be92755cbf schemagen: add regression test for nullable cyclic $ref targets (issue #14)
The generator already preserves nullability for self-referential object
$defs thanks to prior fixes, but issue #14 had no regression coverage.

Add a test using the exact reproduction schema from the issue and verify
that both the outer and recursive `self` fields accept `null`.
2026-06-30 12:42:03 -04:00
andrew d281c35811 Merge pull request 'schemagen: reject self-referential array $ref cycles instead of crashing' (#49) from weaselbot/weaseljson:weaselbot/issue-33 into main
Reviewed-on: weaselab/weaseljson#49
2026-06-30 16:37:02 +00:00
andrew f6617a1558 Merge pull request 'schemagen: escape control characters in generated C++ string literals' (#48) from weaselbot/weaseljson:weaselbot/issue-35 into main
Reviewed-on: weaselab/weaseljson#48
2026-06-30 16:30:21 +00:00
weaselbot ababd3a8fd schemagen: reject self-referential array $ref cycles instead of crashing
Cyclic array definitions (directly or through a chain of array $defs)
created a self-referential TArr, which then caused infinite recursion in
base_cpp, storage_cpp, and _walk_arrays.

Object-only cycles are already broken with std::unique_ptr, but
array-only cycles have no object edge for break_cycles to cut.

Detect them after building an array's items by following TArr.elem links
and raise a clear GenError so generation fails gracefully rather than
overflowing the Python stack.

Closes #33
2026-06-30 12:29:37 -04:00
weaselbot 681892107f schemagen: escape control characters in generated C++ string literals
Fixes #35.

Add a helper to escape C++ string literals so that JSON control characters
(\n, \r, \t, and other bytes below 0x20) are emitted as escape sequences
instead of raw bytes. Use it for:

- field comments that include the JSON property key
- object key comparison literals in matchKey()
- enum name arrays

Also add regression tests that generate and syntax-check headers for
schemas containing newlines and other control characters in property keys
and enum values.
2026-06-30 12:26:10 -04:00
andrew e22bc039ae Merge pull request 'schemagen: fix quadratic leading-zero strip in integer fallback' (#47) from weaselbot/weaseljson:weaselbot/issue-34 into main
Reviewed-on: weaselab/weaseljson#47
2026-06-30 16:16:44 +00:00
weaselbot ceb16e5405 schemagen: fix quadratic leading-zero strip in integer fallback
Replace the O(k^2) loop that erased leading zeros one byte at a time
from the front of a std::string with a single linear scan and one
erase(0, n) call.

Also adds regression tests for issue #34:
- correctness cases for numbers with leading fractional zeros
- a static check that the generated code no longer contains the
  quadratic pattern
- a large-input case (100k leading zeros) that reproduces the
  vulnerable shape

Closes #34
2026-06-30 12:09:26 -04:00
andrew 241c29073b Merge pull request 'schemagen: handle WeaselJsonParser_create failure in RootBuilder' (#46) from weaselbot/weaseljson:weaselbot/issue-36 into main
Reviewed-on: weaselab/weaseljson#46
2026-06-30 15:35:20 +00:00
weaselbot abeaae7ed7 schemagen: handle WeaselJsonParser_create failure in RootBuilder
If WeaselJsonParser_create returns nullptr (e.g. negative stack size or allocation failure), set the existing error_ flag so that subsequent feed()/finish() calls return WeaselJson_REJECT instead of dereferencing the null parser_.

Also add a regression test in test_gen.cpp that constructs a RootBuilder with an invalid stack size and verifies it rejects without crashing.

Closes #36
2026-06-30 11:26:59 -04:00
5 changed files with 497 additions and 8 deletions
+9
View File
@@ -68,6 +68,15 @@ int main() {
expectReject(json, "unknown key in strict root"); expectReject(json, "unknown key in strict root");
} }
// ---- invalid stack size is rejected without crashing ----
{
RootBuilder b(-1);
char buf[] = "null";
WeaselJsonStatus s = b.feed(buf, sizeof(buf) - 1);
CHECK(s == WeaselJson_REJECT);
printf("ok invalid stack size rejected, not crashed\n");
}
{ {
std::string json = R"({ std::string json = R"({
"name": "Ada É", "name": "Ada É",
+349
View File
@@ -274,6 +274,211 @@ class SchemagenAdditionalPropertiesTest(unittest.TestCase):
self.assertEqual(rc, 0, msg=stderr) self.assertEqual(rc, 0, msg=stderr)
class SchemagenCyclicArrayTest(unittest.TestCase):
"""Regression tests for issue #33: cyclic array $ref targets."""
def run_schemagen(self, schema, args=None):
"""Run schemagen on a schema dict. Returns (returncode, stdout, stderr)."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fp:
json.dump(schema, fp)
schema_path = fp.name
try:
cmd = [sys.executable, SCRIPT, schema_path]
if args:
cmd.extend(args)
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
return result.returncode, result.stdout, result.stderr
finally:
os.unlink(schema_path)
def test_direct_self_referential_array_rejected(self):
schema = {
"type": "array",
"items": {"$ref": "#/$defs/Node"},
"$defs": {
"Node": {
"type": "array",
"items": {"$ref": "#/$defs/Node"},
}
},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertNotEqual(rc, 0)
self.assertIn("recursive array type is not supported", stderr)
self.assertIn("Node", stderr)
def test_object_field_to_self_referential_array_rejected(self):
schema = {
"type": "object",
"properties": {"items": {"$ref": "#/$defs/Items"}},
"$defs": {
"Items": {
"type": "array",
"items": {"$ref": "#/$defs/Items"},
}
},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertNotEqual(rc, 0)
self.assertIn("recursive array type is not supported", stderr)
self.assertIn("Items", stderr)
def test_chain_of_array_refs_rejected(self):
schema = {
"type": "object",
"properties": {"x": {"$ref": "#/$defs/A"}},
"$defs": {
"A": {"type": "array", "items": {"$ref": "#/$defs/B"}},
"B": {"type": "array", "items": {"$ref": "#/$defs/A"}},
},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertNotEqual(rc, 0)
self.assertIn("recursive array type is not supported", stderr)
def test_non_recursive_array_refs_still_allowed(self):
schema = {
"type": "object",
"properties": {"roles": {"$ref": "#/$defs/Roles"}},
"$defs": {
"Role": {"enum": ["admin", "user"]},
"Roles": {
"type": "array",
"items": {"$ref": "#/$defs/Role"},
},
},
}
rc, stdout, stderr = self.run_schemagen(schema)
self.assertEqual(rc, 0, msg=stderr)
self.assertIn("std::optional<std::vector<Role>> roles;", stdout)
class SchemagenCyclicNullableObjectTest(unittest.TestCase):
"""Regression tests for issue #14: nullable cyclic $ref targets."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.lib_src = os.path.join(self.repo_root, "src", "lib.cpp")
self.compiler = shutil.which("c++")
def _compile_harness(self, tmpdir, schema, harness):
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
result = subprocess.run(
[
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(result.returncode, 0, msg=result.stderr)
lib_obj = os.path.join(tmpdir, "lib.o")
comp_lib = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
os.path.join(self.repo_root, "third_party", "include"),
"-I",
os.path.join(self.repo_root, "third_party", "valgrind"),
"-c",
self.lib_src,
"-o",
lib_obj,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp_lib.returncode, 0, msg=comp_lib.stderr)
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(harness)
exe_path = os.path.join(tmpdir, "test")
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-I",
self.include_dir,
"-I",
tmpdir,
lib_obj,
cpp_path,
"-o",
exe_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
run = subprocess.run([exe_path], capture_output=True, text=True, check=False)
self.assertEqual(run.returncode, 0, msg=run.stdout + run.stderr)
def test_self_referential_nullable_object_accepts_nested_null(self):
"""A nullable object $def must stay nullable on recursive $refs."""
if not self.compiler:
self.skipTest("C++ compiler not available")
schema = {
"type": "object",
"properties": {"self": {"$ref": "#/$defs/Self"}},
"$defs": {
"Self": {
"type": ["object", "null"],
"properties": {"self": {"$ref": "#/$defs/Self"}},
}
},
}
harness = textwrap.dedent(
"""
#include "gen.h"
#include <cstdio>
#include <cstring>
struct Case { const char *s; WeaselJsonStatus expected; };
int main() {
Case cases[] = {
{ R"({"self": null})", WeaselJson_OK },
{ R"({"self": {"self": null}})", WeaselJson_OK },
{ R"({"self": {"self": {}}})", WeaselJson_OK },
};
for (const auto &c : cases) {
test_schema::RootBuilder b;
char buf[256];
std::strncpy(buf, c.s, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\\0';
WeaselJsonStatus st = b.feed(buf, std::strlen(buf));
st = b.finish();
if (st != c.expected) {
std::printf("case %s expected %d got %d\\n", c.s, c.expected, st);
return 1;
}
}
return 0;
}
"""
)
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, schema, harness)
class SchemagenIntegerBoundaryTest(unittest.TestCase): class SchemagenIntegerBoundaryTest(unittest.TestCase):
"""Regression tests for issue #19: integer slot parsing near int64 boundaries.""" """Regression tests for issue #19: integer slot parsing near int64 boundaries."""
@@ -382,6 +587,8 @@ class SchemagenIntegerBoundaryTest(unittest.TestCase):
("1e-3", "WeaselJson_REJECT", 0), ("1e-3", "WeaselJson_REJECT", 0),
("1000e-3", "WeaselJson_OK", 1), ("1000e-3", "WeaselJson_OK", 1),
("100.0e-2", "WeaselJson_OK", 1), ("100.0e-2", "WeaselJson_OK", 1),
("0.0001e4", "WeaselJson_OK", 1),
("0.001e3", "WeaselJson_OK", 1),
("123.0", "WeaselJson_OK", 123), ("123.0", "WeaselJson_OK", 123),
("9e18", "WeaselJson_OK", 9000000000000000000), ("9e18", "WeaselJson_OK", 9000000000000000000),
("10e18", "WeaselJson_REJECT", 0), ("10e18", "WeaselJson_REJECT", 0),
@@ -425,6 +632,7 @@ class SchemagenIntegerBoundaryTest(unittest.TestCase):
('{"age":-9223372036854775809}', "WeaselJson_REJECT", 0), ('{"age":-9223372036854775809}', "WeaselJson_REJECT", 0),
('{"age":1e3}', "WeaselJson_OK", 1000), ('{"age":1e3}', "WeaselJson_OK", 1000),
('{"age":2.0}', "WeaselJson_OK", 2), ('{"age":2.0}', "WeaselJson_OK", 2),
('{"age":0.0001e4}', "WeaselJson_OK", 1),
('{"age":0.001}', "WeaselJson_REJECT", 0), ('{"age":0.001}', "WeaselJson_REJECT", 0),
( (
'{"age":-9223372036854775808.0}', '{"age":-9223372036854775808.0}',
@@ -468,6 +676,147 @@ class SchemagenIntegerBoundaryTest(unittest.TestCase):
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, schema, harness) self._compile_harness(tmpdir, schema, harness)
def test_integer_no_quadratic_leading_zero_loop(self):
"""Regression test for issue #34: leading-zero stripping must not be quadratic."""
with tempfile.TemporaryDirectory() as tmpdir:
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump({"type": "integer"}, fp)
result = subprocess.run(
[sys.executable, SCRIPT, schema_path],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(result.returncode, 0, msg=result.stderr)
self.assertIn("parseJsonInt64", result.stdout)
self.assertNotIn("digits.erase(digits.begin())", result.stdout)
def test_integer_large_fractional_leading_zeros(self):
"""Numbers with many leading fractional zeros must parse correctly."""
if not self.compiler:
self.skipTest("C++ compiler not available")
harness = textwrap.dedent(
"""
#include "gen.h"
#include <cstdio>
#include <string>
int main() {
const int n = 100000;
std::string s = std::string("0.") + std::string(n - 1, '0') + "1e" + std::to_string(n);
test_schema::RootBuilder b;
WeaselJsonStatus st = b.feed(s.data(), static_cast<int>(s.size()));
st = b.finish();
if (st != WeaselJson_OK) {
std::printf("expected OK, got %d\\n", st);
return 1;
}
if (b.take() != 1) {
std::printf("expected value 1\\n");
return 2;
}
return 0;
}
"""
)
with tempfile.TemporaryDirectory() as tmpdir:
self._compile_harness(tmpdir, {"type": "integer"}, harness)
class SchemagenStringEscapeTest(unittest.TestCase):
"""Regression tests for issue #35: control characters in string literals."""
def setUp(self):
self.repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
self.include_dir = os.path.join(self.repo_root, "include")
self.compiler = shutil.which("c++")
def generate_and_compile(self, schema):
"""Run schemagen on schema and syntax-check the resulting header."""
with tempfile.TemporaryDirectory() as tmpdir:
schema_path = os.path.join(tmpdir, "schema.json")
with open(schema_path, "w") as fp:
json.dump(schema, fp)
header_path = os.path.join(tmpdir, "gen.h")
cmd = [
sys.executable,
SCRIPT,
schema_path,
"-o",
header_path,
"--namespace",
"test_schema",
]
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
self.assertEqual(result.returncode, 0, msg=result.stderr)
if self.compiler:
cpp_path = os.path.join(tmpdir, "test.cpp")
with open(cpp_path, "w") as fp:
fp.write(
'#include "gen.h"\n'
"int main() {\n"
" test_schema::RootBuilder b;\n"
" test_schema::Root r = b.take();\n"
" (void)r;\n"
"}\n"
)
comp = subprocess.run(
[
self.compiler,
"-std=c++20",
"-fsyntax-only",
"-I",
self.include_dir,
"-I",
tmpdir,
cpp_path,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(comp.returncode, 0, msg=comp.stderr)
with open(header_path) as fp:
return fp.read()
def test_newline_in_property_key(self):
"""A JSON key containing a newline must become a valid C++ literal."""
schema = {"type": "object", "properties": {"a\nb": {"type": "string"}}}
out = self.generate_and_compile(schema)
self.assertIn('// "a\\nb"', out)
self.assertIn('if (key == "a\\nb") return 0;', out)
def test_newline_in_enum_value(self):
"""An enum value containing a newline must become a valid C++ literal."""
schema = {"type": "object", "properties": {"x": {"enum": ["a\nb"]}}}
out = self.generate_and_compile(schema)
self.assertIn('static constexpr const char *X_names[] = { "a\\nb" };', out)
def test_mixed_control_chars_in_enum_value(self):
"""Mixed control characters in an enum value must be escaped."""
schema = {
"type": "object",
"properties": {
"x": {"enum": ["x\ny\rz\tw\vq\x00\x01"]},
},
}
out = self.generate_and_compile(schema)
self.assertIn(
'static constexpr const char *X_names[] = { "x\\ny\\rz\\tw\\vq\\u0000\\u0001" };',
out,
)
def test_backslash_and_quote_still_escaped(self):
"""Existing escaping for backslash and double quote must remain correct."""
schema = {"type": "object", "properties": {'a"b\\c': {"type": "string"}}}
out = self.generate_and_compile(schema)
self.assertIn('// "a\\"b\\\\c"', out)
self.assertIn('if (key == "a\\"b\\\\c") return 0;', out)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+73 -8
View File
@@ -18,6 +18,43 @@ import keyword
import sys import sys
def _escape_cpp_string(s):
"""Return *s* escaped for use inside a C++ double-quoted string literal.
JSON strings may contain control characters; emitting them verbatim into
generated C++ source breaks tokenization. This helper escapes backslashes
and double quotes, maps common control characters to their short escape
sequences, and uses universal character names (\\u00XX) for any other
character below 0x20.
"""
out = []
for ch in s:
cp = ord(ch)
if cp == 0x09:
out.append("\\t")
elif cp == 0x0A:
out.append("\\n")
elif cp == 0x0B:
out.append("\\v")
elif cp == 0x0C:
out.append("\\f")
elif cp == 0x0D:
out.append("\\r")
elif cp == 0x08:
out.append("\\b")
elif cp == 0x07:
out.append("\\a")
elif cp < 0x20:
out.append(f"\\u{cp:04X}")
elif ch == "\\":
out.append("\\\\")
elif ch == '"':
out.append('\\"')
else:
out.append(ch)
return "".join(out)
class GenError(Exception): class GenError(Exception):
pass pass
@@ -235,6 +272,24 @@ class Builder:
"""Names generated internally that must not collide with user types.""" """Names generated internally that must not collide with user types."""
return name in ("Root", "RootScalar") return name in ("Root", "RootScalar")
@staticmethod
def _array_reaches(target, ty):
"""Return True if `target` can be reached from `ty` by following
TArr element types. This detects self-referential array cycles that
cannot be expressed as C++ structs."""
seen = set()
stack = [ty]
while stack:
cur = stack.pop()
if cur is target:
return True
if id(cur) in seen:
continue
seen.add(id(cur))
if isinstance(cur, TArr):
stack.append(cur.elem)
return False
def ref_name(self, ref): def ref_name(self, ref):
if not ref.startswith("#/"): if not ref.startswith("#/"):
raise GenError(f"only local $ref supported, got: {ref}") raise GenError(f"only local $ref supported, got: {ref}")
@@ -322,6 +377,12 @@ class Builder:
) )
t.elem = elem t.elem = elem
t.elem_nullable = elem_nullable t.elem_nullable = elem_nullable
# Self-referential array cycles (directly or through a chain of
# array definitions) cannot be represented as a C++ value type.
if self._array_reaches(t, elem):
raise GenError(
"recursive array type is not supported: " f"{defname or hint!r}"
)
return result return result
scalar = { scalar = {
@@ -618,7 +679,8 @@ namespace {ns} {{"""
and f.ty.kind in ("int", "dbl", "bool") and f.ty.kind in ("int", "dbl", "bool")
): ):
init = " = 0" if f.ty.kind != "bool" else " = false" init = " = 0" if f.ty.kind != "bool" else " = false"
out.append(f' {store} {f.cpp}{init}; // "{f.key}"') esc_key = _escape_cpp_string(f.key)
out.append(f' {store} {f.cpp}{init}; // "{esc_key}"')
out.append("};") out.append("};")
out.append("") out.append("")
return "\n".join(out) return "\n".join(out)
@@ -746,7 +808,7 @@ namespace {ns} {{"""
for name, obj in self.b.objects.items(): for name, obj in self.b.objects.items():
lines.append(f" case Kind::{name}:") lines.append(f" case Kind::{name}:")
for i, fld in enumerate(obj.fields): for i, fld in enumerate(obj.fields):
esc = fld.key.replace("\\", "\\\\").replace('"', '\\"') esc = _escape_cpp_string(fld.key)
lines.append(f' if (key == "{esc}") return {i};') lines.append(f' if (key == "{esc}") return {i};')
lines.append(" return -1;") lines.append(" return -1;")
lines.append(" default: return -1;") lines.append(" default: return -1;")
@@ -829,10 +891,7 @@ namespace {ns} {{"""
def _enum_name_arrays(self): def _enum_name_arrays(self):
out = [] out = []
for e in self.b.enums.values(): for e in self.b.enums.values():
lits = ", ".join( lits = ", ".join('"' + _escape_cpp_string(v) + '"' for v in e.values)
'"' + v.replace("\\", "\\\\").replace('"', '\\"') + '"'
for v in e.values
)
out.append( out.append(
f" static constexpr const char *{e.name}_names[] = {{ {lits} }};" f" static constexpr const char *{e.name}_names[] = {{ {lits} }};"
) )
@@ -892,6 +951,10 @@ public:
explicit RootBuilder(int stackSize = 1024) {{ explicit RootBuilder(int stackSize = 1024) {{
cb_ = makeCallbacks(); cb_ = makeCallbacks();
parser_ = WeaselJsonParser_create(stackSize, &cb_, this, 0); parser_ = WeaselJsonParser_create(stackSize, &cb_, this, 0);
if (!parser_) {{
error_ = true;
return;
}}
{self._ctor_body()} {self._ctor_body()}
}} }}
~RootBuilder() {{ if (parser_) WeaselJsonParser_destroy(parser_); }} ~RootBuilder() {{ if (parser_) WeaselJsonParser_destroy(parser_); }}
@@ -996,8 +1059,10 @@ private:
}} }}
int64_t finalExp = exp - fracDigits + trim; int64_t finalExp = exp - fracDigits + trim;
if (finalExp < 0) return false; if (finalExp < 0) return false;
while (!digits.empty() && digits.front() == '0') digits.erase(digits.begin()); size_t leadingZeros = 0;
if (digits.empty()) {{ out = 0; return true; }} while (leadingZeros < digits.size() && digits[leadingZeros] == '0') ++leadingZeros;
if (leadingZeros == digits.size()) {{ out = 0; return true; }}
if (leadingZeros > 0) digits.erase(0, leadingZeros);
constexpr uint64_t kMaxNeg = 9223372036854775808ULL; constexpr uint64_t kMaxNeg = 9223372036854775808ULL;
constexpr uint64_t kMaxPos = 9223372036854775807ULL; constexpr uint64_t kMaxPos = 9223372036854775807ULL;
+10
View File
@@ -140,6 +140,7 @@ struct Parser3 {
std::ignore = push({N_VALUE, N_WHITESPACE, T_EOF}); std::ignore = push({N_VALUE, N_WHITESPACE, T_EOF});
inKey = false; inKey = false;
rejected = false; rejected = false;
overflowed = false;
utf8Codepoint = 0; utf8Codepoint = 0;
utf16Surrogate = 0; utf16Surrogate = 0;
minCodepoint = 0; minCodepoint = 0;
@@ -163,6 +164,7 @@ struct Parser3 {
Utf8Dfa strDfa; Utf8Dfa strDfa;
bool inKey = false; bool inKey = false;
bool rejected = false; bool rejected = false;
bool overflowed = false;
#ifndef HAS_MUSTTAIL #ifndef HAS_MUSTTAIL
char *stashBufForTrampoline; char *stashBufForTrampoline;
@@ -1074,6 +1076,10 @@ inline WeaselJsonStatus Parser3::parse(char *buf, int len) {
return WeaselJson_REJECT; return WeaselJson_REJECT;
} }
if (this->overflowed) [[unlikely]] {
return WeaselJson_OVERFLOW;
}
if (len < 0) [[unlikely]] { if (len < 0) [[unlikely]] {
this->rejected = true; this->rejected = true;
return WeaselJson_REJECT; return WeaselJson_REJECT;
@@ -1087,6 +1093,8 @@ inline WeaselJsonStatus Parser3::parse(char *buf, int len) {
symbolTables.continuations[top()](this, buf, buf + len); symbolTables.continuations[top()](this, buf, buf + len);
if (status == WeaselJson_REJECT) { if (status == WeaselJson_REJECT) {
this->rejected = true; this->rejected = true;
} else if (status == WeaselJson_OVERFLOW) {
this->overflowed = true;
} }
return WeaselJsonStatus(status); return WeaselJsonStatus(status);
#else #else
@@ -1097,6 +1105,8 @@ inline WeaselJsonStatus Parser3::parse(char *buf, int len) {
; ;
if (result == WeaselJson_REJECT) { if (result == WeaselJson_REJECT) {
this->rejected = true; this->rejected = true;
} else if (result == WeaselJson_OVERFLOW) {
this->overflowed = true;
} }
return WeaselJsonStatus(result); return WeaselJsonStatus(result);
#endif #endif
+56
View File
@@ -202,6 +202,62 @@ TEST_CASE("parser3") {
} }
} }
TEST_CASE("overflow state is sticky") {
auto c = noopCallbacks();
// stackSize 3 is exactly big enough to hold reset()'s bootstrap, but too
// small for nested arrays. Overflows must be terminal like rejects: a later
// end-of-data call must never report OK for an incomplete document.
auto *parser = WeaselJsonParser_create(3, &c, nullptr, 0);
REQUIRE(parser != nullptr);
std::string doc = "[[";
REQUIRE(WeaselJsonParser_parse(parser, doc.data(), doc.size()) ==
WeaselJson_OVERFLOW);
// After overflow, the end-of-data call must not return OK.
REQUIRE(WeaselJsonParser_parse(parser, nullptr, 0) != WeaselJson_OK);
// It should keep reporting a terminal failure.
REQUIRE(WeaselJsonParser_parse(parser, nullptr, 0) != WeaselJson_OK);
// Further data chunks must also stay terminal.
std::string more = "]]";
REQUIRE(WeaselJsonParser_parse(parser, more.data(), more.size()) !=
WeaselJson_OK);
WeaselJsonParser_destroy(parser);
}
TEST_CASE("overflow is sticky for nested objects") {
auto c = noopCallbacks();
auto *parser = WeaselJsonParser_create(4, &c, nullptr, 0);
REQUIRE(parser != nullptr);
std::string doc = "{\"a\":{ \"a\":";
REQUIRE(WeaselJsonParser_parse(parser, doc.data(), doc.size()) ==
WeaselJson_OVERFLOW);
REQUIRE(WeaselJsonParser_parse(parser, nullptr, 0) != WeaselJson_OK);
WeaselJsonParser_destroy(parser);
}
TEST_CASE("reset clears overflow state") {
auto c = noopCallbacks();
auto *parser = WeaselJsonParser_create(3, &c, nullptr, 0);
REQUIRE(parser != nullptr);
std::string doc = "[[";
REQUIRE(WeaselJsonParser_parse(parser, doc.data(), doc.size()) ==
WeaselJson_OVERFLOW);
// After reset the parser should accept a minimal document again.
WeaselJsonParser_reset(parser);
std::string copy = "1";
REQUIRE(WeaselJsonParser_parse(parser, copy.data(), copy.size()) ==
WeaselJson_AGAIN);
REQUIRE(WeaselJsonParser_parse(parser, nullptr, 0) == WeaselJson_OK);
WeaselJsonParser_destroy(parser);
}
TEST_CASE("rejected state is sticky") { TEST_CASE("rejected state is sticky") {
auto c = noopCallbacks(); auto c = noopCallbacks();
auto *parser = WeaselJsonParser_create(1024, &c, nullptr, 0); auto *parser = WeaselJsonParser_create(1024, &c, nullptr, 0);