After the parser returns WeaselJson_REJECT once, subsequent calls to
WeaselJsonParser_parse (including the final len==0 EOF call) must keep
returning WeaselJson_REJECT instead of potentially reporting OK.
- Add a `rejected` flag to Parser3.
- Clear the flag in reset().
- Check the flag at the start of parse() and immediately return REJECT.
- Set the flag whenever a continuation returns REJECT.
Add a test covering the exact reproduction from issue #12.
Remove the standalone "Run schemagen tests" workflow step and instead
add the big-schema regression test to contrib/schemagen/CMakeLists.txt
so ctest picks it up alongside schemagen_example. Update README to
document both `ctest` and the convenience `./run_tests.sh`.
Closes#3
Replace the 32-bit `uint32_t seen` bitmask with a `std::vector<uint64_t>`
bitset sized to the actual field count. `requiredMask` is now a vector of
the same word count, and required-field checks use per-word masking so that
optional fields do not cause false rejections.
Adds big.schema.json + test_big.cpp regression tests covering the issue
reproducer (40 required properties, missing/duplicate at index 32), plus
run_tests.sh to exercise both test_gen.cpp and test_big.cpp.
Fixes#3
Addresses review feedback: keep the schemagen-specific CMake rules
close to the tool instead of inline in the top-level CMakeLists.txt.
The subdirectory file is added from the root and guarded by the same
Python3 availability check that was already in use.
Distinct JSON enum values can sanitize to the same C++ identifier
(e.g. "foo-bar" and "foo_bar" both become `foo_bar`), producing an
invalid `enum class` with duplicate constants.
Add `unique_enum_identifiers()` which appends a numeric suffix to
later collisions while preserving enum declaration order, so the
index-to-JSON-value mapping used by the generated parser stays intact.
Also add contrib/schemagen/test_schemagen.py and wire the schemagen
Python tests plus the existing example.schema.json/test_gen.cpp example
into ctest via CMakeLists.txt.
Closes#4
WeaselJsonParser_reset is documented to restore the parser to its
newly-created state, but reset() only rewound the symbol stack. The
inKey flag and transient DFA/codepoint state from the previous parse
leaked into the next parse, so a top-level string after a mid-key
reset was delivered via on_key_data instead of on_string_data.
Reset inKey to false and clear utf8Codepoint, utf16Surrogate,
minCodepoint, numDfa, and strDfa so the next document starts fresh.
Add a test that reproduces the reported misrouting.
For array types whose items are nullable ({"type": ["T", "null"]}),
the generated builder previously called valueComplete() on cbNull() without
appending anything to the owning vector. Null entries were silently dropped,
so vector indices no longer matched JSON array indices.
Generate isArrayKind() / appendNull() helpers and have cbNull() append a
default-constructed element when the current frame is an array. For
std::optional<T> items this appends an empty optional; for std::unique_ptr<T>
items it appends a null pointer. Add nullable string/integer array fields to
the example schema and test coverage to verify indices are preserved.
Fixes#5
JSON Schema's 'integer' type matches any number with a zero fractional
part, but the generated builder rejected forms like 1e3 or 2.0 because it
only ran std::from_chars<int64_t>. Keep that exact path for plain integer
literals (so large values near INT64_MAX stay exact), and fall back to
parsing as double for the rest, requiring an integral value within int64
range. Add <cmath> for std::trunc and cover the new cases in test_gen.
The no-musttail trampoline (063872e) only rerouted keepGoing dispatch.
Continuations that tail-call each other directly still recursed on the C
stack when MUSTTAIL is a no-op, so deep input overflowed it -- nested
arrays via n_value <-> n_array2, and runs of string escapes via
n_string2 -> n_string2 (the latter isn't even bounded by stackSize).
Add a TAILCALL(fn) macro: a direct musttail tail call when available,
otherwise a bounce to the parse() trampoline (which re-dispatches
continuations[top()] == fn, valid because fn's symbol is already on top
of the stack at every site). Route the 10 direct continuation tail calls
through it. The musttail build is unchanged; the fallback no longer
recurses without bound.
Seeds the n_value<->n_array2 recursion that overflows the C stack in the
no-musttail fallback build (a good mutation seed once stackSize is raised
past the harness default of 1024).
The no-musttail trampoline used WeaselJsonStatus(-1) as a re-dispatch
sentinel, but WeaselJsonStatus has no fixed underlying type, so its valid
range is only [0,3]; forming -1 is undefined behavior in C++. A compiler
may assume the value is in range (e.g. -fstrict-enums) and fold the
trampoline's '== WeaselJsonStatus(-1)' check to false, breaking the loop.
Introduce an internal 'using ContinuationStatus = int' (a WeaselJsonStatus
value, or kBounce) for the Continuation typedef, every continuation/helper
function, and keepGoing. parse() converts back to WeaselJsonStatus only at
the public boundary, where the value is always 0..3. The public enum is
unchanged, so C consumers are unaffected.
A stackSize smaller than the bootstrap symbols pushed by reset() (or a
negative one) left the parser with an empty stack: reset() swallowed the
push() overflow via std::ignore, and the first parse() then read past the
empty stack (top() dereferences *(stackPtr-1)), crashing. Detect this in
create() by checking empty() after construction and returning null, and
guard against negative stackSize wrapping the allocation size.
Also fix an incremental-build bug: the ld -r bundle step only had an
order-only dependency on the object library under Ninja, so editing
lib.cpp rebuilt the object but never relinked the libraries. Add the
object files to DEPENDS so the bundle (and the .so/.a) rebuild on change.
Generates a C++ type and a streaming builder parser from a JSON Schema,
built on weaseljson. The builder copies incoming bytes directly into their
final destinations in the result struct (no intermediate DOM) and hands
back ownership via take() once parsing completes.
Supports objects/structs, required vs optional (std::optional) fields,
nullable types, string enums, arrays, $ref including recursion (broken
with unique_ptr), and additionalProperties (strict reject or skip).
Unsupported schema constructs are rejected at generation time; schema
violations are rejected at parse time.
Older versions of hardening-check (as shipped by some Ubuntu releases)
don't support --nobranchprotection and error out. Probe --help at
configure time and only add the flag when it's available; old versions
that lack the flag also don't check for branch protection, so omitting
it is safe.
On aarch64, char is unsigned by default, so *buf < 0x20 never held for
bytes >= 0x80, causing strDfa.scan to be skipped for multi-byte UTF-8
sequences in batch mode. Cast to int8_t to get the same signed comparison
used by the SIMD path.
Also add aarch64-symbol-imports.txt with the GLIBC 2.17 versioned symbols
imported by the library on that platform.
The Makefile generator does not create a rule for individual object files
when $<TARGET_OBJECTS:...> is used in DEPENDS of add_custom_command. Depending
on the target name instead establishes a proper target-level dependency.
hardening-check tests both CET (x86-only) and branch protection
(arm64-only). Pass the ignore flag for whichever doesn't apply to the
build arch so the test passes on all supported architectures.
weaselbot
andrew