The create() documentation described how the reserve is implemented (six
extra slots for literals and \uXXXX escapes that straddle a chunk
boundary). That is an implementation detail; the header should only
promise the behaviour callers can rely on, i.e. that the accept/reject/
overflow decision depends on the document's bytes and on stackSize, not
on how they are split across parse() calls. The derivation of the upper
bound lives next to kTransientSymbols in src/parser3.h, where it can be
kept in sync with the pushes.
Also make the chunking test cover every split point, not just fixed
strides: each document is split in two at every possible offset, so every
literal, escape and surrogate pair in the fixture straddles a chunk
boundary. Verified that this fails on the pre-fix parser ("true",
stackSize 3, cut 1: whole buffer accepts, two chunks overflow).
WeaselJson_OVERFLOW is documented as "json is too deeply nested", and
stackSize as a nesting-depth budget. In practice the pushdown stack also
held tokenizer progress for a `true`/`false`/`null` literal or a \uXXXX
escape that straddled the end of a chunk: the whole-buffer path consumes
those characters inline and pushes nothing, while the slow path pushed up
to six symbols (a surrogate pair) onto the same stack. So the stackSize a
document needed depended on where the parse() boundaries fell: e.g. "true"
was accepted at stackSize 3 when fed whole but returned the sticky
WeaselJson_OVERFLOW when fed one byte at a time.
Reserve slots for those symbols in addition to the supplied stackSize,
as a fixed upper bound rather than by changing the encoding: the largest
single push of transient symbols is t_hex2()'s six symbols for a surrogate
pair, and no transient symbol is ever pushed while another one is on the
stack, so kTransientSymbols = 6 is enough. Nesting pushes are still bounded
by stackSize, so OVERFLOW now means exactly what the header says and the
accept/reject/overflow decision is a function of the input bytes and
stackSize only.
Document the reserve in weaseljson.h and cover the property with tests:
a doctest sweeping stackSize 3..16 over documents whose literals, escapes
and surrogate pairs land on every chunk boundary, and a fuzz harness that
now sweeps small stack sizes and requires the status of every chunking
(stride 1..15 plus every single split point) to match the whole-buffer
status exactly.
`Parser3::parse` previously formed `buf + len` immediately, so passing a
negative `len` from the C API caused undefined pointer arithmetic. Add an
explicit `len < 0` check that returns `WeaselJson_REJECT` (and makes the
rejected state sticky) before any `buf + len` computation.
Also document the non-negative length precondition in the public header
and add a regression test.
Closes#24
A stackSize smaller than the bootstrap symbols pushed by reset() (or a
negative one) left the parser with an empty stack: reset() swallowed the
push() overflow via std::ignore, and the first parse() then read past the
empty stack (top() dereferences *(stackPtr-1)), crashing. Detect this in
create() by checking empty() after construction and returning null, and
guard against negative stackSize wrapping the allocation size.
Also fix an incremental-build bug: the ld -r bundle step only had an
order-only dependency on the object library under Ninja, so editing
lib.cpp rebuilt the object but never relinked the libraries. Add the
object files to DEPENDS so the bundle (and the .so/.a) rebuild on change.
weaselbot
andrew