parser3: reserve stack slots for chunk-split literals and escapes
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 1m3s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64, false) (pull_request) Successful in 54s
CI / pre-commit (pull_request) Failing after 49s
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64, true) (pull_request) Successful in 1m46s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64, false) (pull_request) Successful in 1m36s

WeaselJson_OVERFLOW is documented as "json is too deeply nested", and
stackSize as a nesting-depth budget. In practice the pushdown stack also
held tokenizer progress for a `true`/`false`/`null` literal or a \uXXXX
escape that straddled the end of a chunk: the whole-buffer path consumes
those characters inline and pushes nothing, while the slow path pushed up
to six symbols (a surrogate pair) onto the same stack. So the stackSize a
document needed depended on where the parse() boundaries fell: e.g. "true"
was accepted at stackSize 3 when fed whole but returned the sticky
WeaselJson_OVERFLOW when fed one byte at a time.

Reserve slots for those symbols in addition to the supplied stackSize,
as a fixed upper bound rather than by changing the encoding: the largest
single push of transient symbols is t_hex2()'s six symbols for a surrogate
pair, and no transient symbol is ever pushed while another one is on the
stack, so kTransientSymbols = 6 is enough. Nesting pushes are still bounded
by stackSize, so OVERFLOW now means exactly what the header says and the
accept/reject/overflow decision is a function of the input bytes and
stackSize only.

Document the reserve in weaseljson.h and cover the property with tests:
a doctest sweeping stackSize 3..16 over documents whose literals, escapes
and surrogate pairs land on every chunk boundary, and a fuzz harness that
now sweeps small stack sizes and requires the status of every chunking
(stride 1..15 plus every single split point) to match the whole-buffer
status exactly.
This commit is contained in:
weaselbot committed 2026-10-27 16:20:00 -04:00
1 parent 730a4cd0f6
commit ac619af24d
5 files changed
+183 -64

No files matched your search

+11 -5
View File
@@ -36,7 +36,7 @@ enum WeaselJsonStatus {
WeaselJson_AGAIN,
/** Invalid json */
WeaselJson_REJECT,
/** json is too deeply nested */
/** json is too deeply nested for the parser's stack size */
WeaselJson_OVERFLOW,
/** Tried to call parse on a null parser */
WeaselJson_NULL,
@@ -50,10 +50,16 @@ enum WeaselJsonFlags {
};
/** Create a parser. Increasing stack size increases memory usage but also
* increases the depth of nested json accepted. `callbacks` and `userdata` must
* outlive the returned parser. Returns null if there's insufficient available
* memory, or if `stackSize` is negative or too small to hold a minimal
* document. */
* increases the depth of nested json accepted. `stackSize` is an exact budget:
* whether a document is accepted, rejected or reported as too deeply nested
* depends only on its bytes and on `stackSize`, not on how the bytes are split
* across WeaselJsonParser_parse calls. To get that guarantee, the parser
* allocates 6 slots beyond `stackSize` for tokenizer progress: literals
* ("true", "false", "null") and \uXXXX escapes that straddle the end of a chunk
* record their unmatched characters there instead of spending the nesting
* budget. `callbacks` and `userdata` must outlive the returned parser. Returns
* null if there's insufficient available memory, or if `stackSize` is negative
* or too small to hold a minimal document. */
WeaselJsonParser *WeaselJsonParser_create(int stackSize,
const WeaselJsonCallbacks *callbacks,
void *userdata, int flags);