Compare commits

..
1 Commits
Author SHA1 Message Date
weaselbot 5785912b6c Address review: keep the transient-token reserve out of the public header
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 56s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64, false) (pull_request) Successful in 1m2s
CI / pre-commit (pull_request) Failing after 47s
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64, true) (pull_request) Successful in 1m41s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64, false) (pull_request) Successful in 1m36s
The create() documentation described how the reserve is implemented (six
extra slots for literals and \uXXXX escapes that straddle a chunk
boundary). That is an implementation detail; the header should only
promise the behaviour callers can rely on, i.e. that the accept/reject/
overflow decision depends on the document's bytes and on stackSize, not
on how they are split across parse() calls. The derivation of the upper
bound lives next to kTransientSymbols in src/parser3.h, where it can be
kept in sync with the pushes.

Also make the chunking test cover every split point, not just fixed
strides: each document is split in two at every possible offset, so every
literal, escape and surrogate pair in the fixture straddles a chunk
boundary. Verified that this fails on the pre-fix parser ("true",
stackSize 3, cut 1: whole buffer accepts, two chunks overflow).
2026-10-27 17:31:16 -04:00
+4 -4
View File
@@ -167,10 +167,10 @@ struct Parser3 {
} }
// Push tokenizer progress symbols for a literal or escape that straddles a // Push tokenizer progress symbols for a literal or escape that straddles a
// chunk boundary. Unlike push(), these may grow past `stackEnd` into the // chunk boundary. These go in the reserve past `stackEnd`, so they don't
// reserve, so they don't spend the caller's nesting budget and can never make // spend the caller's nesting budget and can't make the parser overflow on
// the parser overflow on input it would have accepted in one piece. See // input it would have accepted in one piece. See kTransientSymbols for why
// kTransientSymbols for why the reserve is big enough to always succeed. // the reserve is big enough to always succeed.
[[nodiscard]] WeaselJsonStatus [[nodiscard]] WeaselJsonStatus
pushTransient(std::initializer_list<Symbol> symbols) { pushTransient(std::initializer_list<Symbol> symbols) {
assert(ptrdiff_t(symbols.size()) <= kTransientSymbols); assert(ptrdiff_t(symbols.size()) <= kTransientSymbols);