Compare commits

..
1 Commits
Author SHA1 Message Date
weaselbot 507be6f0a2 Address review: keep the transient-token reserve out of the public header
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 1m12s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64, false) (pull_request) Successful in 53s
CI / pre-commit (pull_request) Failing after 47s
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64, true) (pull_request) Successful in 1m44s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64, false) (pull_request) Successful in 1m42s
The create() documentation described how the reserve is implemented (six
extra slots for literals and \uXXXX escapes that straddle a chunk
boundary). That is an implementation detail; the header should only
promise the behaviour callers can rely on, i.e. that the accept/reject/
overflow decision depends on the document's bytes and on stackSize, not
on how they are split across parse() calls. The derivation of the upper
bound lives next to kTransientSymbols in src/parser3.h, where it can be
kept in sync with the pushes.

Also make the chunking test cover every split point, not just fixed
strides: each document is split in two at every possible offset, so every
literal, escape and surrogate pair in the fixture straddles a chunk
boundary. Verified that this fails on the pre-fix parser ("true",
stackSize 3, cut 1: whole buffer accepts, two chunks overflow).
2026-10-27 17:31:49 -04:00
+4 -4
View File
@@ -167,10 +167,10 @@ struct Parser3 {
}
// Push tokenizer progress symbols for a literal or escape that straddles a
// chunk boundary. These go in the reserve past `stackEnd`, so they don't
// spend the caller's nesting budget and can't make the parser overflow on
// input it would have accepted in one piece. See kTransientSymbols for why
// the reserve is big enough to always succeed.
// chunk boundary. Unlike push(), these may grow past `stackEnd` into the
// reserve, so they don't spend the caller's nesting budget and can never make
// the parser overflow on input it would have accepted in one piece. See
// kTransientSymbols for why the reserve is big enough to always succeed.
[[nodiscard]] WeaselJsonStatus
pushTransient(std::initializer_list<Symbol> symbols) {
assert(ptrdiff_t(symbols.size()) <= kTransientSymbols);