Compare commits

..
1 Commits
Author SHA1 Message Date
weaselbot 5785912b6c Address review: keep the transient-token reserve out of the public header
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 56s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64, false) (pull_request) Successful in 1m2s
CI / pre-commit (pull_request) Failing after 47s
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64, true) (pull_request) Successful in 1m41s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64, false) (pull_request) Successful in 1m36s
The create() documentation described how the reserve is implemented (six
extra slots for literals and \uXXXX escapes that straddle a chunk
boundary). That is an implementation detail; the header should only
promise the behaviour callers can rely on, i.e. that the accept/reject/
overflow decision depends on the document's bytes and on stackSize, not
on how they are split across parse() calls. The derivation of the upper
bound lives next to kTransientSymbols in src/parser3.h, where it can be
kept in sync with the pushes.

Also make the chunking test cover every split point, not just fixed
strides: each document is split in two at every possible offset, so every
literal, escape and surrogate pair in the fixture straddles a chunk
boundary. Verified that this fails on the pre-fix parser ("true",
stackSize 3, cut 1: whole buffer accepts, two chunks overflow).
2026-10-27 17:31:16 -04:00
2 changed files with 6 additions and 14 deletions

No files matched your search

+2 -10
View File
@@ -12,15 +12,7 @@ jobs:
run: pipx install pre-commit run: pipx install pre-commit
- name: Run pre-commit - name: Run pre-commit
# `pipx install` writes the app to PIPX_BIN_DIR, which is not run: ~/.local/bin/pre-commit run --all-files --show-diff-on-failure
# necessarily ~/.local/bin and not necessarily on PATH, so don't
# hardcode a path that only some runner images happen to use.
run: |
if command -v pre-commit >/dev/null 2>&1; then
pre-commit run --all-files --show-diff-on-failure
else
pipx run pre-commit run --all-files --show-diff-on-failure
fi
build: build:
strategy: strategy:
@@ -84,7 +76,7 @@ jobs:
echo "MinIO credentials not configured; skipping upload" echo "MinIO credentials not configured; skipping upload"
exit 0 exit 0
fi fi
curl -fLs "https://minio.weaselab.dev/public/$(uname -m)/mc.RELEASE.2025-08-13T08-35-41Z" \ curl -Ls "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/mc" \
-o /tmp/mc && chmod +x /tmp/mc -o /tmp/mc && chmod +x /tmp/mc
dest="minio/jenkins/weaseljson/${{ gitea.run_number }}/${{ matrix.name }}/" dest="minio/jenkins/weaseljson/${{ gitea.run_number }}/${{ matrix.name }}/"
/tmp/mc cp build/*.deb "$dest" /tmp/mc cp build/*.deb "$dest"
+4 -4
View File
@@ -167,10 +167,10 @@ struct Parser3 {
} }
// Push tokenizer progress symbols for a literal or escape that straddles a // Push tokenizer progress symbols for a literal or escape that straddles a
// chunk boundary. Unlike push(), these may grow past `stackEnd` into the // chunk boundary. These go in the reserve past `stackEnd`, so they don't
// reserve, so they don't spend the caller's nesting budget and can never make // spend the caller's nesting budget and can't make the parser overflow on
// the parser overflow on input it would have accepted in one piece. See // input it would have accepted in one piece. See kTransientSymbols for why
// kTransientSymbols for why the reserve is big enough to always succeed. // the reserve is big enough to always succeed.
[[nodiscard]] WeaselJsonStatus [[nodiscard]] WeaselJsonStatus
pushTransient(std::initializer_list<Symbol> symbols) { pushTransient(std::initializer_list<Symbol> symbols) {
assert(ptrdiff_t(symbols.size()) <= kTransientSymbols); assert(ptrdiff_t(symbols.size()) <= kTransientSymbols);