When a parse step returns WeaselJson_OVERFLOW (input too deeply nested for the configured stack), the parser is left in a corrupted state and WeaselJson_OVERFLOW is not made sticky the way WeaselJson_REJECT is. A subsequent WeaselJsonParser_parse(parser, nullptr, 0) (the documented end-of-data call) can then return WeaselJson_OK, accepting an incomplete, invalid document as valid JSON.
Root cause
Two things combine:
Parser3::push (src/parser3.h, lines 118-122) returns WeaselJson_OVERFLOWwithout modifying the stack when there is not enough room:
(Same pattern in n_array3 at lines 552-553, n_object2 at 447-449, n_object3 at 495.) On overflow the popped frame is gone and nothing replaces it, so the pushdown stack no longer represents the grammar state.
Parser3::parse (src/parser3.h, lines 1086-1090 and 1098-1099) only marks the parser terminal when the status is WeaselJson_REJECT:
WeaselJson_OVERFLOW does not set rejected, so the next parse() call dispatches through the now-corrupted stack.
For the minimal case below, after parse("[[", 2) overflows, the stack has lost the N_ARRAY2/N_ARRAY3 frames and top() is T_EOF; the EOF call then runs t_eof, which sees buf == bufEnd and returns WeaselJson_OK.
Reproduction
stackSize = 3 is a supported configuration: WeaselJsonParser_create(3, ...) succeeds (it is exactly big enough to hold reset()'s bootstrap {N_VALUE, N_WHITESPACE, T_EOF}, as confirmed by the create rejects too-small stack test in src/test.cpp).
#include<cstdio>#include<string>#include"callbacks.h"#include"weaseljson.h"intmain(){autoc=noopCallbacks();auto*p=WeaselJsonParser_create(3,&c,nullptr,0);std::stringdoc="[[";autos1=WeaselJsonParser_parse(p,doc.data(),doc.size());printf("parse(\"[[\") = %d (expected OVERFLOW=3)\n",s1);autos2=WeaselJsonParser_parse(p,nullptr,0);printf("parse(EOF) = %d (got OK=0, should be REJECT=2)\n",s2);WeaselJsonParser_destroy(p);return0;}
parse("[[") = 3 (expected OVERFLOW=3)
parse(EOF) = 0 (got OK=0, should be REJECT=2)
[[ is not valid JSON (two unclosed arrays), so WeaselJson_OK ("Accept input") is wrong. The bug is not specific to stackSize = 3; it reproduces with other stack sizes and document shapes (e.g. {"a":{"a":... nested objects at stackSize 4 and 5, and nested arrays at stackSize 3). The exact post-overflow status depends on which frame the corrupted stack happens to expose, which is why it sometimes returns OK and sometimes REJECT.
Expected behavior
WeaselJson_OVERFLOW should be a terminal state: once it is returned, every subsequent WeaselJsonParser_parse call (including the len == 0 end-of-data call) should keep returning WeaselJson_OVERFLOW (or WeaselJson_REJECT), never WeaselJson_OK for input that was never actually accepted. The README states that too-deeply-nested documents "are rejected"; today an incomplete too-deeply-nested document can instead be reported as accepted.
Impact
A caller that feeds data, observes WeaselJson_OVERFLOW, and then issues the documented end-of-data call WeaselJsonParser_parse(parser, nullptr, 0) to finalize can be told the document is valid (WeaselJson_OK) when it is in fact incomplete and invalid. The compareWithSimdjson fuzz harness in src/fuzz.cpp does not catch this because it only calls the EOF finish call when the data call returned WeaselJson_AGAIN, and skips comparison when ours == WeaselJson_OVERFLOW.
## Summary
When a parse step returns `WeaselJson_OVERFLOW` (input too deeply nested for the configured stack), the parser is left in a corrupted state and `WeaselJson_OVERFLOW` is **not** made sticky the way `WeaselJson_REJECT` is. A subsequent `WeaselJsonParser_parse(parser, nullptr, 0)` (the documented end-of-data call) can then return `WeaselJson_OK`, accepting an incomplete, invalid document as valid JSON.
## Root cause
Two things combine:
1. `Parser3::push` (`src/parser3.h`, lines 118-122) returns `WeaselJson_OVERFLOW` *without modifying the stack* when there is not enough room:
```cpp
[[nodiscard]] WeaselJsonStatus push(std::initializer_list<Symbol> symbols) {
if (stackEnd - stackPtr < ptrdiff_t(symbols.size())) [[unlikely]] {
return WeaselJson_OVERFLOW;
}
...
```
Several continuations `pop()` a symbol *before* calling `push()` and propagate the overflow directly, e.g. `n_array2` (`src/parser3.h`, lines 525-528):
```cpp
default:
self->pop();
if (auto s = self->push({N_VALUE, N_ARRAY3})) {
return s;
}
TAILCALL(n_value);
```
(Same pattern in `n_array3` at lines 552-553, `n_object2` at 447-449, `n_object3` at 495.) On overflow the popped frame is gone and nothing replaces it, so the pushdown stack no longer represents the grammar state.
2. `Parser3::parse` (`src/parser3.h`, lines 1086-1090 and 1098-1099) only marks the parser terminal when the status is `WeaselJson_REJECT`:
```cpp
ContinuationStatus status =
symbolTables.continuations[top()](this, buf, buf + len);
if (status == WeaselJson_REJECT) {
this->rejected = true;
}
```
`WeaselJson_OVERFLOW` does not set `rejected`, so the next `parse()` call dispatches through the now-corrupted stack.
For the minimal case below, after `parse("[[", 2)` overflows, the stack has lost the `N_ARRAY2`/`N_ARRAY3` frames and `top()` is `T_EOF`; the EOF call then runs `t_eof`, which sees `buf == bufEnd` and returns `WeaselJson_OK`.
## Reproduction
`stackSize = 3` is a supported configuration: `WeaselJsonParser_create(3, ...)` succeeds (it is exactly big enough to hold `reset()`'s bootstrap `{N_VALUE, N_WHITESPACE, T_EOF}`, as confirmed by the `create rejects too-small stack` test in `src/test.cpp`).
```cpp
#include <cstdio>
#include <string>
#include "callbacks.h"
#include "weaseljson.h"
int main() {
auto c = noopCallbacks();
auto *p = WeaselJsonParser_create(3, &c, nullptr, 0);
std::string doc = "[[";
auto s1 = WeaselJsonParser_parse(p, doc.data(), doc.size());
printf("parse(\"[[\") = %d (expected OVERFLOW=3)\n", s1);
auto s2 = WeaselJsonParser_parse(p, nullptr, 0);
printf("parse(EOF) = %d (got OK=0, should be REJECT=2)\n", s2);
WeaselJsonParser_destroy(p);
return 0;
}
```
Build and run:
```sh
c++ -std=c++20 -Iinclude -Isrc repro.cpp -L build -lweaseljson -o repro -Wl,-rpath,build
./repro
```
Actual output:
```
parse("[[") = 3 (expected OVERFLOW=3)
parse(EOF) = 0 (got OK=0, should be REJECT=2)
```
`[[` is not valid JSON (two unclosed arrays), so `WeaselJson_OK` ("Accept input") is wrong. The bug is not specific to `stackSize = 3`; it reproduces with other stack sizes and document shapes (e.g. `{"a":{"a":...` nested objects at `stackSize` 4 and 5, and nested arrays at `stackSize` 3). The exact post-overflow status depends on which frame the corrupted stack happens to expose, which is why it sometimes returns `OK` and sometimes `REJECT`.
## Expected behavior
`WeaselJson_OVERFLOW` should be a terminal state: once it is returned, every subsequent `WeaselJsonParser_parse` call (including the `len == 0` end-of-data call) should keep returning `WeaselJson_OVERFLOW` (or `WeaselJson_REJECT`), never `WeaselJson_OK` for input that was never actually accepted. The README states that too-deeply-nested documents "are rejected"; today an incomplete too-deeply-nested document can instead be reported as accepted.
## Impact
A caller that feeds data, observes `WeaselJson_OVERFLOW`, and then issues the documented end-of-data call `WeaselJsonParser_parse(parser, nullptr, 0)` to finalize can be told the document is valid (`WeaselJson_OK`) when it is in fact incomplete and invalid. The `compareWithSimdjson` fuzz harness in `src/fuzz.cpp` does not catch this because it only calls the EOF finish call when the data call returned `WeaselJson_AGAIN`, and skips comparison when `ours == WeaselJson_OVERFLOW`.
weaselbot
was assigned by andrew2026-07-13 15:29:04 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
When a parse step returns
WeaselJson_OVERFLOW(input too deeply nested for the configured stack), the parser is left in a corrupted state andWeaselJson_OVERFLOWis not made sticky the wayWeaselJson_REJECTis. A subsequentWeaselJsonParser_parse(parser, nullptr, 0)(the documented end-of-data call) can then returnWeaselJson_OK, accepting an incomplete, invalid document as valid JSON.Root cause
Two things combine:
Parser3::push(src/parser3.h, lines 118-122) returnsWeaselJson_OVERFLOWwithout modifying the stack when there is not enough room:Several continuations
pop()a symbol before callingpush()and propagate the overflow directly, e.g.n_array2(src/parser3.h, lines 525-528):(Same pattern in
n_array3at lines 552-553,n_object2at 447-449,n_object3at 495.) On overflow the popped frame is gone and nothing replaces it, so the pushdown stack no longer represents the grammar state.Parser3::parse(src/parser3.h, lines 1086-1090 and 1098-1099) only marks the parser terminal when the status isWeaselJson_REJECT:WeaselJson_OVERFLOWdoes not setrejected, so the nextparse()call dispatches through the now-corrupted stack.For the minimal case below, after
parse("[[", 2)overflows, the stack has lost theN_ARRAY2/N_ARRAY3frames andtop()isT_EOF; the EOF call then runst_eof, which seesbuf == bufEndand returnsWeaselJson_OK.Reproduction
stackSize = 3is a supported configuration:WeaselJsonParser_create(3, ...)succeeds (it is exactly big enough to holdreset()'s bootstrap{N_VALUE, N_WHITESPACE, T_EOF}, as confirmed by thecreate rejects too-small stacktest insrc/test.cpp).Build and run:
Actual output:
[[is not valid JSON (two unclosed arrays), soWeaselJson_OK("Accept input") is wrong. The bug is not specific tostackSize = 3; it reproduces with other stack sizes and document shapes (e.g.{"a":{"a":...nested objects atstackSize4 and 5, and nested arrays atstackSize3). The exact post-overflow status depends on which frame the corrupted stack happens to expose, which is why it sometimes returnsOKand sometimesREJECT.Expected behavior
WeaselJson_OVERFLOWshould be a terminal state: once it is returned, every subsequentWeaselJsonParser_parsecall (including thelen == 0end-of-data call) should keep returningWeaselJson_OVERFLOW(orWeaselJson_REJECT), neverWeaselJson_OKfor input that was never actually accepted. The README states that too-deeply-nested documents "are rejected"; today an incomplete too-deeply-nested document can instead be reported as accepted.Impact
A caller that feeds data, observes
WeaselJson_OVERFLOW, and then issues the documented end-of-data callWeaselJsonParser_parse(parser, nullptr, 0)to finalize can be told the document is valid (WeaselJson_OK) when it is in fact incomplete and invalid. ThecompareWithSimdjsonfuzz harness insrc/fuzz.cppdoes not catch this because it only calls the EOF finish call when the data call returnedWeaselJson_AGAIN, and skips comparison whenours == WeaselJson_OVERFLOW.