This works around the compiler bug rather than fixing preserve_none semantics: clang on aarch64 lets a preserve_none callee use x19 as its frame base without saving it, so a caller that also uses x19 as its frame base resumes with a clobbered frame (llvm/llvm-project#70577, llvm/llvm-project#177519). It is only reliably observable with ASan, whose larger frames force x19 to be the frame base.
Changes
src/preserve_none.h: define PRESERVE_NONE only when there is a musttail chain to optimize and we are not on aarch64 with ASan. The __SANITIZE_ADDRESS__ spelling is normalized from __has_feature(address_sanitizer) for compilers that do not predefine it. Without musttail the parse() trampoline calls every continuation with an ordinary call, which is exactly the shape that preserve_none must not be combined with (that build returned WeaselJson_REJECT for "").
src/parser3.h: drop PRESERVE_NONE from scan_string()/scan_string_impl(). n_string2() calls them with an ordinary (non-tail) call and keeps buf/bufEnd live across it, so they must preserve their caller's callee-saved registers. This is the exact chain that crashed; on aarch64 the change is performance-neutral (identical instructions/byte in bench3).
src/test.cpp: new "minimal valid documents" test ("", [], {}, whole and chunked) that parses through parser3::Parser3 directly so the continuations are compiled into the test binary and run under its flags. It segfaults in n_string2() without this change in an aarch64 clang + ASan build.
Verification (aarch64, clang 22.1.8 / gcc 16.1.1)
configuration
before
after
issue reproducer, clang++ -O0 -fsanitize=address
SEGV in n_string2
parse=1, eof=0
clang ASan build (-fsanitize=address -O1), ctest
fuzz_corpus SEGV on the first corpus file
25/25 pass (fuzz_corpus, all 1840 corpus files)
ASan build, preserve_none kept but musttail.h stubbed out
WeaselJson_REJECT for ""
25/25 pass
new "minimal valid documents" test, ASan build, this branch's src/test.cpp against unpatched headers
SIGSEGV in n_string2
passes
clang Release (no sanitizer), ctest
27/27
27/27
gcc Release, ctest
27/27
27/27
conflict_set_{shared,static}_symbols were skipped in the ASan run and python_bindings needs a ./build directory next to the sources (it passes there); both are unrelated to this change.
preserve_none still applies to the normal aarch64 and x86-64 builds (verified by preprocessing: the attribute is present without ASan and absent with it).
Closes #78.
This works around the compiler bug rather than fixing `preserve_none` semantics: clang on aarch64 lets a `preserve_none` callee use x19 as its frame base without saving it, so a caller that also uses x19 as its frame base resumes with a clobbered frame (llvm/llvm-project#70577, llvm/llvm-project#177519). It is only reliably observable with ASan, whose larger frames force x19 to be the frame base.
## Changes
- `src/preserve_none.h`: define `PRESERVE_NONE` only when there is a `musttail` chain to optimize and we are not on aarch64 with ASan. The `__SANITIZE_ADDRESS__` spelling is normalized from `__has_feature(address_sanitizer)` for compilers that do not predefine it. Without `musttail` the `parse()` trampoline calls every continuation with an ordinary call, which is exactly the shape that `preserve_none` must not be combined with (that build returned `WeaselJson_REJECT` for `""`).
- `src/parser3.h`: drop `PRESERVE_NONE` from `scan_string()`/`scan_string_impl()`. `n_string2()` calls them with an ordinary (non-tail) call and keeps `buf`/`bufEnd` live across it, so they must preserve their caller's callee-saved registers. This is the exact chain that crashed; on aarch64 the change is performance-neutral (identical instructions/byte in `bench3`).
- `src/test.cpp`: new "minimal valid documents" test (`""`, `[]`, `{}`, whole and chunked) that parses through `parser3::Parser3` directly so the continuations are compiled into the test binary and run under its flags. It segfaults in `n_string2()` without this change in an aarch64 clang + ASan build.
## Verification (aarch64, clang 22.1.8 / gcc 16.1.1)
| configuration | before | after |
|---|---|---|
| issue reproducer, `clang++ -O0 -fsanitize=address` | SEGV in `n_string2` | `parse=1`, `eof=0` |
| clang ASan build (`-fsanitize=address -O1`), ctest | `fuzz_corpus` SEGV on the first corpus file | 25/25 pass (`fuzz_corpus`, all 1840 corpus files) |
| ASan build, `preserve_none` kept but `musttail.h` stubbed out | `WeaselJson_REJECT` for `""` | 25/25 pass |
| new "minimal valid documents" test, ASan build, this branch's `src/test.cpp` against unpatched headers | SIGSEGV in `n_string2` | passes |
| clang Release (no sanitizer), ctest | 27/27 | 27/27 |
| gcc Release, ctest | 27/27 | 27/27 |
`conflict_set_{shared,static}_symbols` were skipped in the ASan run and `python_bindings` needs a `./build` directory next to the sources (it passes there); both are unrelated to this change.
`preserve_none` still applies to the normal aarch64 and x86-64 builds (verified by preprocessing: the attribute is present without ASan and absent with it).
clang miscompiles preserve_none on aarch64: a preserve_none callee uses
x19 as its frame base without saving it, so a caller that also uses x19
as its frame base resumes with a clobbered frame
(llvm/llvm-project#70577, llvm/llvm-project#177519). ASan's larger
frames make it reliable: n_string2() is left with nulled locals and the
parser segfaults on the valid document `""`.
preserve_none is only an optimization for the musttail chain, so fall
back to the default calling convention where it is not trustworthy:
- Define PRESERVE_NONE only when musttail is available (without it the
parse() trampoline calls every continuation with an ordinary call, and
those calls returned WeaselJson_REJECT for `""`) and we are not
building for aarch64 with ASan.
- Stop annotating scan_string()/scan_string_impl() with it: n_string2()
calls them with an ordinary call and keeps buf and bufEnd live across
it, so they must preserve their caller's callee-saved registers.
Add a regression test for the minimal valid documents, parsing through
parser3::Parser3 directly so the continuations run under the flags of
the test binary; it segfaults in the aarch64+ASan build without this
change.
Closes#78.
All checks were successful
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 1m1s
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #78.
This works around the compiler bug rather than fixing
preserve_nonesemantics: clang on aarch64 lets apreserve_nonecallee use x19 as its frame base without saving it, so a caller that also uses x19 as its frame base resumes with a clobbered frame (llvm/llvm-project#70577, llvm/llvm-project#177519). It is only reliably observable with ASan, whose larger frames force x19 to be the frame base.Changes
src/preserve_none.h: definePRESERVE_NONEonly when there is amusttailchain to optimize and we are not on aarch64 with ASan. The__SANITIZE_ADDRESS__spelling is normalized from__has_feature(address_sanitizer)for compilers that do not predefine it. Withoutmusttailtheparse()trampoline calls every continuation with an ordinary call, which is exactly the shape thatpreserve_nonemust not be combined with (that build returnedWeaselJson_REJECTfor"").src/parser3.h: dropPRESERVE_NONEfromscan_string()/scan_string_impl().n_string2()calls them with an ordinary (non-tail) call and keepsbuf/bufEndlive across it, so they must preserve their caller's callee-saved registers. This is the exact chain that crashed; on aarch64 the change is performance-neutral (identical instructions/byte inbench3).src/test.cpp: new "minimal valid documents" test ("",[],{}, whole and chunked) that parses throughparser3::Parser3directly so the continuations are compiled into the test binary and run under its flags. It segfaults inn_string2()without this change in an aarch64 clang + ASan build.Verification (aarch64, clang 22.1.8 / gcc 16.1.1)
clang++ -O0 -fsanitize=addressn_string2parse=1,eof=0-fsanitize=address -O1), ctestfuzz_corpusSEGV on the first corpus filefuzz_corpus, all 1840 corpus files)preserve_nonekept butmusttail.hstubbed outWeaselJson_REJECTfor""src/test.cppagainst unpatched headersn_string2conflict_set_{shared,static}_symbolswere skipped in the ASan run andpython_bindingsneeds a./builddirectory next to the sources (it passes there); both are unrelated to this change.preserve_nonestill applies to the normal aarch64 and x86-64 builds (verified by preprocessing: the attribute is present without ASan and absent with it).View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.