Work around aarch64+ASan clang preserve_none miscompile (#78) #80

Merged
andrew merged 1 commits from weaselbot/weaseljson:weaselbot/issue-78 into main 2026-11-19 20:32:14 +00:00
Member

Closes #78.

This works around the compiler bug rather than fixing preserve_none semantics: clang on aarch64 lets a preserve_none callee use x19 as its frame base without saving it, so a caller that also uses x19 as its frame base resumes with a clobbered frame (llvm/llvm-project#70577, llvm/llvm-project#177519). It is only reliably observable with ASan, whose larger frames force x19 to be the frame base.

Changes

  • src/preserve_none.h: define PRESERVE_NONE only when there is a musttail chain to optimize and we are not on aarch64 with ASan. The __SANITIZE_ADDRESS__ spelling is normalized from __has_feature(address_sanitizer) for compilers that do not predefine it. Without musttail the parse() trampoline calls every continuation with an ordinary call, which is exactly the shape that preserve_none must not be combined with (that build returned WeaselJson_REJECT for "").
  • src/parser3.h: drop PRESERVE_NONE from scan_string()/scan_string_impl(). n_string2() calls them with an ordinary (non-tail) call and keeps buf/bufEnd live across it, so they must preserve their caller's callee-saved registers. This is the exact chain that crashed; on aarch64 the change is performance-neutral (identical instructions/byte in bench3).
  • src/test.cpp: new "minimal valid documents" test ("", [], {}, whole and chunked) that parses through parser3::Parser3 directly so the continuations are compiled into the test binary and run under its flags. It segfaults in n_string2() without this change in an aarch64 clang + ASan build.

Verification (aarch64, clang 22.1.8 / gcc 16.1.1)

configuration before after
issue reproducer, clang++ -O0 -fsanitize=address SEGV in n_string2 parse=1, eof=0
clang ASan build (-fsanitize=address -O1), ctest fuzz_corpus SEGV on the first corpus file 25/25 pass (fuzz_corpus, all 1840 corpus files)
ASan build, preserve_none kept but musttail.h stubbed out WeaselJson_REJECT for "" 25/25 pass
new "minimal valid documents" test, ASan build, this branch's src/test.cpp against unpatched headers SIGSEGV in n_string2 passes
clang Release (no sanitizer), ctest 27/27 27/27
gcc Release, ctest 27/27 27/27

conflict_set_{shared,static}_symbols were skipped in the ASan run and python_bindings needs a ./build directory next to the sources (it passes there); both are unrelated to this change.

preserve_none still applies to the normal aarch64 and x86-64 builds (verified by preprocessing: the attribute is present without ASan and absent with it).

Closes #78. This works around the compiler bug rather than fixing `preserve_none` semantics: clang on aarch64 lets a `preserve_none` callee use x19 as its frame base without saving it, so a caller that also uses x19 as its frame base resumes with a clobbered frame (llvm/llvm-project#70577, llvm/llvm-project#177519). It is only reliably observable with ASan, whose larger frames force x19 to be the frame base. ## Changes - `src/preserve_none.h`: define `PRESERVE_NONE` only when there is a `musttail` chain to optimize and we are not on aarch64 with ASan. The `__SANITIZE_ADDRESS__` spelling is normalized from `__has_feature(address_sanitizer)` for compilers that do not predefine it. Without `musttail` the `parse()` trampoline calls every continuation with an ordinary call, which is exactly the shape that `preserve_none` must not be combined with (that build returned `WeaselJson_REJECT` for `""`). - `src/parser3.h`: drop `PRESERVE_NONE` from `scan_string()`/`scan_string_impl()`. `n_string2()` calls them with an ordinary (non-tail) call and keeps `buf`/`bufEnd` live across it, so they must preserve their caller's callee-saved registers. This is the exact chain that crashed; on aarch64 the change is performance-neutral (identical instructions/byte in `bench3`). - `src/test.cpp`: new "minimal valid documents" test (`""`, `[]`, `{}`, whole and chunked) that parses through `parser3::Parser3` directly so the continuations are compiled into the test binary and run under its flags. It segfaults in `n_string2()` without this change in an aarch64 clang + ASan build. ## Verification (aarch64, clang 22.1.8 / gcc 16.1.1) | configuration | before | after | |---|---|---| | issue reproducer, `clang++ -O0 -fsanitize=address` | SEGV in `n_string2` | `parse=1`, `eof=0` | | clang ASan build (`-fsanitize=address -O1`), ctest | `fuzz_corpus` SEGV on the first corpus file | 25/25 pass (`fuzz_corpus`, all 1840 corpus files) | | ASan build, `preserve_none` kept but `musttail.h` stubbed out | `WeaselJson_REJECT` for `""` | 25/25 pass | | new "minimal valid documents" test, ASan build, this branch's `src/test.cpp` against unpatched headers | SIGSEGV in `n_string2` | passes | | clang Release (no sanitizer), ctest | 27/27 | 27/27 | | gcc Release, ctest | 27/27 | 27/27 | `conflict_set_{shared,static}_symbols` were skipped in the ASan run and `python_bindings` needs a `./build` directory next to the sources (it passes there); both are unrelated to this change. `preserve_none` still applies to the normal aarch64 and x86-64 builds (verified by preprocessing: the attribute is present without ASan and absent with it).
weaselbot added 1 commit 2026-11-10 20:16:02 +00:00
Work around aarch64+ASan clang preserve_none miscompile
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64, true) (pull_request) Successful in 1m1s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64, false) (pull_request) Successful in 54s
CI / pre-commit (pull_request) Successful in 1m1s
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64, true) (pull_request) Successful in 1m54s
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64, false) (pull_request) Successful in 1m41s
539cc264ff
clang miscompiles preserve_none on aarch64: a preserve_none callee uses
x19 as its frame base without saving it, so a caller that also uses x19
as its frame base resumes with a clobbered frame
(llvm/llvm-project#70577, llvm/llvm-project#177519). ASan's larger
frames make it reliable: n_string2() is left with nulled locals and the
parser segfaults on the valid document `""`.

preserve_none is only an optimization for the musttail chain, so fall
back to the default calling convention where it is not trustworthy:

- Define PRESERVE_NONE only when musttail is available (without it the
  parse() trampoline calls every continuation with an ordinary call, and
  those calls returned WeaselJson_REJECT for `""`) and we are not
  building for aarch64 with ASan.
- Stop annotating scan_string()/scan_string_impl() with it: n_string2()
  calls them with an ordinary call and keeps buf and bufEnd live across
  it, so they must preserve their caller's callee-saved registers.

Add a regression test for the minimal valid documents, parsing through
parser3::Parser3 directly so the continuations run under the flags of
the test binary; it segfaults in the aarch64+ASan build without this
change.

Closes #78.
andrew merged commit 907e064425 into main 2026-11-19 20:32:14 +00:00
andrew deleted branch weaselbot/issue-78 2026-11-19 20:32:19 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: weaselab/weaseljson#80