The aarch64 NEON paths in getNodeIndex/getNodeIndexExists, getChildGeq(Node16*), scan16, and checkMaxBetweenExclusiveImpl<Node16> loaded the full 16-element Node16::index array (and the Node48::reverseIndex array via scan16) with NEON intrinsics and then masked the result down to [0, numChildren). Only the in-use slots are initialized; the unused bytes are indeterminate, so the wide loads were undefined behavior in C++ ([basic.indet]) even though the trailing lanes were discarded. MSan reports this on x86-64; on aarch64 it is the same UB but MSan's imprecise modeling doesn't flag it at -O0, so there's no red->green test (as noted in the issue discussion).
This mirrors the existing x86-64 fix (commit 6fed133): implement the index operations in file-level assembly, where loading and operating on indeterminate values is well-defined (it is not UB in assembly).
Add simd_aarch64.S with find_eq_16, find_ge_16, and mask_in_range_16. AArch64 lacks pmovmskb, so (like the prior NEON code) these return a 64-bit "nibble mask" rather than a 16-bit bitmask; the C++ call sites keep their existing nibble-mask arithmetic and only swap the inline NEON load/compare for the assembly call.
The childMaxVersion compares stay in C++ NEON intrinsics, matching x86-64's compare16: those slots are always initialized to zero by the allocator, so the wide loads are defined.
The assembly functions are only ever called directly (never indirectly), so they carry no BTI landing pads; the object is still marked BTI/PAC/GCS-aware with the same aeabi_feature_and_bits attributes the compiler emits for -mbranch-protection=standard, keeping the binary BTI-enabled (and warning-free) under -z force-bti.
CMakeLists.txt builds simd_aarch64.S into the object library and the SIMD test/bench/fuzz targets on aarch64.
Verified on aarch64 (gcc 16 and clang 22): the new assembly matches the original NEON semantics over 2M randomized trials (including wrapping range edges), and the full ctest suite passes (8552/8552, excluding valgrind which doesn't support this CPU) for both the SIMD and USE_SIMD_FALLBACK builds, plus a Release build's symbol-visibility tests.
The aarch64 NEON paths in `getNodeIndex`/`getNodeIndexExists`, `getChildGeq(Node16*)`, `scan16`, and `checkMaxBetweenExclusiveImpl<Node16>` loaded the full 16-element `Node16::index` array (and the `Node48::reverseIndex` array via `scan16`) with NEON intrinsics and then masked the result down to `[0, numChildren)`. Only the in-use slots are initialized; the unused bytes are indeterminate, so the wide loads were undefined behavior in C++ ([basic.indet]) even though the trailing lanes were discarded. MSan reports this on x86-64; on aarch64 it is the same UB but MSan's imprecise modeling doesn't flag it at -O0, so there's no red->green test (as noted in the issue discussion).
This mirrors the existing x86-64 fix (commit 6fed133): implement the index operations in file-level assembly, where loading and operating on indeterminate values is well-defined (it is not UB in assembly).
- Add `simd_aarch64.S` with `find_eq_16`, `find_ge_16`, and `mask_in_range_16`. AArch64 lacks `pmovmskb`, so (like the prior NEON code) these return a 64-bit "nibble mask" rather than a 16-bit bitmask; the C++ call sites keep their existing nibble-mask arithmetic and only swap the inline NEON load/compare for the assembly call.
- The `childMaxVersion` compares stay in C++ NEON intrinsics, matching x86-64's `compare16`: those slots are always initialized to zero by the allocator, so the wide loads are defined.
- The assembly functions are only ever called directly (never indirectly), so they carry no BTI landing pads; the object is still marked BTI/PAC/GCS-aware with the same `aeabi_feature_and_bits` attributes the compiler emits for `-mbranch-protection=standard`, keeping the binary BTI-enabled (and warning-free) under `-z force-bti`.
- `CMakeLists.txt` builds `simd_aarch64.S` into the object library and the SIMD test/bench/fuzz targets on aarch64.
Verified on aarch64 (gcc 16 and clang 22): the new assembly matches the original NEON semantics over 2M randomized trials (including wrapping range edges), and the full `ctest` suite passes (8552/8552, excluding valgrind which doesn't support this CPU) for both the SIMD and `USE_SIMD_FALLBACK` builds, plus a Release build's symbol-visibility tests.
Closes #68
The aarch64 NEON paths in getNodeIndex/getNodeIndexExists,
getChildGeq(Node16*), scan16, and checkMaxBetweenExclusiveImpl<Node16>
loaded the full 16-element Node16::index array (and the Node48
reverseIndex array via scan16) with NEON intrinsics, then masked the
result down to [0, numChildren). Only the in-use slots are initialized;
the unused bytes are indeterminate, so the wide loads were undefined
behavior in C++ (per [basic.indet]) even though the trailing lanes were
discarded. MSan reports this on x86-64; on aarch64 it is the same UB but
MSan's imprecise modeling doesn't flag it at -O0, so there is no red->green
test.
Mirror the existing x86-64 fix (commit 6fed133): implement the index
operations in file-level assembly, where loading and operating on
indeterminate values is well-defined. Add simd_aarch64.S with
find_eq_16, find_ge_16, and mask_in_range_16. AArch64 lacks pmovmskb, so
(like the prior NEON code) these return a 64-bit nibble mask rather than
a 16-bit bitmask; the C++ call sites keep their existing nibble-mask
arithmetic and only swap the inline NEON load/compare for the assembly
call. The childMaxVersion compares stay in C++ NEON intrinsics, matching
x86-64's compare16: those slots are always initialized to zero by the
allocator, so the wide loads are defined.
The assembly functions carry `bti c` landing pads and the same
aeabi_feature_and_bits attributes the compiler emits for
-mbranch-protection=standard, so the object stays BTI/PAC/GCS-compatible
(and warning-free under -z force-bti). CMakeLists.txt builds simd_aarch64.S
into the object library and the SIMD test/bench/fuzz targets on aarch64.
Closes#68
andrew
requested changes 2026-08-03 12:23:27 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
andrew
The aarch64 NEON paths in
getNodeIndex/getNodeIndexExists,getChildGeq(Node16*),scan16, andcheckMaxBetweenExclusiveImpl<Node16>loaded the full 16-elementNode16::indexarray (and theNode48::reverseIndexarray viascan16) with NEON intrinsics and then masked the result down to[0, numChildren). Only the in-use slots are initialized; the unused bytes are indeterminate, so the wide loads were undefined behavior in C++ ([basic.indet]) even though the trailing lanes were discarded. MSan reports this on x86-64; on aarch64 it is the same UB but MSan's imprecise modeling doesn't flag it at -O0, so there's no red->green test (as noted in the issue discussion).This mirrors the existing x86-64 fix (commit
6fed133): implement the index operations in file-level assembly, where loading and operating on indeterminate values is well-defined (it is not UB in assembly).simd_aarch64.Swithfind_eq_16,find_ge_16, andmask_in_range_16. AArch64 lackspmovmskb, so (like the prior NEON code) these return a 64-bit "nibble mask" rather than a 16-bit bitmask; the C++ call sites keep their existing nibble-mask arithmetic and only swap the inline NEON load/compare for the assembly call.childMaxVersioncompares stay in C++ NEON intrinsics, matching x86-64'scompare16: those slots are always initialized to zero by the allocator, so the wide loads are defined.aeabi_feature_and_bitsattributes the compiler emits for-mbranch-protection=standard, keeping the binary BTI-enabled (and warning-free) under-z force-bti.CMakeLists.txtbuildssimd_aarch64.Sinto the object library and the SIMD test/bench/fuzz targets on aarch64.Verified on aarch64 (gcc 16 and clang 22): the new assembly matches the original NEON semantics over 2M randomized trials (including wrapping range edges), and the full
ctestsuite passes (8552/8552, excluding valgrind which doesn't support this CPU) for both the SIMD andUSE_SIMD_FALLBACKbuilds, plus a Release build's symbol-visibility tests.Closes #68
@@ -0,0 +1,82 @@// SIMD operations on potentially-indeterminate Node16::index[16] bytes.// Written in assembly so msan doesn't track the loads, and so that loadingIt's not so msan doesn't track the loads, it's because it's not UB in asm
@@ -0,0 +24,4 @@.globl find_eq_16.type find_eq_16, %functionfind_eq_16:hint 34 // bti cDo we need these? They shouldn't ever be called indirectly