1086 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
andrew	6d9810fe7f	Poison Node48 reverseIndex slot on erase ... When a Node48 loses a child, the vacated reverseIndex slot would otherwise remain defined, letting valgrind miss reads that rely on the stale value. Mark it undefined, mirroring the existing poisoning of the vacated children slot.	2026-08-03 17:38:21 -04:00
andrew	bae2b22e95	Replace SIMD assembly with initialized Node16::index + valgrind client request ... Commit 6fed133 moved SIMD operations on Node16::index into file-level assembly (later extended to aarch64) to avoid reading indeterminate bytes (UB in C++). Revert that in favor of a simpler, architecture-independent fix: - Restore the original HAS_AVX / HAS_ARM_NEON intrinsics at the affected call sites, keeping the newer conflictMask16 helper and scalar fallback. - memset Node16::index in both copy constructors (from Node3 and Node48) so the [numChildren, 16) tail is always initialized, making the full 16-byte SIMD loads well-defined. This also fixes the same latent UB in the NEON path and the Node16->Node16 full-array memcpy. - Issue a VALGRIND_MAKE_MEM_UNDEFINED client request on the tail so valgrind still flags any read that relies on the unused slots rather than numChildren. - Drop simd.h, simd_x86_64.S, simd_aarch64.S, and the CMake ASM wiring.	2026-08-03 17:32:41 -04:00
andrew	b11e92ee0b	Merge pull request 'Move aarch64 Node16 SIMD index loads into assembly' (#76) from weaselbot/conflict-set:weaselbot/issue-68 into main ... Reviewed-on: #76	2026-08-03 13:42:45 +00:00
weaselbot	e02dd41f74	Address review: drop unused bti c landing pads, fix rationale comment	2026-08-03 09:17:28 -04:00
andrew	b0e326aacc	Merge pull request 'ci: run MSan on arm64' (#74) from weaselbot/conflict-set:weaselbot/issue-73 into main ... Reviewed-on: #74	2026-08-03 12:27:49 +00:00
andrew	1179c1f56e	Merge pull request 'Fix ARM NEON scan helpers truncating 64-bit versions' (#75) from weaselbot/conflict-set:weaselbot/issue-69 into main ... Reviewed-on: #75	2026-08-03 12:26:03 +00:00
andrew	bdbac1556d	Merge pull request 'Correct rationale in simd_x86_64.S comment' (#72) from weaselbot/conflict-set:weaselbot/issue-71 into main ... Reviewed-on: #72	2026-08-03 04:59:26 +00:00
andrew	0103837488	Merge pull request 'Run valgrind in arm64 release in CI' (#70) from valgrind-in-arm into main ... Reviewed-on: #70	2026-08-03 04:34:34 +00:00
weaselbot	d040ed63a4	Move aarch64 Node16 SIMD index loads into assembly ... The aarch64 NEON paths in getNodeIndex/getNodeIndexExists, getChildGeq(Node16*), scan16, and checkMaxBetweenExclusiveImpl<Node16> loaded the full 16-element Node16::index array (and the Node48 reverseIndex array via scan16) with NEON intrinsics, then masked the result down to [0, numChildren). Only the in-use slots are initialized; the unused bytes are indeterminate, so the wide loads were undefined behavior in C++ (per [basic.indet]) even though the trailing lanes were discarded. MSan reports this on x86-64; on aarch64 it is the same UB but MSan's imprecise modeling doesn't flag it at -O0, so there is no red->green test. Mirror the existing x86-64 fix (commit 6fed133): implement the index operations in file-level assembly, where loading and operating on indeterminate values is well-defined. Add simd_aarch64.S with find_eq_16, find_ge_16, and mask_in_range_16. AArch64 lacks pmovmskb, so (like the prior NEON code) these return a 64-bit nibble mask rather than a 16-bit bitmask; the C++ call sites keep their existing nibble-mask arithmetic and only swap the inline NEON load/compare for the assembly call. The childMaxVersion compares stay in C++ NEON intrinsics, matching x86-64's compare16: those slots are always initialized to zero by the allocator, so the wide loads are defined. The assembly functions carry `bti c` landing pads and the same aeabi_feature_and_bits attributes the compiler emits for -mbranch-protection=standard, so the object stays BTI/PAC/GCS-compatible (and warning-free under -z force-bti). CMakeLists.txt builds simd_aarch64.S into the object library and the SIMD test/bench/fuzz targets on aarch64. Closes #68	2026-08-02 23:48:28 -04:00
weaselbot	6b1f597d77	Fix ARM NEON scan helpers truncating 64-bit versions ... When USE_64_BIT=1, InternalVersionT stores an int64_t, but the aarch64 NEON fast paths in scan16 (both the indexed Node48 variant and the Node256 variant) and checkMaxBetweenExclusiveImpl(Node16) only copied the low 32 bits of each version into uint32x4_t lanes and compared against a truncated 32-bit readVersion. This produced wrong conflict/commit decisions once any version exceeded 2^32 - 1. Extract the per-version "greater than readVersion" computation into a shared conflictMask16 helper. For 64-bit versions it uses vcgtq_s64 on the full int64 values and narrows the resulting 16-byte mask into the same nibble-packed layout the callers already consume. The 32-bit path is preserved unchanged. Closes #69	2026-08-02 23:35:32 -04:00
weaselbot	70630649d9	ci: run MSan on arm64 ... Add a debug-arm64 matrix entry to the test job that runs the MSan fuzz tests on an arm64 runner. The arm instrumented libc++ toolchain (22.1.8) is built against LLVM 22, so this entry installs clang-22 from apt.llvm.org while the existing amd64 entries keep clang-21. Generalize the test job matrix with runner/arch/llvm_version/msan_url fields so the runner, apt cache key, LLVM toolchain version, and MSan toolchain download URL are selected per matrix entry. Closes #73	2026-08-02 23:28:40 -04:00
weaselbotandandrew	0b12a037c4	Correct rationale in simd_x86_64.S comment ... The SIMD operations are written in assembly because loading and operating on indeterminate values is UB in C++ but well-defined in assembly. msan not tracking the loads is a side effect, not the reason. Closes #71	2026-08-03 02:37:18 +00:00
andrew	79113ecb88	Run valgrind in arm64 release in CI	2026-08-03 02:36:45 +00:00
andrew	5c16b8ee90	Add .note.gnu.property section for CET/CFI to simd_x86_64.S ... The assembly file was missing the GNU property note that records control-flow integrity (CET) support, causing hardening-check to report 'Control flow integrity: no, not found!'. Add the note matching what clang emits with -fcf-protection.	2026-08-02 22:31:56 -04:00
andrew	b19981ee3c	Only pass -stdlib=libc++ for c++	2026-08-02 21:34:22 -04:00
andrew	12650e2132	Update README.md	2026-08-02 21:16:27 -04:00
andrew	6fed133212	Remove UB from indeterminate value handling ... Move SIMD operations on potentially-indeterminate Node16::index bytes into file-level assembly, where loading and operating on indeterminate values is well-defined (unlike C++). Restructure scalar fallback loops to iterate [0, numChildren) instead of [0, kMaxNodes). Fix TrivialSpan construction from indeterminate pointers in check::Job::init and insertPointWritesOrSorted to only construct when end.len > 0. Add MSan toolchain to the debug CI build to catch these issues going forward.	2026-08-02 21:06:53 -04:00
andrew	9d15af772e	Run msan in CI. Closes #66	2026-07-23 15:16:58 -04:00
andrew	22159c38c5	Add Grafana dashboard for server bench metrics	2026-07-23 13:38:56 -04:00
andrew	8afe5ec75b	Add new coverage to corpus	2026-07-14 14:33:51 -04:00
andrew	6eaa0799d5	Update version ... This branch is going to be 0.1.x now	2026-07-14 12:26:13 -04:00
andrew	cf783a0cdb	ci: move LaTeX/paper deps to amd64-only release step	2026-07-14 11:57:52 -04:00
andrew	16bd98ab12	Download mc from our minio ... And remove vestigial dockerfile	2026-07-14 11:54:57 -04:00
andrew	374dcf13b7	ci: trim pre-commit deps; skip LLVM install for gcc matrix	2026-07-14 11:45:05 -04:00
andrew	d96d862cea	Merge pull request 'Fix null pointers passed to memcmp/memcpy in skip_list' (#65) from weaselbot/conflict-set:weaselbot/issue-64 into main ... Reviewed-on: #65	2026-07-13 21:55:52 +00:00
weaselbot	2231c093df	Fix null pointers passed to memcmp/memcpy in skip_list ... In addWrites, the sizing constructor std::vector<KeyInfo>(count * 2) was used instead of reserve(count * 2), leaving count*2 default- constructed (null/empty key) entries that were then compared via operator<, passing nullptr to memcmp. Switch to reserve so only real entries exist. Additionally guard the memcmp/memcpy calls in operator<, SkipList::less, and copyToArena against empty spans (where data() may be nullptr), which is reachable on the first setOldestVersion with an empty removal key. Closes #64	2026-07-13 12:04:22 -04:00
andrew	d479f16204	Merge pull request 'Return 0 instead of -1 from hash_table getBytes()' (#63) from weaselbot/conflict-set:weaselbot/issue-62 into main ... Reviewed-on: #63	2026-07-08 13:58:42 +00:00
weaselbot	dee3a8f640	Return 0 instead of -1 from hash_table getBytes() ... The hash_table implementation cannot accurately track memory usage and was returning -1 from ConflictSet::getBytes() and ConflictSet_getBytes(). That violates the API contract that getBytes() returns a non-negative byte count. Change both entry points to return 0 and document in ConflictSet.h that implementations which do not track memory usage may return 0. Add a regression test in test_conflict_set.py that loads the hash_table implementation and verifies getBytes() is non-negative. Closes #62	2026-07-07 11:00:52 -04:00
andrew	732d19efa1	Merge pull request 'Fix RealDataBench subspan and version API contract violations' (#61) from weaselbot/conflict-set:weaselbot/issue-58 into main ... Reviewed-on: #61	2026-06-29 18:49:58 +00:00
andrew	9449190d02	Merge pull request 'Fix undefined behavior on empty input in strinc() and prefixRange()' (#60) from weaselbot/conflict-set:weaselbot/issue-59 into main ... Reviewed-on: #60	2026-06-29 18:26:04 +00:00
weaselbot	4fcdc5d7e9	Fix RealDataBench subspan and version API contract violations ... - Use subspan count `line.size() - 2` instead of `line.size()` to avoid reading past the line bounds, and guard lines shorter than the two-byte prefix. - Clamp `readVersion` and `setOldestVersion` arguments to `0` so the conflict set never receives negative, non-monotonic versions during the warmup phase. Fixes #58.	2026-06-29 14:01:23 -04:00
weaselbot	63f9a139da	Fix undefined behavior on empty input in strinc() and prefixRange() ... strinc() in ConflictSet.cpp used std::string_view::size() (size_t) and subtracted 1 without first checking for an empty string. For the root node, getSearchPath() returns the empty string, so every debug correctness check underflowed size_t and relied on implementation-defined conversion to signed int. prefixRange() in Bench.cpp had the same loop shape. Although TrivialSpan::size() returns int, on an empty (or all-0xff) key the function then asserted and continued executing, allocating a zero-length buffer and writing before its start. Changes: - In strinc(), initialize index as signed int(str.size()) - 1 so the loop is skipped for empty input, and return ok=false cleanly. - In prefixRange(), initialize index the same way and call std::abort() after the assert so invalid input cannot fall through to an out-of-bounds write. - Replace C-style uint8_t casts with explicit static_casts. Closes #59	2026-06-29 13:52:02 -04:00
andrew	8f9f345c64	Merge pull request 'Keep Python wrapper key buffers alive in WriteRange/ReadRange' (#43) from weaselbot/conflict-set:weaselbot/issue-42 into main ... Reviewed-on: #43 Reviewed-by: andrew <andrew@weaselab.dev>	2026-06-23 01:17:45 +00:00
andrew	b9b2d69dd5	Merge pull request 'Disallow copying ConflictSet in C++98/C++03' (#51) from weaselbot/conflict-set:weaselbot/issue-48 into main ... Reviewed-on: #51	2026-06-22 23:44:59 +00:00
andrew	d3c8f4afc6	Merge pull request 'Include <sys/syscall.h> in ServerBench.cpp for SYS_perf_event_open' (#44) from weaselbot/conflict-set:weaselbot/issue-41 into main ... Reviewed-on: #44	2026-06-22 20:01:43 +00:00
andrew	549724f09e	Merge pull request 'Assert return values in test_update_zero_should_commit / conflict' (#50) from weaselbot/conflict-set:weaselbot/issue-49 into main ... Reviewed-on: #50	2026-06-22 19:29:55 +00:00
andrew	52eb13cc0b	Merge pull request 'Fix CMake Unix Makefiles parallel build race for conflict-set.o' (#52) from weaselbot/conflict-set:weaselbot/issue-47 into main ... Reviewed-on: #52	2026-06-22 19:08:09 +00:00
andrew	f30887f280	Merge pull request 'Fix move-assignment leak and self-assignment in ConflictSet' (#55) from weaselbot/conflict-set:weaselbot/issue-54 into main ... Reviewed-on: #55	2026-06-22 18:40:25 +00:00
andrew	ccd637deab	We already know __cplusplus is defined	2026-06-22 14:18:33 -04:00
weaselbotandandrew	971deb477c	Keep Python key buffers alive in WriteRange/ReadRange ... `write()` and `read()` created _Key objects from ephemeral ctypes arrays backed by local bytearray objects. Once the helpers returned, those local variables were freed, leaving the C library with dangling pointers when addWrites()/check() later read the keys. Store the backing bytearray on the returned WriteRange/ReadRange objects as private `_begin_buf` / `_end_buf` attributes. Python keeps them alive for the lifetime of the range object, so the C pointer is always valid. Closes #42	2026-06-22 14:09:18 -04:00
weaselbotandandrew	ff0722728a	Include <sys/syscall.h> in ServerBench.cpp for SYS_perf_event_open ... `ServerBench.cpp` calls `syscall(SYS_perf_event_open, ...)` but did not include `<sys/syscall.h>`, relying on `<unistd.h>` to transitively provide the `SYS_*` constants. On toolchains where that does not happen, the build fails with `SYS_perf_event_open` not declared. Add the missing include so `server_bench` compiles on any platform that provides `perf_event_open`. Resolves weaselab/conflict-set#41.	2026-06-22 14:09:14 -04:00
weaselbotandandrew	60881419b8	Assert return values in test_update_zero_should_commit / conflict ... Add the missing [Result.COMMIT] / [Result.CONFLICT] assertions so that the two regression tests actually verify the expected conflict-set outcome, not just that the calls do not crash.	2026-06-22 14:09:10 -04:00
weaselbotandandrew	4515af3662	CMake: add target-level dependencies for conflict-set-object ... The custom command that links conflict-set.o depends on $<TARGET_OBJECTS:conflict-set-object>, but that generator expression does not create a target-level dependency. With the Unix Makefiles generator, parallel builds can start building the consuming libraries before the object library's build rule is available, producing: gmake[3]: *** No rule to make target 'CMakeFiles/conflict-set-object.dir/ConflictSet.cpp.o', needed by 'conflict-set.o'. Stop. Add add_dependencies() so that conflict-set and conflict-set-static cannot build until conflict-set-object has produced its object files. Closes #47	2026-06-22 14:09:04 -04:00
weaselbotandandrew	4dc5f7f75c	Fix self-move-assignment and leak in ConflictSet move-assignment ... The user-declared move-assignment operator overwrote `impl` without first destroying the existing implementation object, leaking all memory and resources owned by the left-hand side. Self-move-assignment also set `impl` to nullptr, leaving the object invalid and leaking the old state. Fix all three implementations (ConflictSet.cpp, SkipList.cpp, HashTable.cpp) to guard against self-assignment and to destroy/free the old `impl` before taking ownership of `other.impl`.	2026-06-22 14:08:59 -04:00
weaselbot	7eaac2a184	Make ConflictSet non-copyable in C++98/C++03 ... `ConflictSet(const ConflictSet&)` and `operator=(const ConflictSet&)` were only deleted for C++11 and later. In C++98/C++03 the compiler implicitly generated public copy operations, so copying a ConflictSet shared the opaque `Impl*` and caused a double-free on destruction. Declare both operations private and leave them undefined when `__cplusplus <= 199711L`, matching the standard pre-C++11 idiom for move-only types. Guard the declarations with `defined(__cplusplus)` so they are not exposed to C90 compilation units. Closes #48	2026-06-22 13:43:04 -04:00
andrew	e9c904a86b	CMakeLists.txt: don't require hardening-check --help to exit 0 ... hardening-check --help returns exit code 1, so the previous hardening_check_help_result EQUAL 0 guard skipped parsing the help output entirely. As a result, architecture-specific flags such as --nobranchprotection on x86_64 were never added to the test command. Parse the help output regardless of exit code; the option-string regex checks are sufficient.	2026-06-22 13:05:33 -04:00
andrew	789ae8cbb9	ci: install clang/LLVM 21 in Gitea Actions workflows ... Switch all CI jobs from the distro-packaged clang to the apt.llvm.org clang-21 / llvm-21 toolchain, and register the versioned binaries as alternatives so that CC/CXX=clang/clang++ and tools like llvm-cov and llvm-objcopy use the newer release automatically.	2026-06-22 12:26:19 -04:00
andrew	d70e6a2455	Merge pull request 'Set restype=None for void-returning C functions' (#53) from weaselbot/conflict-set:weaselbot/issue-46 into main ... Reviewed-on: #53	2026-06-22 01:12:47 +00:00
andrew	8a5168f232	CMakeLists: only pass hardening-check arch flags the tool supports ... The amd64 CI runner's hardening-check does not recognize --nobranchprotection, causing the hardening_check test to fail at configure time. Query the tool's help output and only include the architecture-specific skip flags when they are advertised.	2026-06-21 21:07:26 -04:00
weaselbotandandrew	742d920aa1	Set restype=None for void-returning C functions in conflict_set.py ... ConflictSet_check, ConflictSet_addWrites, ConflictSet_setOldestVersion, and ConflictSet_destroy return void in C, but the Python wrapper left their ctypes restype at the default c_int. Set restype = None for each to match the C API contract and avoid undefined behavior from reading the return register of void functions.	2026-06-21 19:25:27 -04:00