Make SHOW_MEMORY builds work: fix deferred node capacity assert and eraseTree use-after-free #91
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
80f084cc43 |
Compile eraseTree's search path length bookkeeping out of non-SHOW_MEMORY builds
CI / test (arm64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 22, https://minio.weaselab.dev/public/aarch64/msan-toolchain-22.1.8.tar.zst, debug-arm64, ubuntu-latest-arm64) (pull_request) Successful in 5m8s
CI / pre-commit (pull_request) Successful in 6m12s
CI / release (arm64, , ubuntu-latest-arm64) (pull_request) Successful in 3m49s
CI / test (amd64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 21, https://minio.weaselab.dev/public/x86_64/msan-toolchain-21.1.8.tar.zst, debug, ubuntu-latest-amd64) (pull_request) Successful in 4m55s
CI / test (amd64, -DCMAKE_CXX_FLAGS=-DUSE_64_BIT=1, 21, , 64-bit-versions, ubuntu-latest-amd64) (pull_request) Successful in 3m57s
CI / test (amd64, -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, 21, , gcc, ubuntu-latest-amd64) (pull_request) Successful in 3m54s
CI / test (amd64, -DUSE_SIMD_FALLBACK=ON, 21, , simd-fallback, ubuntu-latest-amd64) (pull_request) Successful in 3m34s
CI / release (amd64, -DMSAN_TOOLCHAIN_PATH=/opt/msan, ubuntu-latest-amd64) (pull_request) Successful in 5m56s
CI / coverage (pull_request) Successful in 3m58s
`eraseTree` only carries the search path lengths of queued nodes around so that the SHOW_MEMORY key accounting doesn't have to walk `parent` pointers of nodes that have already been released. Put the lengths on a parallel stack that only exists when SHOW_MEMORY is enabled, so that other builds keep the upstream worklist of bare `Node *` and don't compute anything extra. |
||
|
|
c0bd10effa |
Drop redundant coverage exclusion markers in the excluded tail region
CI / test (arm64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 22, https://minio.weaselab.dev/public/aarch64/msan-toolchain-22.1.8.tar.zst, debug-arm64, ubuntu-latest-arm64) (pull_request) Successful in 3m48s
CI / release (arm64, , ubuntu-latest-arm64) (pull_request) Successful in 3m18s
CI / pre-commit (pull_request) Successful in 2m15s
CI / test (amd64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 21, https://minio.weaselab.dev/public/x86_64/msan-toolchain-21.1.8.tar.zst, debug, ubuntu-latest-amd64) (pull_request) Successful in 3m54s
CI / test (amd64, -DCMAKE_CXX_FLAGS=-DUSE_64_BIT=1, 21, , 64-bit-versions, ubuntu-latest-amd64) (pull_request) Successful in 3m23s
CI / test (amd64, -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, 21, , gcc, ubuntu-latest-amd64) (pull_request) Successful in 3m21s
CI / test (amd64, -DUSE_SIMD_FALLBACK=ON, 21, , simd-fallback, ubuntu-latest-amd64) (pull_request) Successful in 3m24s
CI / release (amd64, -DMSAN_TOOLCHAIN_PATH=/opt/msan, ubuntu-latest-amd64) (pull_request) Successful in 5m44s
CI / coverage (pull_request) Successful in 3m54s
`getNodeSize`'s unreachable `default:` sits inside ConflictSet.cpp's
`GCOVR_EXCL_START`/`GCOVR_EXCL_STOP` block, so the `GCOVR_EXCL_LINE` markers on
those two lines are redundant: they cannot change the report, but they make
gcovr print
(WARNING) GCOVR_EXCL_LINE found on line 6002 in excluded region started on
line 5535
once per line for each of the four reports the coverage job writes. Drop them,
matching the rest of the excluded tail, which carries no exclusion markers.
Context: the previous coverage job for this branch did not fail in any of its
steps. Build, Test (5754/5754 passed), Coverage report (`gcovr
--fail-under-line 100` reported 100% for ConflictSet.cpp, LongestCommonPrefix.h
and Metrics.h), the upload and the test summary all reported success; the job
was marked failed while tearing the job container down. The post-actions
(actions/cache, actions/checkout) could no longer `docker exec` into the
container and the runner's rootless podman socket was gone ("Cannot connect to
the Docker daemon at unix:///run/user/992/podman/podman.sock", "Error while stop
job container"); the actions/cache service on 192.168.0.110:41379 had been
refusing connections for the whole run.
Verified locally on this commit: Debug, gcc and `--coverage` builds of the
coverage job's configuration; 5754/5754 tests pass; merged line coverage (union
over the library, SHOW_MEMORY and fuzz-driver objects, honouring the
GCOVR_EXCL_START/STOP regions) is 100% for all three measured files, and no
GCOVR_EXCL_LINE markers remain inside an excluded region.
|
||
|
|
df986810ff |
Give the SHOW_MEMORY instrumentation a single definition per function
CI / pre-commit (pull_request) Successful in 2m8s
CI / test (arm64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 22, https://minio.weaselab.dev/public/aarch64/msan-toolchain-22.1.8.tar.zst, debug-arm64, ubuntu-latest-arm64) (pull_request) Successful in 3m49s
CI / test (amd64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 21, https://minio.weaselab.dev/public/x86_64/msan-toolchain-21.1.8.tar.zst, debug, ubuntu-latest-amd64) (pull_request) Successful in 3m54s
CI / release (arm64, , ubuntu-latest-arm64) (pull_request) Successful in 3m25s
CI / test (amd64, -DCMAKE_CXX_FLAGS=-DUSE_64_BIT=1, 21, , 64-bit-versions, ubuntu-latest-amd64) (pull_request) Successful in 3m22s
CI / test (amd64, -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, 21, , gcc, ubuntu-latest-amd64) (pull_request) Successful in 3m18s
CI / test (amd64, -DUSE_SIMD_FALLBACK=ON, 21, , simd-fallback, ubuntu-latest-amd64) (pull_request) Successful in 3m25s
CI / release (amd64, -DMSAN_TOOLCHAIN_PATH=/opt/msan, ubuntu-latest-amd64) (pull_request) Successful in 5m50s
CI / coverage (pull_request) Failing after 3m10s
The coverage job compiles ConflictSet.cpp twice: once without SHOW_MEMORY,
like the rest of the build, and once with it for the SHOW_MEMORY test. Each
config previously saw a *different* definition of the accounting helpers:
no-op `constexpr` stubs near the top of the file when SHOW_MEMORY was off,
and the real definitions at the bottom when it was on.
gcovr reads the .gcda files for a source file and merges them, and its
default (strict) function merge mode asserts that a function name always
maps to one line. Since `addNode`, `removeNode`, `addKey`, both `removeKey`
overloads and `getSearchPathLength` were reported on two different lines,
report generation aborted with
AssertionError: Got function getSearchPathLength(Node*) on multiple
lines: 99, 6002.
Define those functions in exactly one place instead, and compile the
accounting out from within the bodies. Then every configuration reports each
function on the same line and the coverage data can be merged. The empty
calls are inlined away: the Release shared library contains no calls to
them.
With the merge fixed, the accounting lines that only exist in the SHOW_MEMORY
build show up in the report, so the coverage job's 100% line requirement
covers them. Extend the SHOW_MEMORY test to grow a node through every node
type (so `getNodeSize` covers every case) and to erase entries one at a time
via `setOldestVersion` (so `removeKey` and the `internal_setOldestVersion`
cross-check are covered). The remaining `abort()`s in the accounting are
defensive checks that can never be taken on a passing run; mark them
GCOVR_EXCL_LINE like the rest of the unreachable code in this file.
|
||
|
|
bba328f79f |
Fix SHOW_MEMORY crashes on valid input
CI / test (arm64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 22, https://minio.weaselab.dev/public/aarch64/msan-toolchain-22.1.8.tar.zst, debug-arm64, ubuntu-latest-arm64) (pull_request) Successful in 4m8s
CI / release (arm64, , ubuntu-latest-arm64) (pull_request) Successful in 4m9s
CI / pre-commit (pull_request) Successful in 2m9s
CI / test (amd64, -DCMAKE_BUILD_TYPE=Debug -DMSAN_TOOLCHAIN_PATH=/opt/msan, 21, https://minio.weaselab.dev/public/x86_64/msan-toolchain-21.1.8.tar.zst, debug, ubuntu-latest-amd64) (pull_request) Successful in 3m56s
CI / test (amd64, -DCMAKE_CXX_FLAGS=-DUSE_64_BIT=1, 21, , 64-bit-versions, ubuntu-latest-amd64) (pull_request) Successful in 3m29s
CI / test (amd64, -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, 21, , gcc, ubuntu-latest-amd64) (pull_request) Successful in 3m21s
CI / test (amd64, -DUSE_SIMD_FALLBACK=ON, 21, , simd-fallback, ubuntu-latest-amd64) (pull_request) Successful in 3m24s
CI / release (amd64, -DMSAN_TOOLCHAIN_PATH=/opt/msan, ubuntu-latest-amd64) (pull_request) Successful in 5m48s
CI / coverage (pull_request) Failing after 3m26s
`Node::getCapacity()` asserted `!releaseDeferred`, but releasing a deferred node requires its capacity, so any node replacement aborted an assertion- enabled `SHOW_MEMORY` build. `partialKeyCapacity` is not repurposed by `deferRelease`, so there is no reason to reject deferred nodes. Drop the assertion. `eraseTree` released a node before visiting its children, so the `parent` pointers walked by `removeKey` pointed to freed memory (and `deferRelease` repurposes `parent` as a list link). Carry the search path length of each node in the worklist, computed while its parent is still alive, and account for it with the new `removeKey` overload. Add a test covering the SHOW_MEMORY instrumentation, which the rest of the test suite doesn't compile with. |