Saturate and gate all llvm-cov coverage metrics

Require 100% region, function, instantiation, line, branch, and MC/DC
coverage in run-coverage.sh. Enable MC/DC instrumentation and rewrite the
two decisions that Clang could not instrument cleanly: the crossed-room
ternary, and the nested boolean under negation in crossedCompleteEndLeOne.
This commit is contained in:
andrew committed 2026-11-18 15:10:02 -05:00
1 parent f23d8a28d3
commit bf149c26b5
4 files changed
+46 -27

No files matched your search

+2 -1
View File
@@ -75,7 +75,8 @@ proof.
differential fuzzing, and by a deterministic finite sweep of small differential fuzzing, and by a deterministic finite sweep of small
readable instances; `cpp/README.md` describes the model, the tests, the readable instances; `cpp/README.md` describes the model, the tests, the
coverage check, and the benchmark. The summarised implementation has 100% coverage check, and the benchmark. The summarised implementation has 100%
line and branch coverage via `cpp/run-coverage.sh`. coverage on every metric `llvm-cov` reports (region, function, instantiation,
line, branch, and MC/DC) via `cpp/run-coverage.sh`.
## Continuous integration ## Continuous integration
+8 -7
View File
@@ -25,7 +25,7 @@ The Lean build is untouched.
| `tests/fuzz_raw.cpp` | libFuzzer safety test on arbitrary instances under ASan/UBSan. | | `tests/fuzz_raw.cpp` | libFuzzer safety test on arbitrary instances under ASan/UBSan. |
| `tests/bench.cpp` | nanobench literal-vs-fast benchmark. | | `tests/bench.cpp` | nanobench literal-vs-fast benchmark. |
| `run-fuzz.sh` | Rebuilds and runs both fuzzers for a fixed time. | | `run-fuzz.sh` | Rebuilds and runs both fuzzers for a fixed time. |
| `run-coverage.sh` | Builds with Clang source-based coverage and requires 100% line and branch coverage of `src/fast.cpp`. | | `run-coverage.sh` | Builds with Clang source-based coverage and requires 100% region, function, instantiation, line, branch, and MC/DC coverage of `src/fast.cpp`. |
| `third_party/nanobench.h` | Vendored nanobench (commit `7252da9d96bd0d71f508e9b2813ab1f57fdfaec1`). | | `third_party/nanobench.h` | Vendored nanobench (commit `7252da9d96bd0d71f508e9b2813ab1f57fdfaec1`). |
## Data model ## Data model
@@ -126,12 +126,13 @@ written, it is not a proof of equivalence with `literal.cpp`.
./cpp/run-coverage.sh ./cpp/run-coverage.sh
``` ```
The script configures a Clang source-based-coverage build, runs The script configures a Clang source-based-coverage build with MC/DC
`sweep_test`, and fails unless `src/fast.cpp` has 100% line and branch instrumentation, runs `sweep_test`, and fails unless `src/fast.cpp` has 100%
coverage. `sweep_test` alone covers every region, function, line, and region, function, instantiation, line, branch, and MC/DC coverage. `sweep_test`
branch side of the summarised implementation. The region, line, and branch alone covers every region, function, instantiation, line, branch side, and
totals are not recorded here: they change with the implementation, and the MC/DC condition of the summarised implementation. MC/DC instrumentation requires
coverage run reports the current ones. Clang 18 or newer. The totals are not recorded here: they change with the
implementation, and the coverage run reports the current ones.
Because equivalence is only promised on readable inputs, `src/fast.cpp` deletes Because equivalence is only promised on readable inputs, `src/fast.cpp` deletes
both unreachable control-flow branches and predicate operands that an enclosing both unreachable control-flow branches and predicate operands that an enclosing
+30 -13
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build with Clang source-based coverage and require 100% line and branch # Build with Clang source-based coverage and require 100% region, function,
# coverage of the summarised implementation (`src/fast.cpp`). `sweep_test` # instantiation, line, branch, and MC/DC coverage of the summarised
# deterministically enumerates a finite class of small readable instances and is # implementation (`src/fast.cpp`). `sweep_test` deterministically enumerates a
# sufficient on its own. # finite class of small readable instances and is sufficient on its own.
set -euo pipefail set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -72,7 +72,7 @@ cmake -S "$here" -B "$build" -G Ninja \
-DCMAKE_BUILD_TYPE=Debug \ -DCMAKE_BUILD_TYPE=Debug \
-DCR_BUILD_FUZZERS=OFF \ -DCR_BUILD_FUZZERS=OFF \
-DCR_BUILD_BENCH=OFF \ -DCR_BUILD_BENCH=OFF \
-DCMAKE_CXX_FLAGS="-fprofile-instr-generate -fcoverage-mapping" \ -DCMAKE_CXX_FLAGS="-fprofile-instr-generate -fcoverage-mapping -fcoverage-mcdc" \
-DCMAKE_EXE_LINKER_FLAGS="-fprofile-instr-generate" -DCMAKE_EXE_LINKER_FLAGS="-fprofile-instr-generate"
cmake --build "$build" --target sweep_test cmake --build "$build" --target sweep_test
@@ -86,18 +86,35 @@ LLVM_PROFILE_FILE="$profdir/sweep-%p.profraw" "$build/sweep_test"
report() { report() {
"$cov" report "$build/sweep_test" \ "$cov" report "$build/sweep_test" \
-instr-profile="$profdir/cov.profdata" src/fast.cpp -instr-profile="$profdir/cov.profdata" \
--show-branch-summary \
--show-instantiation-summary \
--show-mcdc-summary \
--show-region-summary \
src/fast.cpp
} }
output="$(report)" output="$(report)"
printf '%s\n' "$output" printf '%s\n' "$output"
line_cover="$(printf '%s\n' "$output" | awk '/fast\.cpp/ { print $10 }')"
branch_cover="$(printf '%s\n' "$output" | awk '/fast\.cpp/ { print $13 }')" # The row fields follow the summary order enabled above:
if [[ "$line_cover" != "100.00%" ]]; then # region, function, instantiation, line, branch, and MC/DC coverage.
echo "fast.cpp line coverage is $line_cover, expected 100.00%" >&2 row="$(printf '%s\n' "$output" | awk '/fast\.cpp/ { print; exit }')"
if [[ -z "$row" ]]; then
echo "coverage report did not contain src/fast.cpp" >&2
exit 1 exit 1
fi fi
if [[ "$branch_cover" != "100.00%" ]]; then read -r -a fields <<< "$row"
echo "fast.cpp branch coverage is $branch_cover, expected 100.00%" >&2 metrics=(region function instantiation line branch MC/DC)
columns=(3 6 9 12 15 18)
failed=0
for i in "${!metrics[@]}"; do
cover="${fields[${columns[$i]}]}"
if [[ "$cover" != "100.00%" ]]; then
echo "fast.cpp ${metrics[$i]} coverage is ${cover:-missing}, expected 100.00%" >&2
failed=1
fi
done
if [[ "$failed" -ne 0 ]]; then
exit 1 exit 1
fi fi
echo "fast.cpp: 100.00% line and branch coverage" echo "fast.cpp: 100.00% region, function, instantiation, line, branch, and MC/DC coverage"
+6 -6
View File
@@ -652,10 +652,9 @@ struct Impl {
: c.r->begin.key; : c.r->begin.key;
return mk(std::move(begin), after(o.firstK())); return mk(std::move(begin), after(o.firstK()));
} }
OptKeyView begin = bool beginAtEnd = optLt(c.r->end_.key, c.r->begin.key) &&
(optLt(c.r->end_.key, c.r->begin.key) && closedFormCrossedRoom(c, o)) closedFormCrossedRoom(c, o);
? c.r->end_.key OptKeyView begin = beginAtEnd ? c.r->end_.key : c.r->begin.key;
: c.r->begin.key;
return mk(std::move(begin), topKey()); return mk(std::move(begin), topKey());
} }
@@ -685,8 +684,9 @@ struct Impl {
bool crossedCompleteEndLeOne(const Ctx& c) const { bool crossedCompleteEndLeOne(const Ctx& c) const {
if (c.r->end_.offset == 1 && bSlack(c) < c.below) if (c.r->end_.offset == 1 && bSlack(c) < c.below)
return true; return true;
return c.r->end_.offset <= 0 && if (c.r->end_.offset > 0) return false;
!(c.below <= bSlack(c) && clampNonposBottom(c)); bool clampedBottom = c.below <= bSlack(c) && clampNonposBottom(c);
return !clampedBottom;
} }
// The overshoot branch above already covered the offset-zero overshoot // The overshoot branch above already covered the offset-zero overshoot