Compare commits

..
Author SHA1 Message Date
andrew af1e2299de Include childType in ChildAndMaxVersion
Tests / Clang total: 3339, passed: 3339
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3339, passed: 3339
Tests / Debug total: 3337, passed: 3337
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-09-23 18:38:06 -07:00
andrew 230e96063d Make children tagged pointers 2024-09-23 17:58:09 -07:00
andrew f41a62471b Add tail-call based interleaving approach
Tests / Clang total: 3339, passed: 3339
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3339, passed: 3339
Tests / Debug total: 3337, passed: 3337
Tests / SIMD fallback total: 3339, passed: 3339
Tests / Release [gcc] total: 3339, passed: 3339
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2482, passed: 2482
Tests / Coverage total: 2511, passed: 2511
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.09% (1846/1863) * Branch Coverage: 67.65% (1447/2139) * Complexity Density: 0.00 * Lines of Code: 1863 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-23 12:52:15 -07:00
andrew d8f85dedc4 Add to corpus
Tests / Clang total: 3339, passed: 3339
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3339, passed: 3339
Tests / Debug total: 3337, passed: 3337
Tests / SIMD fallback total: 3339, passed: 3339
Tests / Release [gcc] total: 3339, passed: 3339
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2482, passed: 2482
Tests / Coverage total: 2511, passed: 2511
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.09% (1846/1863) * Branch Coverage: 67.65% (1447/2139) * Complexity Density: 0.00 * Lines of Code: 1863 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-14 15:02:47 -07:00
andrew 656939560b Add GCOVR_EXCL_LINE on covered line 2024-09-14 14:35:51 -07:00
andrew 5580f9b71d Run script tests under valgrind too
Tests / Clang total: 3307, passed: 3307
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3307, passed: 3307
Tests / Debug total: 3305, passed: 3305
Tests / SIMD fallback total: 3307, passed: 3307
Tests / Release [gcc] total: 3307, passed: 3307
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2458, passed: 2458
Tests / Coverage total: 2487, passed: 2487
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.03% (1846/1864) * Branch Coverage: 67.65% (1447/2139) * Complexity Density: 0.00 * Lines of Code: 1864 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-09-14 13:51:45 -07:00
andrew 628d16b7e6 Annotate zero argument as maybe_unused 2024-09-14 13:47:08 -07:00
andrew d9e4a7d1b6 Remove bogus assert
Tests / Clang total: 3296, passed: 3296
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3296, passed: 3296
Tests / Debug total: 3294, passed: 3294
Tests / SIMD fallback total: 3296, passed: 3296
Tests / Release [gcc] total: 3296, passed: 3296
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |2|0|2|0|:zzz:
Tests / Release [gcc,aarch64] total: 2458, passed: 2458
Tests / Coverage total: 2476, passed: 2476
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.77% (1841/1864) * Branch Coverage: 67.55% (1445/2139) * Complexity Density: 0.00 * Lines of Code: 1864 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-09-14 10:58:48 -07:00
andrew 52201fa4c7 Use signed compare
Tests / Clang total: 3296, failed: 38, passed: 3258
Tests / 64 bit versions total: 3296, passed: 3296
Tests / Debug total: 3294, failed: 76, passed: 3218
Tests / SIMD fallback total: 3296, passed: 3296
Tests / Release [gcc] total: 3296, failed: 38, passed: 3258
Tests / Release [gcc,aarch64] total: 2458, passed: 2458
Tests / Coverage total: 2476, failed: 76, passed: 2400
weaselab/conflict-set/pipeline/head There was a failure building this commit
Unsigned compare is incorrect here
2024-09-13 22:55:22 -07:00
andrew 0814822d82 avx512 implementations for fixupMaxVersion
Tests / Clang total: 3296, failed: 1, passed: 3295
Tests / 64 bit versions total: 3296, passed: 3296
Tests / Debug total: 3294, failed: 1, passed: 3293
Tests / SIMD fallback total: 3296, passed: 3296
Tests / Release [gcc] total: 3296, failed: 1, passed: 3295
Tests / Release [gcc,aarch64] total: 2458, passed: 2458
Tests / Coverage total: 2476, failed: 1, passed: 2475
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-09-13 22:01:56 -07:00
andrew 41df2398e8 Add to corpus 2024-09-13 20:35:52 -07:00
andrew 84c4d0fcba More usages of getNodeIndexExists
Tests / Clang total: 3244, passed: 3244
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3244, passed: 3244
Tests / Debug total: 3242, passed: 3242
Tests / SIMD fallback total: 3244, passed: 3244
Tests / Release [gcc] total: 3244, passed: 3244
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2419, passed: 2419
Tests / Coverage total: 2437, passed: 2437
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.24% (1823/1837) * Branch Coverage: 68.06% (1447/2126) * Complexity Density: 0.00 * Lines of Code: 1837 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-12 22:22:16 -07:00
andrew 6241533dfb Improve codegen for getChildExists(Node{3,16}*, ...) 2024-09-12 22:05:00 -07:00
andrew 0abf6a1ecf Improve Node3 search codegen
Tests / Clang total: 3244, passed: 3244
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3244, passed: 3244
Tests / Debug total: 3242, passed: 3242
Tests / SIMD fallback total: 3244, passed: 3244
Tests / Release [gcc] total: 3244, passed: 3244
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2419, passed: 2419
Tests / Coverage total: 2437, passed: 2437
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.23% (1807/1821) * Branch Coverage: 68.22% (1434/2102) * Complexity Density: 0.00 * Lines of Code: 1821 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-12 17:12:41 -07:00
andrew 867136ff1b Return pointer to next function 2024-09-09 21:59:49 -07:00
andrew 4b8f7320d3 Call function pointer in job 2024-09-09 21:00:31 -07:00
andrew 6628092384 Tinker with interleaveBoundedCyclicList 2024-09-09 20:25:40 -07:00
andrew a0a4f1afea Only compile nanobench once 2024-09-09 20:10:55 -07:00
andrew ca479c03ce Induce a cache miss in interleaving test 2024-09-09 17:55:35 -07:00
andrew 0a2e133ab9 Add InterleavingTest to explore #5 2024-09-09 17:27:58 -07:00
andrew b0b31419b0 Remove vestigial emscripten from cmake
Tests / Clang total: 3244, passed: 3244
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3244, passed: 3244
Tests / Debug total: 3242, passed: 3242
Tests / SIMD fallback total: 3244, passed: 3244
Tests / Release [gcc] total: 3244, passed: 3244
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2419, passed: 2419
Tests / Coverage total: 2437, passed: 2437
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.23% (1803/1817) * Branch Coverage: 68.36% (1426/2086) * Complexity Density: 0.00 * Lines of Code: 1817 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-06 20:29:59 -07:00
andrew 5c0cc1edf5 Add longestCommonPrefix bench
Tests / Clang total: 3244, passed: 3244
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 3244, passed: 3244
Tests / Debug total: 3242, passed: 3242
Tests / SIMD fallback total: 3244, passed: 3244
Tests / Release [gcc] total: 3244, passed: 3244
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2419, passed: 2419
Tests / Coverage total: 2437, passed: 2437
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.23% (1803/1817) * Branch Coverage: 68.36% (1426/2086) * Complexity Density: 0.00 * Lines of Code: 1817 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-05 16:30:58 -07:00
andrew de47aa53b0 Add to corpus 2024-09-05 16:24:41 -07:00
andrew 56893f9702 Update stale comments
Tests / Clang total: 2879, passed: 2879
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 2879, passed: 2879
Tests / Debug total: 2877, passed: 2877
Tests / SIMD fallback total: 2879, passed: 2879
Tests / Release [gcc] total: 2879, passed: 2879
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2146, passed: 2146
Tests / Coverage total: 2163, passed: 2163
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.23% (1803/1817) * Branch Coverage: 68.36% (1426/2086) * Complexity Density: 0.00 * Lines of Code: 1817 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-09-05 10:06:02 -07:00
andrew e2234be10f Update README.md 2024-09-04 13:08:20 -07:00
andrew ce853680f2 Finish checkRangeRightSide conversion 2024-09-04 12:36:43 -07:00
andrew 5c39c1d64f CheckRangeRightSide conversion WIP 2024-09-04 12:25:16 -07:00
andrew 55b73c8ddb Convert CheckRangeRightSide WIP 2024-09-04 12:22:36 -07:00
andrew b9503f8258 Converting CheckRangeRightSide WIP 2024-09-04 12:19:53 -07:00
andrew c4c4531bd3 Simplify 2024-09-04 12:12:54 -07:00
andrew 2037d37c66 checkRangeLeftSide 2024-09-04 12:11:16 -07:00
andrew 6fe6a244af Get CheckRangeLeftSide closer, WIP 2024-09-04 12:08:16 -07:00
andrew 8a4b370e2a Make CheckRange{Left,Right}Side into normal functions, WIP 2024-09-04 12:04:58 -07:00
andrew 394f09f9fb WIP adding metrics to skiplist 2024-08-30 16:06:43 -07:00
andrew 5e06a30357 Add optional microbenchmarks for rezero/scan avx512
Tests / Clang total: 2879, passed: 2879
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 2879, passed: 2879
Tests / Debug total: 2877, passed: 2877
Tests / SIMD fallback total: 2879, passed: 2879
Tests / Release [gcc] total: 2879, passed: 2879
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2146, passed: 2146
Tests / Coverage total: 2163, passed: 2163
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.92% (1834/1854) * Branch Coverage: 66.85% (1496/2238) * Complexity Density: 0.00 * Lines of Code: 1854 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-08-29 12:12:24 -07:00
andrew cb6e4292f2 Add to corpus
Tests / Clang total: 2879, passed: 2879
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 2879, passed: 2879
Tests / Debug total: 2877, passed: 2877
Tests / SIMD fallback total: 2879, passed: 2879
Tests / Release [gcc] total: 2879, passed: 2879
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2146, passed: 2146
Tests / Coverage total: 2163, passed: 2163
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.92% (1834/1854) * Branch Coverage: 66.85% (1496/2238) * Complexity Density: 0.00 * Lines of Code: 1854 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-08-26 14:33:16 -07:00
andrew 154a48ded0 Bump version
Tests / Clang total: 2843, passed: 2843
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / 64 bit versions total: 2843, passed: 2843
Tests / Debug total: 2841, passed: 2841
Tests / SIMD fallback total: 2843, passed: 2843
Tests / Release [gcc] total: 2843, passed: 2843
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 2119, passed: 2119
Tests / Coverage total: 2136, passed: 2136
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.92% (1834/1854) * Branch Coverage: 66.85% (1496/2238) * Complexity Density: 0.00 * Lines of Code: 1854 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-08-26 14:25:13 -07:00
210 changed files with 881 additions and 754 deletions
-306
View File
@@ -1,306 +0,0 @@
name: CI
on: [push, pull_request]
env:
CC: clang
CXX: clang++
jobs:
pre-commit:
runs-on: ubuntu-latest-amd64
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: /var/cache/apt/archives
key: apt-amd64-${{ hashFiles('.gitea/workflows/ci.yml') }}
- name: Install dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y git nodejs pre-commit
- uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Run pre-commit
run: |
git config --global --add safe.directory "$PWD"
pre-commit run --all-files --show-diff-on-failure
test:
strategy:
fail-fast: false
matrix:
include:
- name: 64-bit-versions
cmake_args: -DCMAKE_CXX_FLAGS=-DUSE_64_BIT=1
- name: debug
cmake_args: -DCMAKE_BUILD_TYPE=Debug
- name: simd-fallback
cmake_args: -DUSE_SIMD_FALLBACK=ON
- name: gcc
cmake_args: -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++
runs-on: ubuntu-latest-amd64
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: /var/cache/apt/archives
key: apt-amd64-${{ hashFiles('.gitea/workflows/ci.yml') }}
- name: Install common dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y \
build-essential ccache cmake gcc g++ libc6-dbg \
ninja-build python3 valgrind zstd
sudo curl -Ls "https://minio.weaselab.dev/public/$(uname -m)/mc.RELEASE.2025-08-13T08-35-41Z" \
-o /usr/local/bin/mc && sudo chmod +x /usr/local/bin/mc
- name: Install LLVM toolchain
if: matrix.name != 'gcc'
run: |
. /etc/os-release
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
echo "deb http://apt.llvm.org/${VERSION_CODENAME}/ llvm-toolchain-${VERSION_CODENAME}-21 main" | sudo tee /etc/apt/sources.list.d/llvm.list
sudo apt-get update -qq
sudo apt-get install -y \
clang-21 llvm-21 lld-21 mold
for tool in clang clang++ llvm-ar llvm-nm llvm-ranlib llvm-objcopy llvm-cov llvm-symbolizer lld ld.lld; do
sudo update-alternatives --install /usr/bin/${tool} ${tool} /usr/bin/${tool}-21 100
done
- name: Build
run: |
export CCACHE_DIR="$GITHUB_WORKSPACE/.ccache"
rm -rf build
cmake -S . -B build -G Ninja -DCMAKE_CXX_COMPILER_LAUNCHER=ccache ${{ matrix.cmake_args }}
ninja -C build
ccache -s
- name: Test
run: |
cd build
ctest --no-compress-output --test-output-size-passed 100000 --test-output-size-failed 100000 -T Test -j "$(nproc)" --timeout 90 > /dev/null
- name: Upload test results to MinIO
if: always()
env:
MINIO_ACCESS_KEY: ${{ secrets.MINIO_ACCESS_KEY }}
MC_HOST_minio: https://${{ secrets.MINIO_ACCESS_KEY }}:${{ secrets.MINIO_SECRET_KEY }}@minio.weaselab.dev
run: |
if [ -z "$MINIO_ACCESS_KEY" ]; then
echo "MinIO credentials not configured; skipping upload"
exit 0
fi
zstd build/Testing/*/Test.xml
mc cp build/Testing/*/Test.xml.zst "minio/jenkins/conflict-set/${{ gitea.run_number }}/${{ matrix.name }}/"
- name: Test summary
if: always()
run: |
python3 ctest_summary.py build/Testing/*/Test.xml \
--link "https://minio.weaselab.dev/jenkins/conflict-set/${{ gitea.run_number }}/${{ matrix.name }}/Test.xml.zst" \
| tee -a "$GITHUB_STEP_SUMMARY"
release:
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-latest-amd64
arch: amd64
- runner: ubuntu-latest-arm64
arch: arm64
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: /var/cache/apt/archives
key: apt-${{ matrix.arch }}-${{ hashFiles('.gitea/workflows/ci.yml') }}
- name: Install dependencies
run: |
. /etc/os-release
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
echo "deb http://apt.llvm.org/${VERSION_CODENAME}/ llvm-toolchain-${VERSION_CODENAME}-21 main" | sudo tee /etc/apt/sources.list.d/llvm.list
sudo apt-get update -qq
sudo apt-get install -y \
build-essential ccache clang-21 cmake devscripts \
libc6-dbg llvm-21 lld-21 mold ninja-build rpm \
valgrind zstd
sudo curl -Ls "https://minio.weaselab.dev/public/$(uname -m)/mc.RELEASE.2025-08-13T08-35-41Z" \
-o /usr/local/bin/mc && sudo chmod +x /usr/local/bin/mc
for tool in clang clang++ llvm-ar llvm-nm llvm-ranlib llvm-objcopy llvm-cov llvm-symbolizer lld ld.lld; do
sudo update-alternatives --install /usr/bin/${tool} ${tool} /usr/bin/${tool}-21 100
done
- name: Install paper build dependencies
if: matrix.arch == 'amd64'
run: |
sudo apt-get install -y \
biber latexmk \
texlive-bibtex-extra texlive-fonts-recommended \
texlive-latex-extra texlive-pictures
- uses: actions/cache@v4
with:
path: .ccache
key: ccache-release-${{ matrix.arch }}-${{ gitea.sha }}
restore-keys: |
ccache-release-${{ matrix.arch }}-
- name: Build
run: |
export CCACHE_DIR="$GITHUB_WORKSPACE/.ccache"
rm -rf build
cmake -S . -B build -G Ninja -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_FLAGS=-DNVALGRIND
ninja -C build
ccache -s
- name: Test
run: |
cd build
# On arm64, valgrind needs the MAKE_MEM_DEFINED client requests for
# https://git.weaselab.dev/weaselab/conflict-set/issues/39, but this
# build has -DNVALGRIND, which compiles them out. Skip valgrind
# tests here; they run annotated in the test job.
ctest --no-compress-output --test-output-size-passed 100000 --test-output-size-failed 100000 ${{ matrix.arch == 'arm64' && '-E valgrind' || '' }} -T Test -j "$(nproc)" --timeout 90 > /dev/null
- name: Package
run: |
cd build
cpack -G DEB
cpack -G RPM
- name: Build paper
if: matrix.arch == 'amd64'
run: |
cd paper
make
- name: Upload artifacts to MinIO
if: always()
shell: bash
env:
MINIO_ACCESS_KEY: ${{ secrets.MINIO_ACCESS_KEY }}
MC_HOST_minio: https://${{ secrets.MINIO_ACCESS_KEY }}:${{ secrets.MINIO_SECRET_KEY }}@minio.weaselab.dev
run: |
if [ -z "$MINIO_ACCESS_KEY" ]; then
echo "MinIO credentials not configured; skipping upload"
exit 0
fi
dest="minio/jenkins/conflict-set/${{ gitea.run_number }}/release-${{ matrix.arch }}/"
zstd build/Testing/*/Test.xml
mc cp build/Testing/*/Test.xml.zst "$dest"
if compgen -G "build/*.deb" > /dev/null; then
mc cp build/*.deb "$dest"
fi
if compgen -G "build/*.rpm" > /dev/null; then
mc cp build/*.rpm "$dest"
fi
if compgen -G "paper/*.pdf" > /dev/null; then
mc cp paper/*.pdf "$dest"
fi
- name: Test summary
if: always()
run: |
python3 ctest_summary.py build/Testing/*/Test.xml \
--link "https://minio.weaselab.dev/jenkins/conflict-set/${{ gitea.run_number }}/release-${{ matrix.arch }}/Test.xml.zst" \
| tee -a "$GITHUB_STEP_SUMMARY"
coverage:
runs-on: ubuntu-latest-amd64
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: /var/cache/apt/archives
key: apt-amd64-${{ hashFiles('.gitea/workflows/ci.yml') }}
- name: Install dependencies
run: |
. /etc/os-release
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
echo "deb http://apt.llvm.org/${VERSION_CODENAME}/ llvm-toolchain-${VERSION_CODENAME}-21 main" | sudo tee /etc/apt/sources.list.d/llvm.list
sudo apt-get update -qq
sudo apt-get install -y \
build-essential ccache clang-21 cmake gcovr \
libc6-dbg llvm-21 lld-21 mold ninja-build python3 valgrind zstd
sudo curl -Ls "https://minio.weaselab.dev/public/$(uname -m)/mc.RELEASE.2025-08-13T08-35-41Z" \
-o /usr/local/bin/mc && sudo chmod +x /usr/local/bin/mc
for tool in clang clang++ llvm-ar llvm-nm llvm-ranlib llvm-objcopy llvm-cov llvm-symbolizer lld ld.lld; do
sudo update-alternatives --install /usr/bin/${tool} ${tool} /usr/bin/${tool}-21 100
done
- uses: actions/cache@v4
with:
path: .ccache
key: ccache-coverage-${{ gitea.sha }}
restore-keys: |
ccache-coverage-
- name: Build
run: |
export CCACHE_DIR="$GITHUB_WORKSPACE/.ccache"
rm -rf build
cmake -S . -B build -G Ninja -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_FLAGS=--coverage -DCMAKE_CXX_FLAGS=--coverage \
-DCMAKE_BUILD_TYPE=Debug -DDISABLE_TSAN=ON
ninja -C build
ccache -s
- name: Test
run: |
cd build
ctest --no-compress-output --test-output-size-passed 100000 --test-output-size-failed 100000 -T Test -j "$(nproc)" --timeout 90 > /dev/null
- name: Coverage report
shell: bash
run: |
gcov_args=(-f ConflictSet.cpp -f LongestCommonPrefix.h -f Metrics.h
--gcov-executable "llvm-cov gcov" --exclude-noncode-lines)
gcovr "${gcov_args[@]}" --cobertura > build/coverage.xml
gcovr "${gcov_args[@]}"
mkdir -p build/coverage_html
gcovr "${gcov_args[@]}" --html-details build/coverage_html/index.html
gcovr "${gcov_args[@]}" --fail-under-line 100 > /dev/null
- name: Upload results to MinIO
if: always()
env:
MINIO_ACCESS_KEY: ${{ secrets.MINIO_ACCESS_KEY }}
MC_HOST_minio: https://${{ secrets.MINIO_ACCESS_KEY }}:${{ secrets.MINIO_SECRET_KEY }}@minio.weaselab.dev
run: |
if [ -z "$MINIO_ACCESS_KEY" ]; then
echo "MinIO credentials not configured; skipping upload"
exit 0
fi
dest="minio/jenkins/conflict-set/${{ gitea.run_number }}/coverage/"
zstd build/Testing/*/Test.xml
mc cp build/Testing/*/Test.xml.zst "$dest"
if [ -e build/coverage.xml ]; then
mc cp build/coverage.xml "$dest"
fi
if [ -d build/coverage_html ]; then
mc cp -r build/coverage_html "$dest"
fi
- name: Test summary
if: always()
run: |
python3 ctest_summary.py build/Testing/*/Test.xml \
--link "https://minio.weaselab.dev/jenkins/conflict-set/${{ gitea.run_number }}/coverage/Test.xml.zst" \
| tee -a "$GITHUB_STEP_SUMMARY"
echo "" | tee -a "$GITHUB_STEP_SUMMARY"
echo "📊 [Coverage report](https://minio.weaselab.dev/jenkins/conflict-set/${{ gitea.run_number }}/coverage/coverage_html/index.html)" | tee -a "$GITHUB_STEP_SUMMARY"
-1
View File
@@ -7,7 +7,6 @@
void showMemory(const ConflictSet &cs);
#endif
#define ANKERL_NANOBENCH_IMPLEMENT
#include "third_party/nanobench.h"
constexpr int kNumKeys = 1000000;
+33 -20
View File
@@ -72,12 +72,6 @@ else()
add_link_options(-Wl,--gc-sections)
endif()
if(EMSCRIPTEN)
# https://github.com/emscripten-core/emscripten/issues/15377#issuecomment-1285167486
add_link_options(-lnodefs.js -lnoderawfs.js)
add_link_options(-s ALLOW_MEMORY_GROWTH)
endif()
if(NOT USE_SIMD_FALLBACK)
cmake_push_check_state()
list(APPEND CMAKE_REQUIRED_FLAGS -mavx)
@@ -144,6 +138,8 @@ include(CTest)
# disable tests if this is being used through e.g. FetchContent
if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
add_library(nanobench ${CMAKE_CURRENT_SOURCE_DIR}/nanobench.cpp)
set(TEST_FLAGS -Wall -Wextra -Wunreachable-code -Wpedantic -UNDEBUG)
# corpus tests, which are tests curated by libfuzzer. The goal is to get broad
@@ -191,6 +187,7 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
target_include_directories(conflict_set_main
PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include)
target_compile_definitions(conflict_set_main PRIVATE ENABLE_MAIN)
target_link_libraries(conflict_set_main PRIVATE nanobench)
if(NOT APPLE)
# libfuzzer target, to generate/manage corpus
@@ -251,6 +248,19 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
add_test(NAME conflict_set_blackbox_${hash} COMMAND driver ${TEST})
endforeach()
find_program(VALGRIND_EXE valgrind)
if(VALGRIND_EXE AND NOT CMAKE_CROSSCOMPILING)
list(LENGTH CORPUS_TESTS len)
math(EXPR last "${len} - 1")
set(partition_size 100)
foreach(i RANGE 0 ${last} ${partition_size})
list(SUBLIST CORPUS_TESTS ${i} ${partition_size} partition)
add_test(NAME conflict_set_blackbox_valgrind_${i}
COMMAND ${VALGRIND_EXE} --error-exitcode=99 --
$<TARGET_FILE:driver> ${partition})
endforeach()
endif()
# scripted tests. Written manually to fill in anything libfuzzer couldn't
# find.
if(NOT CMAKE_CROSSCOMPILING)
@@ -271,19 +281,14 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/test_conflict_set.py test ${TEST}
--build-dir ${CMAKE_CURRENT_BINARY_DIR})
endforeach()
endif()
find_program(VALGRIND_EXE valgrind)
if(VALGRIND_EXE AND NOT CMAKE_CROSSCOMPILING)
list(LENGTH CORPUS_TESTS len)
math(EXPR last "${len} - 1")
set(partition_size 100)
foreach(i RANGE 0 ${last} ${partition_size})
list(SUBLIST CORPUS_TESTS ${i} ${partition_size} partition)
add_test(NAME conflict_set_blackbox_valgrind_${i}
COMMAND ${VALGRIND_EXE} --error-exitcode=99 --
$<TARGET_FILE:driver> ${partition})
if(VALGRIND_EXE AND NOT CMAKE_CROSSCOMPILING)
add_test(
NAME script_test_${TEST}_valgrind
COMMAND
${VALGRIND_EXE} ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/test_conflict_set.py test ${TEST}
--build-dir ${CMAKE_CURRENT_BINARY_DIR})
endif()
endforeach()
endif()
@@ -336,7 +341,7 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
# bench
add_executable(conflict_set_bench Bench.cpp)
target_link_libraries(conflict_set_bench PRIVATE ${PROJECT_NAME})
target_link_libraries(conflict_set_bench PRIVATE ${PROJECT_NAME} nanobench)
set_target_properties(conflict_set_bench PROPERTIES SKIP_BUILD_RPATH ON)
add_executable(real_data_bench RealDataBench.cpp)
target_link_libraries(real_data_bench PRIVATE ${PROJECT_NAME})
@@ -351,6 +356,14 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
add_executable(server_bench ServerBench.cpp)
target_link_libraries(server_bench PRIVATE ${PROJECT_NAME})
set_target_properties(server_bench PROPERTIES SKIP_BUILD_RPATH ON)
add_executable(interleaving_test InterleavingTest.cpp)
# work around lack of musttail for gcc
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_BUILD_TYPE STREQUAL "Debug")
target_compile_options(interleaving_test PRIVATE -Og
-foptimize-sibling-calls)
endif()
target_link_libraries(interleaving_test PRIVATE nanobench)
endif()
# packaging
+410 -290
View File
File diff suppressed because it is too large Load Diff
+1 -7
View File
@@ -119,13 +119,7 @@ ConflictSet::ConflictSet(ConflictSet &&other) noexcept
: impl(std::exchange(other.impl, nullptr)) {}
ConflictSet &ConflictSet::operator=(ConflictSet &&other) noexcept {
if (this != &other) {
if (impl) {
impl->~Impl();
safe_free(impl, sizeof(Impl));
}
impl = std::exchange(other.impl, nullptr);
}
impl = std::exchange(other.impl, nullptr);
return *this;
}
+256
View File
@@ -0,0 +1,256 @@
#include <alloca.h>
#include <cassert>
#ifdef __x86_64__
#include <immintrin.h>
#endif
#include "third_party/nanobench.h"
struct Job {
int *input;
// Returned void* is a function pointer to the next continuation. We have to
// use void* because otherwise the type would be recursive.
typedef void *(*continuation)(Job *);
continuation next;
};
void *stepJob(Job *j) {
auto done = --(*j->input) == 0;
#ifdef __x86_64__
_mm_clflush(j->input);
#endif
return done ? nullptr : (void *)stepJob;
}
void sequential(Job **jobs, int count) {
for (int i = 0; i < count; ++i) {
do {
jobs[i]->next = (Job::continuation)jobs[i]->next(jobs[i]);
} while (jobs[i]->next);
}
}
void sequentialNoFuncPtr(Job **jobs, int count) {
for (int i = 0; i < count; ++i) {
while (stepJob(jobs[i]))
;
}
}
void interleaveSwapping(Job **jobs, int remaining) {
int current = 0;
while (remaining > 0) {
auto next = (Job::continuation)jobs[current]->next(jobs[current]);
jobs[current]->next = next;
if (next == nullptr) {
jobs[current] = jobs[remaining - 1];
--remaining;
} else {
++current;
}
if (current == remaining) {
current = 0;
}
}
}
void interleaveBoundedCyclicList(Job **jobs, int count) {
if (count == 0) {
return;
}
constexpr int kConcurrent = 32;
Job *inProgress[kConcurrent];
int nextJob[kConcurrent];
int started = std::min(kConcurrent, count);
for (int i = 0; i < started; i++) {
inProgress[i] = jobs[i];
nextJob[i] = i + 1;
}
nextJob[started - 1] = 0;
int prevJob = started - 1;
int job = 0;
for (;;) {
auto next = (Job::continuation)inProgress[job]->next(inProgress[job]);
inProgress[job]->next = next;
if (next == nullptr) {
if (started == count) {
if (prevJob == job)
break;
nextJob[prevJob] = nextJob[job];
job = prevJob;
} else {
int temp = started++;
inProgress[job] = jobs[temp];
}
}
prevJob = job;
job = nextJob[job];
}
}
#ifndef __has_attribute
#define __has_attribute(x) 0
#endif
#if __has_attribute(musttail)
#define MUSTTAIL __attribute__((musttail))
#else
#define MUSTTAIL
#endif
struct Context {
constexpr static int kConcurrent = 32;
Job **jobs;
Job *inProgress[kConcurrent];
void (*continuation[kConcurrent])(Context *, int64_t prevJob, int64_t job,
int64_t started, int64_t count);
int nextJob[kConcurrent];
};
void keepGoing(Context *context, int64_t prevJob, int64_t job, int64_t started,
int64_t count) {
prevJob = job;
job = context->nextJob[job];
MUSTTAIL return context->continuation[job](context, prevJob, job, started,
count);
}
void stepJobTailCall(Context *context, int64_t prevJob, int64_t job,
int64_t started, int64_t count);
void complete(Context *context, int64_t prevJob, int64_t job, int64_t started,
int64_t count) {
if (started == count) {
if (prevJob == job) {
return;
}
context->nextJob[prevJob] = context->nextJob[job];
job = prevJob;
} else {
context->inProgress[job] = context->jobs[started++];
context->continuation[job] = stepJobTailCall;
}
prevJob = job;
job = context->nextJob[job];
MUSTTAIL return context->continuation[job](context, prevJob, job, started,
count);
}
void stepJobTailCall(Context *context, int64_t prevJob, int64_t job,
int64_t started, int64_t count) {
auto *j = context->inProgress[job];
auto done = --(*j->input) == 0;
#ifdef __x86_64__
_mm_clflush(j->input);
#endif
if (done) {
MUSTTAIL return complete(context, prevJob, job, started, count);
} else {
context->continuation[job] = stepJobTailCall;
MUSTTAIL return keepGoing(context, prevJob, job, started, count);
}
}
void useTailCalls(Job **jobs, int count) {
if (count == 0) {
return;
}
Context context;
context.jobs = jobs;
int64_t started = std::min(Context::kConcurrent, count);
for (int i = 0; i < started; i++) {
context.inProgress[i] = jobs[i];
context.nextJob[i] = i + 1;
context.continuation[i] = stepJobTailCall;
}
context.nextJob[started - 1] = 0;
int prevJob = started - 1;
int job = 0;
return context.continuation[job](&context, prevJob, job, started, count);
}
void interleaveCyclicList(Job **jobs, int count) {
auto *nextJob = (int *)alloca(sizeof(int) * count);
for (int i = 0; i < count - 1; ++i) {
nextJob[i] = i + 1;
}
nextJob[count - 1] = 0;
int prevJob = count - 1;
int job = 0;
for (;;) {
auto next = (Job::continuation)jobs[job]->next(jobs[job]);
jobs[job]->next = next;
if (next == nullptr) {
if (prevJob == job)
break;
nextJob[prevJob] = nextJob[job];
job = prevJob;
}
prevJob = job;
job = nextJob[job];
}
}
int main() {
ankerl::nanobench::Bench bench;
constexpr int kNumJobs = 10000;
bench.relative(true);
Job jobs[kNumJobs];
Job jobsCopy[kNumJobs];
int iters = 0;
int originalInput[kNumJobs];
for (int i = 0; i < kNumJobs; ++i) {
originalInput[i] = rand() % 5 + 3;
jobs[i].input = new int{originalInput[i]};
jobs[i].next = stepJob;
iters += *jobs[i].input;
}
bench.batch(iters);
for (auto [scheduler, name] :
{std::make_pair(sequentialNoFuncPtr, "sequentialNoFuncPtr"),
std::make_pair(sequential, "sequential"),
std::make_pair(useTailCalls, "useTailCalls"),
std::make_pair(interleaveSwapping, "interleavingSwapping"),
std::make_pair(interleaveBoundedCyclicList,
"interleaveBoundedCyclicList"),
std::make_pair(interleaveCyclicList, "interleaveCyclicList")}) {
for (int i = 0; i < kNumJobs; ++i) {
*jobs[i].input = originalInput[i];
}
memcpy(jobsCopy, jobs, sizeof(jobs));
Job *ps[kNumJobs];
for (int i = 0; i < kNumJobs; ++i) {
ps[i] = jobsCopy + i;
}
scheduler(ps, kNumJobs);
for (int i = 0; i < kNumJobs; ++i) {
if (*jobsCopy[i].input != 0) {
fprintf(stderr, "%s failed\n", name);
abort();
}
}
bench.run(name, [&]() {
for (int i = 0; i < kNumJobs; ++i) {
*jobs[i].input = originalInput[i];
}
memcpy(jobsCopy, jobs, sizeof(jobs));
Job *ps[kNumJobs];
for (int i = 0; i < kNumJobs; ++i) {
ps[i] = jobsCopy + i;
}
scheduler(ps, kNumJobs);
});
}
for (int i = 0; i < kNumJobs; ++i) {
delete jobs[i].input;
}
}
Vendored
+1 -1
View File
@@ -129,7 +129,7 @@ pipeline {
}
steps {
script {
filter_args = "-f ConflictSet.cpp -f LongestCommonPrefix.h"
filter_args = "-f ConflictSet.cpp -f LongestCommonPrefix.h -f Metrics.h"
}
CleanBuildAndTest("-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_FLAGS=--coverage -DCMAKE_CXX_FLAGS=--coverage -DCMAKE_BUILD_TYPE=Debug -DDISABLE_TSAN=ON")
sh """
+2 -2
View File
@@ -129,7 +129,7 @@ longestCommonPrefix(const uint8_t *ap, const uint8_t *bp, int cl) {
}
int i = 0;
int end; // GCOVR_EXCL_LINE
int end;
// kStride * kUnrollCount at a time
end = cl & ~(kStride * kUnrollFactor - 1);
@@ -182,4 +182,4 @@ longestCommonPrefix(const uint8_t *ap, const uint8_t *bp, int cl) {
}
return i;
}
}
+64
View File
@@ -0,0 +1,64 @@
#pragma once
#include "ConflictSet.h"
#include "Internal.h"
#include <assert.h>
#include <atomic>
#include <tuple>
struct Metric {
Metric *prev;
const char *name;
const char *help;
weaselab::ConflictSet::MetricsV1::Type type;
std::atomic<int64_t> value;
protected:
Metric(Metric *&metricList, int &metricsCount, const char *name,
const char *help, weaselab::ConflictSet::MetricsV1::Type type)
: prev(std::exchange(metricList, this)), name(name), help(help),
type(type), value(0) {
++metricsCount;
}
};
struct Gauge : private Metric {
Gauge(Metric *&metricList, int &metricsCount, const char *name,
const char *help)
: Metric(metricList, metricsCount, name, help,
weaselab::ConflictSet::MetricsV1::Gauge) {}
void set(int64_t value) {
this->value.store(value, std::memory_order_relaxed);
}
};
struct Counter : private Metric {
Counter(Metric *&metricList, int &metricsCount, const char *name,
const char *help)
: Metric(metricList, metricsCount, name, help,
weaselab::ConflictSet::MetricsV1::Counter) {}
// Expensive. Accumulate locally and then call add instead of repeatedly
// calling add.
void add(int64_t value) {
assert(value >= 0);
static_assert(std::atomic<int64_t>::is_always_lock_free);
this->value.fetch_add(value, std::memory_order_relaxed);
}
};
inline weaselab::ConflictSet::MetricsV1 *initMetrics(Metric *metricsList,
int metricsCount) {
weaselab::ConflictSet::MetricsV1 *metrics =
(weaselab::ConflictSet::MetricsV1 *)safe_malloc(metricsCount *
sizeof(metrics[0]));
for (auto [i, m] = std::make_tuple(metricsCount - 1, metricsList); i >= 0;
--i, m = m->prev) {
metrics[i].name = m->name;
metrics[i].help = m->help;
metrics[i].p = m;
metrics[i].type = m->type;
}
return metrics;
}
+23 -34
View File
@@ -2,16 +2,7 @@ A data structure for optimistic concurrency control on ranges of bitwise-lexicog
Intended as an alternative to FoundationDB's skip list.
Hardware for all benchmarks is an AMD Ryzen 9 7900 with (2x32GB) 5600MT/s CL28-34-34-89 1.35V RAM.
```
$ clang++ --version
Ubuntu clang version 21.1.8 (6ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-21/bin
```
Hardware for all benchmarks is an AMD Ryzen 9 7900 with (2x32GB) 5600MT/s CL28-34-34-89 1.35V RAM
# Microbenchmark
@@ -19,45 +10,44 @@ InstalledDir: /usr/lib/llvm-21/bin
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 171.37 | 5,835,467.28 | 0.5% | 2,915.46 | 629.31 | 4.633 | 502.91 | 0.0% | 0.01 | `point reads`
| 168.16 | 5,946,774.62 | 0.5% | 2,859.73 | 619.03 | 4.620 | 488.59 | 0.0% | 0.01 | `prefix reads`
| 245.86 | 4,067,294.63 | 0.3% | 3,508.97 | 904.82 | 3.878 | 627.73 | 0.0% | 0.01 | `range reads`
| 381.14 | 2,623,691.76 | 0.2% | 5,127.16 | 1,402.60 | 3.655 | 830.59 | 1.7% | 0.01 | `point writes`
| 376.24 | 2,657,887.07 | 0.3% | 5,062.85 | 1,381.07 | 3.666 | 811.68 | 1.6% | 0.01 | `prefix writes`
| 257.09 | 3,889,688.44 | 3.8% | 3,010.09 | 949.42 | 3.170 | 521.34 | 2.9% | 0.01 | `range writes`
| 629.71 | 1,588,026.28 | 4.0% | 6,809.60 | 2,320.32 | 2.935 | 1,247.08 | 1.3% | 0.01 | `monotonic increasing point writes`
| 126,164.50 | 7,926.16 | 1.7% | 772,460.25 | 463,264.67 | 1.667 | 141,512.00 | 0.2% | 0.01 | `worst case for radix tree`
| 15.33 | 65,217,669.23 | 0.4% | 299.00 | 56.45 | 5.297 | 64.00 | 0.0% | 0.01 | `create and destroy`
| 172.03 | 5,812,791.77 | 0.4% | 3,130.62 | 879.00 | 3.562 | 509.23 | 0.0% | 0.01 | `point reads`
| 167.44 | 5,972,130.71 | 0.2% | 3,065.14 | 862.27 | 3.555 | 494.30 | 0.0% | 0.01 | `prefix reads`
| 238.77 | 4,188,130.84 | 0.9% | 3,589.93 | 1,259.30 | 2.851 | 637.12 | 0.0% | 0.01 | `range reads`
| 424.01 | 2,358,426.70 | 0.2% | 5,620.05 | 2,242.35 | 2.506 | 854.80 | 1.7% | 0.01 | `point writes`
| 418.45 | 2,389,780.56 | 0.4% | 5,525.07 | 2,211.05 | 2.499 | 831.71 | 1.7% | 0.01 | `prefix writes`
| 254.87 | 3,923,568.88 | 2.6% | 3,187.01 | 1,366.50 | 2.332 | 529.11 | 2.7% | 0.02 | `range writes`
| 675.96 | 1,479,374.50 | 3.3% | 7,735.41 | 3,468.60 | 2.230 | 1,386.02 | 1.8% | 0.01 | `monotonic increasing point writes`
| 137,986.20 | 7,247.10 | 0.6% | 789,752.33 | 699,462.00 | 1.129 | 144,824.14 | 0.0% | 0.01 | `worst case for radix tree`
| 21.63 | 46,231,564.03 | 1.0% | 448.00 | 107.14 | 4.181 | 84.00 | 0.0% | 0.01 | `create and destroy`
## Radix tree (this implementation)
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 12.81 | 78,088,759.03 | 0.7% | 187.38 | 47.15 | 3.974 | 33.70 | 0.5% | 0.01 | `point reads`
| 16.77 | 59,626,501.14 | 0.4% | 277.61 | 61.74 | 4.497 | 44.76 | 0.4% | 0.01 | `prefix reads`
| 42.45 | 23,556,818.87 | 0.4% | 856.96 | 156.09 | 5.490 | 126.22 | 0.2% | 0.01 | `range reads`
| 15.76 | 63,453,196.94 | 0.3% | 249.89 | 58.04 | 4.306 | 33.21 | 0.5% | 0.01 | `point writes`
| 29.71 | 33,662,499.39 | 0.1% | 509.14 | 109.33 | 4.657 | 70.26 | 0.5% | 0.01 | `prefix writes`
| 35.77 | 27,956,388.03 | 0.8% | 614.18 | 131.72 | 4.663 | 85.57 | 0.1% | 0.01 | `range writes`
| 80.13 | 12,480,456.47 | 1.5% | 1,279.04 | 294.90 | 4.337 | 234.68 | 0.1% | 0.01 | `monotonic increasing point writes`
| 360,748.00 | 2,772.02 | 2.6% | 4,401,835.00 | 1,318,754.00 | 3.338 | 728,537.00 | 0.0% | 0.01 | `worst case for radix tree`
| 90.69 | 11,027,103.45 | 1.0% | 1,686.00 | 333.80 | 5.051 | 284.00 | 0.0% | 0.01 | `create and destroy`
| 12.88 | 77,653,350.77 | 0.5% | 185.37 | 64.45 | 2.876 | 41.51 | 0.4% | 0.01 | `point reads`
| 14.67 | 68,179,354.49 | 0.1% | 271.44 | 73.40 | 3.698 | 53.70 | 0.3% | 0.01 | `prefix reads`
| 34.84 | 28,701,444.36 | 0.3% | 715.74 | 175.27 | 4.084 | 127.30 | 0.2% | 0.01 | `range reads`
| 17.12 | 58,422,988.28 | 0.2% | 314.30 | 86.11 | 3.650 | 39.82 | 0.4% | 0.01 | `point writes`
| 31.42 | 31,830,804.65 | 0.1% | 591.06 | 158.07 | 3.739 | 82.67 | 0.2% | 0.01 | `prefix writes`
| 37.37 | 26,759,432.70 | 2.2% | 681.98 | 188.95 | 3.609 | 96.10 | 0.1% | 0.01 | `range writes`
| 76.72 | 13,035,140.63 | 2.3% | 1,421.28 | 387.17 | 3.671 | 257.76 | 0.1% | 0.01 | `monotonic increasing point writes`
| 297,452.00 | 3,361.89 | 0.9% | 3,508,083.00 | 1,500,834.67 | 2.337 | 727,525.33 | 0.1% | 0.01 | `worst case for radix tree`
| 87.70 | 11,402,490.60 | 1.0% | 1,795.00 | 442.09 | 4.060 | 297.00 | 0.0% | 0.01 | `create and destroy`
# "Real data" test
Point queries only. Gc ratio is the ratio of time spent doing garbage collection to time spent adding writes or doing garbage collection. Lower is better.
Point queries only, best of three runs. Gc ratio is the ratio of time spent doing garbage collection to time spent adding writes or doing garbage collection. Lower is better.
## skip list
```
Check: 4.42779 seconds, 368.253 MB/s, Add: 4.64942 seconds, 120.586 MB/s, Gc ratio: 28.8784%, Peak idle memory: 5.51735e+06
Check: 4.47891 seconds, 364.05 MB/s, Add: 4.55599 seconds, 123.058 MB/s, Gc ratio: 37.1145%
```
## radix tree
```
Check: 0.891163 seconds, 1829.69 MB/s, Add: 1.23064 seconds, 455.579 MB/s, Gc ratio: 42.9417%, Peak idle memory: 2.28333e+06
Check: 0.953012 seconds, 1710.94 MB/s, Add: 1.30025 seconds, 431.188 MB/s, Gc ratio: 43.9816%, Peak idle memory: 2.28375e+06
```
## hash table
@@ -65,6 +55,5 @@ Check: 0.891163 seconds, 1829.69 MB/s, Add: 1.23064 seconds, 455.579 MB/s, Gc ra
(The hash table implementation doesn't work on range queries, and its purpose is to provide an idea of how fast point queries can be)
```
Check: 0.858484 seconds, 1899.33 MB/s, Add: 0.632886 seconds, 885.868 MB/s, Gc ratio: 41.252%, Peak idle memory: 0
Check: 0.804094 seconds, 2027.81 MB/s, Add: 0.652952 seconds, 858.645 MB/s, Gc ratio: 35.3885%
```
+1 -1
View File
@@ -21,7 +21,7 @@
std::atomic<int64_t> transactions;
constexpr int kBaseSearchDepth = 32;
constexpr int kBaseSearchDepth = 115;
constexpr int kWindowSize = 10000000;
std::string numToKey(int64_t num) {
+88 -16
View File
@@ -22,6 +22,7 @@
#include "ConflictSet.h"
#include "Internal.h"
#include "Metrics.h"
#include <algorithm>
#include <span>
@@ -434,13 +435,14 @@ public:
return result;
}
void detectConflicts(ReadConflictRange *ranges, int count,
ConflictSet::Result *transactionConflictStatus) const {
// Return number of iterations of main loop
int detectConflicts(ReadConflictRange *ranges, int count,
ConflictSet::Result *transactionConflictStatus) const {
const int M = 16;
int nextJob[M];
CheckMax inProgress[M];
if (!count)
return;
return 0;
int started = std::min(M, count);
for (int i = 0; i < started; i++) {
@@ -451,8 +453,9 @@ public:
int prevJob = started - 1;
int job = 0;
int iters = 0;
// vtune: 340 parts
while (true) {
for (;; ++iters) {
if (inProgress[job].advance()) {
if (started == count) {
if (prevJob == job)
@@ -468,6 +471,7 @@ public:
prevJob = job;
job = nextJob[job];
}
return iters;
}
void find(const StringRef *values, Finger *results, int *temp, int count) {
@@ -702,15 +706,27 @@ private:
};
};
struct ReadContext {
int64_t commits_accum = 0;
int64_t conflicts_accum = 0;
int64_t too_olds_accum = 0;
int64_t check_bytes_accum = 0;
};
struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
Impl(int64_t oldestVersion)
: oldestVersion(oldestVersion), newestVersion(oldestVersion),
skipList(oldestVersion) {}
skipList(oldestVersion) {
metrics = initMetrics(metricsList, metricsCount);
}
~Impl() { safe_free(metrics, metricsCount * sizeof(metrics[0])); }
void check(const ConflictSet::ReadRange *reads, ConflictSet::Result *results,
int count) const {
int count) {
ReadContext tls;
Arena arena;
auto *ranges = new (arena) ReadConflictRange[count];
for (int i = 0; i < count; ++i) {
tls.check_bytes_accum += reads[i].begin.len + reads[i].end.len;
ranges[i].begin = {reads[i].begin.p, size_t(reads[i].begin.len)};
ranges[i].end = reads[i].end.len > 0
? StringRef{reads[i].end.p, size_t(reads[i].end.len)}
@@ -718,13 +734,22 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
ranges[i].version = reads[i].readVersion;
results[i] = ConflictSet::Commit;
}
skipList.detectConflicts(ranges, count, results);
int iters = skipList.detectConflicts(ranges, count, results);
for (int i = 0; i < count; ++i) {
if (reads[i].readVersion < oldestVersion ||
reads[i].readVersion < newestVersion - 2e9) {
results[i] = TooOld;
}
tls.commits_accum += results[i] == Commit;
tls.conflicts_accum += results[i] == Conflict;
tls.too_olds_accum += results[i] == TooOld;
}
range_read_iterations_total.add(iters);
range_read_total.add(count);
commits_total.add(tls.commits_accum);
conflicts_total.add(tls.conflicts_accum);
too_olds_total.add(tls.too_olds_accum);
check_bytes_total.add(tls.check_bytes_accum);
}
void addWrites(const ConflictSet::WriteRange *writes, int count,
@@ -788,6 +813,9 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
}
void setOldestVersion(int64_t oldestVersion) {
// This isn't 100% accurate. It overcounts if you hit the end
gc_iterations_total.add(keyUpdates);
assert(oldestVersion >= this->oldestVersion);
this->oldestVersion = oldestVersion;
SkipList::Finger finger;
@@ -802,6 +830,54 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
int64_t totalBytes = 0;
MetricsV1 *metrics;
int metricsCount = 0;
Metric *metricsList = nullptr;
#define GAUGE(name, help) \
Gauge name { metricsList, metricsCount, #name, help }
#define COUNTER(name, help) \
Counter name { metricsList, metricsCount, #name, help }
// ==================== METRICS DEFINITIONS ====================
COUNTER(range_read_total, "Total number of range reads checked");
COUNTER(range_read_iterations_total,
"Total number of iterations of the main loops for range read checks");
COUNTER(commits_total,
"Total number of checks where the result is \"commit\"");
COUNTER(conflicts_total,
"Total number of checks where the result is \"conflict\"");
COUNTER(too_olds_total,
"Total number of checks where the result is \"too old\"");
COUNTER(check_bytes_total, "Total number of key bytes checked");
GAUGE(memory_bytes, "Total number of bytes in use");
COUNTER(nodes_allocated_total,
"The total number of physical tree nodes allocated");
COUNTER(nodes_released_total,
"The total number of physical tree nodes released");
COUNTER(insert_iterations_total,
"The total number of iterations of the main loop for insertion. "
"Includes searches where the entry already existed, and so insertion "
"did not take place");
COUNTER(entries_inserted_total,
"The total number of entries inserted in the tree");
COUNTER(entries_erased_total,
"The total number of entries erased from the tree");
COUNTER(
gc_iterations_total,
"The total number of iterations of the main loop for garbage collection");
COUNTER(write_bytes_total, "Total number of key bytes in calls to addWrites");
GAUGE(oldest_version,
"The lowest version that doesn't result in \"TooOld\" for checks");
GAUGE(newest_version, "The version of the most recent call to addWrites");
// ==================== END METRICS DEFINITIONS ====================
#undef GAUGE
#undef COUNTER
void getMetricsV1(MetricsV1 **metrics, int *count) {
*metrics = this->metrics;
*count = metricsCount;
}
private:
int64_t keyUpdates = 0;
Arena removalArena;
@@ -824,6 +900,7 @@ void internal_addWrites(ConflictSet::Impl *impl,
mallocBytesDelta = 0;
impl->addWrites(writes, count, writeVersion);
impl->totalBytes += mallocBytesDelta;
impl->memory_bytes.set(impl->totalBytes);
#if SHOW_MEMORY
if (impl->totalBytes != mallocBytes) {
abort();
@@ -835,6 +912,7 @@ void internal_setOldestVersion(ConflictSet::Impl *impl, int64_t oldestVersion) {
mallocBytesDelta = 0;
impl->setOldestVersion(oldestVersion);
impl->totalBytes += mallocBytesDelta;
impl->memory_bytes.set(impl->totalBytes);
#if SHOW_MEMORY
if (impl->totalBytes != mallocBytes) {
abort();
@@ -858,12 +936,11 @@ int64_t internal_getBytes(ConflictSet::Impl *impl) { return impl->totalBytes; }
void internal_getMetricsV1(ConflictSet::Impl *impl,
ConflictSet::MetricsV1 **metrics, int *count) {
*metrics = nullptr;
*count = 0;
return impl->getMetricsV1(metrics, count);
}
double internal_getMetricValue(const ConflictSet::MetricsV1 *metric) {
return 0;
return ((Metric *)metric->p)->value.load(std::memory_order_relaxed);
}
void ConflictSet::check(const ReadRange *reads, Result *results,
@@ -903,12 +980,7 @@ ConflictSet::ConflictSet(ConflictSet &&other) noexcept
: impl(std::exchange(other.impl, nullptr)) {}
ConflictSet &ConflictSet::operator=(ConflictSet &&other) noexcept {
if (this != &other) {
if (impl) {
internal_destroy(impl);
}
impl = std::exchange(other.impl, nullptr);
}
impl = std::exchange(other.impl, nullptr);
return *this;
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More