44 Commits

Author SHA1 Message Date
587874841f Fix test that had a decreasing write version
Some checks failed
Tests / Clang total: 1039, passed: 1039
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1039, passed: 1039
Tests / 32-bit versions total: 1039, passed: 1039
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-28 19:57:25 -07:00
648b0b9238 Add an always_inline, with explanatory comment 2024-06-28 19:55:33 -07:00
d3f4afa167 More SIMD for scanning Node256 with 32-bit versions
Some checks failed
Tests / Clang total: 1039, passed: 1039
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1039, passed: 1039
Tests / 32-bit versions total: 1039, passed: 1039
Tests / Release [gcc] total: 1039, passed: 1039
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 780, failed: 1, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-28 19:48:06 -07:00
f762add4d6 Write vectorized 32-bit compare by hand for arm in scan16
Some checks failed
Tests / Clang total: 1039, passed: 1039
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1039, passed: 1039
Tests / Release [gcc] total: 1039, passed: 1039
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 780, failed: 1, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-28 17:28:59 -07:00
b311e5f1f0 Add an experimental, disabled 32 bit internal version
Some checks failed
Tests / Clang total: 1039, passed: 1039
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1039, passed: 1039
Tests / Release [gcc] total: 1039, passed: 1039
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 780, failed: 1, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
I think it's only missing detection for full-precision versions more
than 2e9 apart
2024-06-28 15:53:35 -07:00
ff81890921 Rename MaxVersionT to InternalVersionT
Some checks failed
Tests / Clang total: 1038, passed: 1038
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1038, passed: 1038
Tests / Release [gcc] total: 1038, passed: 1038
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 779, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-28 14:34:48 -07:00
0e96177f5c Allow to easily experiment with 32 bit "max version" type
Some checks failed
Tests / Clang total: 1038, passed: 1038
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1038, passed: 1038
Tests / Release [gcc] total: 1038, passed: 1038
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 779, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-28 13:54:12 -07:00
efb0e52a0a SIMD implementation of scan16 for x86
Some checks failed
Tests / Clang total: 1038, passed: 1038
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1038, passed: 1038
Tests / Release [gcc] total: 1038, passed: 1038
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 779, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
Closes #29
2024-06-27 22:21:41 -07:00
2df7000090 Remove switch on phase from Stepwise left/right step
Some checks failed
Tests / Clang total: 1038, passed: 1038
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1038, passed: 1038
Tests / Release [gcc] total: 1038, passed: 1038
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 779, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-27 20:51:35 -07:00
5378a06c39 Vectorize all bounds checks for Node256 scan
Some checks failed
Tests / Clang total: 1038, passed: 1038
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1038, passed: 1038
Tests / Release [gcc] total: 1038, passed: 1038
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 775, passed: 775
Tests / Coverage total: 779, passed: 779
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-27 17:40:23 -07:00
12c6ed2568 Reorganize to prepare for better vectorized first/last page 2024-06-27 17:21:41 -07:00
a2bf839b19 Update corpus 2024-06-27 17:15:35 -07:00
c065b185ae Vectorize inner page check for Node256 2024-06-27 17:09:45 -07:00
639518bed4 Share "scan16" between Node16 and Node48
Some checks failed
Tests / Clang total: 1130, passed: 1130
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1130, passed: 1130
Tests / Release [gcc] total: 1130, passed: 1130
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 844, passed: 844
Tests / Coverage total: 848, passed: 848
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-27 13:22:51 -07:00
7de983cc15 Simd bounds checking for scan for Node16 2024-06-27 13:08:12 -07:00
1b4b61ddc6 Write Node16 scan in a "more vectorized" style 2024-06-27 12:07:38 -07:00
bff7b85de2 Remove "Child" struct 2024-06-27 10:03:14 -07:00
9108ee209a SoA instead of AoS for child, maxVersion 2024-06-27 09:57:54 -07:00
f8bf1c6eb4 Remove unreachable code
Some checks failed
Tests / Clang total: 1130, passed: 1130
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1130, passed: 1130
Tests / Release [gcc] total: 1130, passed: 1130
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 844, passed: 844
Tests / Coverage total: 848, passed: 848
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-26 22:14:27 -07:00
4da2a01614 Use single &, to show branch-free intent 2024-06-26 22:14:05 -07:00
bb0e654040 Fix missed update for Node48::maxOfMax 2024-06-26 22:11:33 -07:00
cce7d29410 Update our benchmarks in README
Some checks failed
Tests / Clang total: 1130, passed: 1130
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1130, passed: 1130
Tests / Release [gcc] total: 1130, passed: 1130
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 844, passed: 844
Tests / Coverage total: 848, passed: 848
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-06-26 20:59:33 -07:00
13f8d3fa8a Add benchmarks for individual spans, but commented out 2024-06-26 20:57:51 -07:00
02866a8cae Save some bounds checking for scanning Node256 2024-06-26 20:55:18 -07:00
fa86d3e707 "max of max" for Node48 again, but physical instead of logical 2024-06-26 20:41:27 -07:00
7d1d1d7b2a Maintain childMaxVersion == 0 for unused children in Node48 2024-06-26 20:16:50 -07:00
789ecc29b3 Use unsigned compare trick to check in bounds 2024-06-26 19:41:25 -07:00
08f2998a85 Use 8 byte pages for "max of max"
This seems to benchmark better
2024-06-26 19:18:38 -07:00
c882d7663d Maintain "reverseIndex" in Node48 2024-06-26 19:11:34 -07:00
bfea4384ba Branchless inner page check for Node256 2024-06-26 18:28:41 -07:00
6520e3d734 "max of max" for Node48 2024-06-26 17:54:03 -07:00
23ace8aac5 Fill in leftward on right side in worst case for radix tree bench 2024-06-26 17:37:24 -07:00
62e35de320 Update our benchmark in readme 2024-06-26 16:36:02 -07:00
22e4ab01a1 Track "max of max" versions in Node256 2024-06-26 16:28:24 -07:00
b3aeed0caa Warning: interface change! Require versions >= 0 2024-06-26 15:46:36 -07:00
5f3833e965 Change maxVersion to return by value, and add setMaxVersion 2024-06-26 15:33:15 -07:00
8b1cd9c052 Minor improvements to checkMaxBetweenExclusive 2024-06-26 15:06:50 -07:00
bb9bc3d7b5 Measure across different cardinalities for radix worst case bench 2024-06-26 15:06:36 -07:00
89b3354a80 Update README with new benchmark
All checks were successful
Tests / Clang total: 1130, passed: 1130
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1130, passed: 1130
Tests / Release [gcc] total: 1130, passed: 1130
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 844, passed: 844
Tests / Coverage total: 848, passed: 848
weaselab/conflict-set/pipeline/head This commit looks good
2024-06-25 21:47:46 -07:00
488c723726 Improve worst-case radix tree checkRangeRead 2024-06-25 21:22:55 -07:00
76d0785b33 Add worst-case benchmark for radix tree
Closes #27
2024-06-25 20:50:22 -07:00
add0af11ad Don't check paper/version.txt into version control
Some checks reported errors
weaselab/conflict-set/pipeline/head Something is wrong with the build of this commit
But also don't remove it in `make -C paper clean`
2024-06-25 19:21:14 -07:00
2c0adf4a8b Fix test-only bug in script test
Previously conflict_set.py only worked for checking one read conflict
per call
2024-06-25 19:20:19 -07:00
e8ac78cce6 Bump version
All checks were successful
Tests / Clang total: 1130, passed: 1130
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1130, passed: 1130
Tests / Release [gcc] total: 1130, passed: 1130
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 844, passed: 844
Tests / Coverage total: 848, passed: 848
weaselab/conflict-set/pipeline/head This commit looks good
2024-06-12 14:07:34 -07:00
403 changed files with 825 additions and 347 deletions

105
Bench.cpp
View File

@@ -258,4 +258,107 @@ void benchConflictSet() {
}
}
int main(void) { benchConflictSet(); }
constexpr int kKeyLenForWorstCase = 50;
ConflictSet worstCaseConflictSetForRadixRangeRead(int cardinality) {
ConflictSet cs{0};
for (int i = 0; i < kKeyLenForWorstCase; ++i) {
for (int j = 0; j < cardinality; ++j) {
auto b = std::vector<uint8_t>(i, 0);
b.push_back(j);
auto e = std::vector<uint8_t>(i, 255);
e.push_back(255 - j);
weaselab::ConflictSet::WriteRange w[] = {{
{b.data(), int(b.size())},
{nullptr, 0},
},
{
{e.data(), int(e.size())},
{nullptr, 0},
}};
std::sort(std::begin(w), std::end(w),
[](const auto &lhs, const auto &rhs) {
int cl = std::min(lhs.begin.len, rhs.begin.len);
if (cl > 0) {
int c = memcmp(lhs.begin.p, rhs.begin.p, cl);
if (c != 0) {
return c < 0;
}
}
return lhs.begin.len < rhs.begin.len;
});
cs.addWrites(w, sizeof(w) / sizeof(w[0]), 0);
}
}
// Defeat short-circuiting on the left
{
auto k = std::vector<uint8_t>(kKeyLenForWorstCase, 0);
weaselab::ConflictSet::WriteRange w[] = {
{
{k.data(), int(k.size())},
{nullptr, 0},
},
};
cs.addWrites(w, sizeof(w) / sizeof(w[0]), 1);
}
// Defeat short-circuiting on the right
{
auto k = std::vector<uint8_t>(kKeyLenForWorstCase, 255);
weaselab::ConflictSet::WriteRange w[] = {
{
{k.data(), int(k.size())},
{nullptr, 0},
},
};
cs.addWrites(w, sizeof(w) / sizeof(w[0]), 1);
}
return cs;
}
void benchWorstCaseForRadixRangeRead() {
ankerl::nanobench::Bench bench;
std::unique_ptr<ConflictSet> cs[256];
for (int i = 0; i < 256; ++i) {
cs[i] =
std::make_unique<ConflictSet>(worstCaseConflictSetForRadixRangeRead(i));
}
auto begin = std::vector<uint8_t>(kKeyLenForWorstCase - 1, 0);
begin.push_back(1);
auto end = std::vector<uint8_t>(kKeyLenForWorstCase - 1, 255);
end.push_back(254);
weaselab::ConflictSet::Result result;
weaselab::ConflictSet::ReadRange r{
{begin.data(), int(begin.size())}, {end.data(), int(end.size())}, 0};
bench.run("worst case for radix tree", [&]() {
for (int i = 0; i < 256; ++i) {
result = weaselab::ConflictSet::TooOld;
cs[i]->check(&r, &result, 1);
if (result != weaselab::ConflictSet::Commit) {
abort();
}
}
});
// for (int i = 0; i < 256; ++i) {
// bench.run("worst case for radix tree, span " + std::to_string(i), [&]() {
// result = weaselab::ConflictSet::TooOld;
// cs[i]->check(&r, &result, 1);
// if (result != weaselab::ConflictSet::Commit) {
// abort();
// }
// });
// }
}
int main(void) {
benchConflictSet();
benchWorstCaseForRadixRangeRead();
}

View File

@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.18)
project(
conflict-set
VERSION 0.0.6
VERSION 0.0.7
DESCRIPTION
"A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys."
HOMEPAGE_URL "https://git.weaselab.dev/weaselab/conflict-set"
@@ -59,6 +59,10 @@ cmake_pop_check_state()
option(USE_SIMD_FALLBACK
"Use fallback implementations of functions that use SIMD" OFF)
option(
USE_32_BIT_VERSIONS,
"Store 32 bit versions internally, and rely on versions never being different by more than 2e9"
OFF)
# This is encouraged according to
# https://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.clientreq
@@ -103,6 +107,10 @@ if(NOT USE_SIMD_FALLBACK)
endif()
endif()
if(USE_32_BIT_VERSIONS)
add_compile_definitions(INTERNAL_VERSION_32_BIT=1)
endif()
set(CMAKE_CXX_IMPLICIT_LINK_LIBRARIES "")
add_library(${PROJECT_NAME}-object OBJECT ConflictSet.cpp)

File diff suppressed because it is too large Load Diff

View File

@@ -578,8 +578,8 @@ template <class ConflictSetImpl> struct TestDriver {
explicit TestDriver(const uint8_t *data, size_t size)
: arbitrary({data, size}) {}
int64_t writeVersion = 100;
int64_t oldestVersion = 0;
int64_t oldestVersion = arbitrary.bounded(2) ? 0 : 0xfffffff0;
int64_t writeVersion = oldestVersion + 100;
ConflictSetImpl cs{oldestVersion};
ReferenceImpl refImpl{oldestVersion};

11
Jenkinsfile vendored
View File

@@ -59,6 +59,17 @@ pipeline {
CleanBuildAndTest("-DUSE_SIMD_FALLBACK=ON")
}
}
stage('32-bit versions') {
agent {
dockerfile {
args '-v /home/jenkins/ccache:/ccache'
reuseNode true
}
}
steps {
CleanBuildAndTest("-DUSE_32_BIT_VERSIONS=ON")
}
}
stage('Release [gcc]') {
agent {
dockerfile {

View File

@@ -58,27 +58,29 @@ Performance counters:
## Skip list
| ns/op | op/s | err% | total | benchmark |
| -----: | -----------: | ---: | ----: | :---------------------------------- |
| 246.99 | 4,048,700.59 | 0.2% | 0.01 | `point reads` |
| 260.16 | 3,843,784.65 | 0.1% | 0.01 | `prefix reads` |
| 493.35 | 2,026,953.19 | 0.1% | 0.01 | `range reads` |
| 462.05 | 2,164,289.23 | 0.6% | 0.01 | `point writes` |
| 448.19 | 2,231,205.25 | 0.9% | 0.01 | `prefix writes` |
| 255.83 | 3,908,845.72 | 1.5% | 0.02 | `range writes` |
| 582.63 | 1,716,349.02 | 1.3% | 0.01 | `monotonic increasing point writes` |
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 256.89 | 3,892,784.92 | 0.3% | 0.01 | `point reads`
| 272.90 | 3,664,395.04 | 0.2% | 0.01 | `prefix reads`
| 507.22 | 1,971,549.50 | 0.7% | 0.01 | `range reads`
| 452.66 | 2,209,181.91 | 0.5% | 0.01 | `point writes`
| 438.09 | 2,282,619.96 | 0.4% | 0.01 | `prefix writes`
| 253.33 | 3,947,420.36 | 2.5% | 0.02 | `range writes`
| 574.07 | 1,741,936.71 | 0.3% | 0.01 | `monotonic increasing point writes`
| 151,562.50 | 6,597.94 | 1.5% | 0.01 | `worst case for radix tree`
## Radix tree (this implementation)
| ns/op | op/s | err% | total | benchmark |
| -----: | ------------: | ---: | ----: | :---------------------------------- |
| 19.42 | 51,483,206.67 | 0.3% | 0.01 | `point reads` |
| 58.43 | 17,115,612.57 | 0.1% | 0.01 | `prefix reads` |
| 216.09 | 4,627,766.60 | 0.2% | 0.01 | `range reads` |
| 28.35 | 35,267,567.72 | 0.2% | 0.01 | `point writes` |
| 43.43 | 23,026,226.17 | 0.2% | 0.01 | `prefix writes` |
| 50.00 | 20,000,000.00 | 0.0% | 0.01 | `range writes` |
| 92.38 | 10,824,863.69 | 4.1% | 0.01 | `monotonic increasing point writes` |
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 19.83 | 50,420,955.28 | 0.1% | 0.01 | `point reads`
| 55.95 | 17,872,542.40 | 0.5% | 0.01 | `prefix reads`
| 88.28 | 11,327,709.50 | 0.4% | 0.01 | `range reads`
| 29.15 | 34,309,531.64 | 0.5% | 0.01 | `point writes`
| 42.36 | 23,607,424.27 | 1.1% | 0.01 | `prefix writes`
| 50.00 | 20,000,000.00 | 0.0% | 0.01 | `range writes`
| 93.52 | 10,692,413.79 | 3.3% | 0.01 | `monotonic increasing point writes`
| 2,388,417.00 | 418.69 | 0.4% | 0.03 | `worst case for radix tree`
# "Real data" test

View File

@@ -1,3 +1,5 @@
___stack_chk_fail
___stack_chk_guard
__tlv_bootstrap
_abort
_bzero

View File

@@ -115,7 +115,9 @@ class ConflictSet:
def check(self, *reads: ReadRange) -> list[Result]:
r = (ctypes.c_int * len(reads))()
self._lib.ConflictSet_check(self.p, *reads, r, 1)
self._lib.ConflictSet_check(
self.p, (ReadRange * len(reads))(*reads), r, len(reads)
)
return [Result(x) for x in r]
def setOldestVersion(self, version: int) -> None:

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 B

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.5 KiB

Binary file not shown.

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More