7 Commits
Author SHA1 Message Date
andrew e3d1b2e842 Add to corpus
Tests / Clang total: 2096, passed: 2096
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 2094, passed: 2094
Tests / SIMD fallback total: 2096, passed: 2096
Tests / Release [gcc] total: 2096, passed: 2096
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1564, passed: 1564
Tests / Coverage total: 1574, passed: 1574
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.34% (1803/1815) * Branch Coverage: 67.55% (1486/2200) * Complexity Density: 0.00 * Lines of Code: 1815 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
After a bunch of fuzzing on "udon" (my new zen4 machine)
2024-08-09 19:48:22 -07:00
andrew 9f8800af16 Add more to corpus (from fuzzing on osx)
Tests / Clang total: 1784, passed: 1784
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1782, passed: 1782
Tests / SIMD fallback total: 1784, passed: 1784
Tests / Release [gcc] total: 1784, passed: 1784
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1330, passed: 1330
Tests / Coverage total: 1340, passed: 1340
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.12% (1799/1815) * Branch Coverage: 67.50% (1485/2200) * Complexity Density: 0.00 * Lines of Code: 1815 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head Something is wrong with the build of this commit
Maybe we should just accumulate the corpus instead of replacing it? That
should be easier on git right?
2024-08-09 18:06:54 -07:00
andrew 182c065c8e Insert common prefix in addWriteRange
Tests / Clang total: 1420, passed: 1420
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1418, passed: 1418
Tests / SIMD fallback total: 1420, passed: 1420
Tests / Release [gcc] total: 1420, passed: 1420
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1057, passed: 1057
Tests / Coverage total: 1067, passed: 1067
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.12% (1799/1815) * Branch Coverage: 67.50% (1485/2200) * Complexity Density: 0.00 * Lines of Code: 1815 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head There was a failure building this commit
This allows us to use our optimized implementation for setting max
version along the search path instead of a one-off loop
2024-08-09 14:57:04 -07:00
andrew 2dba0d5be3 Have insert return a pointer to the in-tree pointer 2024-08-09 13:58:31 -07:00
andrew a1dfdf355c Use metrics to count change in entry count
This lets us run gc slower safely
2024-08-09 13:44:49 -07:00
andrew 15919cb1c4 Add range writes to server_bench 2024-08-09 13:43:24 -07:00
andrew 5ed9003a83 Bump version
Tests / Clang total: 1420, passed: 1420
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1418, passed: 1418
Tests / SIMD fallback total: 1420, passed: 1420
Tests / Release [gcc] total: 1420, passed: 1420
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1057, passed: 1057
Tests / Coverage total: 1067, passed: 1067
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 99.34% (1817/1829) * Branch Coverage: 67.49% (1503/2227) * Complexity Density: 0.00 * Lines of Code: 1829 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-08-08 12:09:26 -07:00
172 changed files with 35 additions and 47 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.18) cmake_minimum_required(VERSION 3.18)
project( project(
conflict-set conflict-set
VERSION 0.0.11 VERSION 0.0.12
DESCRIPTION DESCRIPTION
"A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys." "A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys."
HOMEPAGE_URL "https://git.weaselab.dev/weaselab/conflict-set" HOMEPAGE_URL "https://git.weaselab.dev/weaselab/conflict-set"
+23 -44
View File
@@ -2922,25 +2922,22 @@ void consumePartialKey(Node *&self, std::span<const uint8_t> &key,
key = key.subspan(partialKeyIndex, key.size() - partialKeyIndex); key = key.subspan(partialKeyIndex, key.size() - partialKeyIndex);
} }
// Returns a pointer to the newly inserted node. Caller must set // Returns a pointer the pointer to the newly inserted node in the tree. Caller
// `entryPresent`, and `entry` fields. All nodes along the search path of the // must set `entryPresent`, and `entry` fields. All nodes along the search path
// result will have `maxVersion` set to `writeVersion` as a postcondition. Nodes // of the result will have `maxVersion` set to `writeVersion` as a
// along the search path may be invalidated. // postcondition. Nodes along the search path may be invalidated.
[[nodiscard]] [[nodiscard]]
Node *insert(Node **self, std::span<const uint8_t> key, Node **insert(Node **self, std::span<const uint8_t> key,
InternalVersionT writeVersion, WriteContext *tls, InternalVersionT writeVersion, WriteContext *tls,
ConflictSet::Impl *impl) { ConflictSet::Impl *impl) {
if ((*self)->partialKeyLen > 0) {
consumePartialKey(*self, key, writeVersion, tls);
}
assert(maxVersion(*self, impl) <= writeVersion); assert(maxVersion(*self, impl) <= writeVersion);
setMaxVersion(*self, impl, writeVersion); setMaxVersion(*self, impl, writeVersion);
for (;; ++tls->accum.insert_iterations) { for (;; ++tls->accum.insert_iterations) {
if (key.size() == 0) { if (key.size() == 0) {
return *self; return self;
} }
auto &child = getOrCreateChild(*self, key.front(), writeVersion, tls); auto &child = getOrCreateChild(*self, key.front(), writeVersion, tls);
@@ -2953,7 +2950,7 @@ Node *insert(Node **self, std::span<const uint8_t> key,
child->parentsIndex = key.front(); child->parentsIndex = key.front();
setMaxVersion(child, impl, writeVersion); setMaxVersion(child, impl, writeVersion);
memcpy(child->partialKey(), key.data() + 1, child->partialKeyLen); memcpy(child->partialKey(), key.data() + 1, child->partialKeyLen);
return child; return &child;
} }
self = &child; self = &child;
@@ -2996,7 +2993,7 @@ void addPointWrite(Node *&root, std::span<const uint8_t> key,
InternalVersionT writeVersion, WriteContext *tls, InternalVersionT writeVersion, WriteContext *tls,
ConflictSet::Impl *impl) { ConflictSet::Impl *impl) {
++tls->accum.point_writes; ++tls->accum.point_writes;
auto *n = insert(&root, key, writeVersion, tls, impl); auto *n = *insert(&root, key, writeVersion, tls, impl);
if (!n->entryPresent) { if (!n->entryPresent) {
++tls->accum.entries_inserted; ++tls->accum.entries_inserted;
auto *p = nextLogical(n); auto *p = nextLogical(n);
@@ -3063,41 +3060,16 @@ void addWriteRange(Node *&root, std::span<const uint8_t> begin,
} }
++tls->accum.range_writes; ++tls->accum.range_writes;
const bool beginIsPrefix = lcp == int(begin.size()); const bool beginIsPrefix = lcp == int(begin.size());
auto remaining = begin.subspan(0, lcp);
auto *n = root; Node **useAsRoot =
insert(&root, begin.subspan(0, lcp), writeVersion, tls, impl);
for (;;) { int consumed = lcp;
if (int(remaining.size()) <= n->partialKeyLen) {
break;
}
int i = longestCommonPrefix(n->partialKey(), remaining.data(),
n->partialKeyLen);
if (i != n->partialKeyLen) {
break;
}
auto *child = getChild(n, remaining[n->partialKeyLen]);
if (child == nullptr) {
break;
}
assert(maxVersion(n, impl) <= writeVersion);
setMaxVersion(n, impl, writeVersion);
remaining = remaining.subspan(n->partialKeyLen + 1,
remaining.size() - (n->partialKeyLen + 1));
n = child;
}
Node **useAsRoot = &getInTree(n, impl);
int consumed = lcp - remaining.size();
begin = begin.subspan(consumed, begin.size() - consumed); begin = begin.subspan(consumed, begin.size() - consumed);
end = end.subspan(consumed, end.size() - consumed); end = end.subspan(consumed, end.size() - consumed);
auto *beginNode = insert(useAsRoot, begin, writeVersion, tls, impl); auto *beginNode = *insert(useAsRoot, begin, writeVersion, tls, impl);
const bool insertedBegin = !beginNode->entryPresent; const bool insertedBegin = !beginNode->entryPresent;
@@ -3114,7 +3086,7 @@ void addWriteRange(Node *&root, std::span<const uint8_t> begin,
assert(writeVersion >= beginNode->entry.pointVersion); assert(writeVersion >= beginNode->entry.pointVersion);
beginNode->entry.pointVersion = writeVersion; beginNode->entry.pointVersion = writeVersion;
auto *endNode = insert(useAsRoot, end, writeVersion, tls, impl); auto *endNode = *insert(useAsRoot, end, writeVersion, tls, impl);
const bool insertedEnd = !endNode->entryPresent; const bool insertedEnd = !endNode->entryPresent;
@@ -3132,7 +3104,7 @@ void addWriteRange(Node *&root, std::span<const uint8_t> begin,
if (beginIsPrefix && insertedEnd) { if (beginIsPrefix && insertedEnd) {
// beginNode may have been invalidated when inserting end. TODO can we do // beginNode may have been invalidated when inserting end. TODO can we do
// better? // better?
beginNode = insert(useAsRoot, begin, writeVersion, tls, impl); beginNode = *insert(useAsRoot, begin, writeVersion, tls, impl);
assert(beginNode->entryPresent); assert(beginNode->entryPresent);
} }
@@ -3246,6 +3218,8 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
InternalVersionT::zero = tls.zero = oldestVersion; InternalVersionT::zero = tls.zero = oldestVersion;
assert(writeVersion >= newestVersionFullPrecision); assert(writeVersion >= newestVersionFullPrecision);
assert(tls.accum.entries_erased == 0);
assert(tls.accum.entries_inserted == 0);
if (oldestExtantVersion < writeVersion - kMaxCorrectVersionWindow) if (oldestExtantVersion < writeVersion - kMaxCorrectVersionWindow)
[[unlikely]] { [[unlikely]] {
@@ -3273,15 +3247,20 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
auto begin = std::span<const uint8_t>(w.begin.p, w.begin.len); auto begin = std::span<const uint8_t>(w.begin.p, w.begin.len);
auto end = std::span<const uint8_t>(w.end.p, w.end.len); auto end = std::span<const uint8_t>(w.end.p, w.end.len);
if (w.end.len > 0) { if (w.end.len > 0) {
keyUpdates += 3;
addWriteRange(root, begin, end, InternalVersionT(writeVersion), &tls, addWriteRange(root, begin, end, InternalVersionT(writeVersion), &tls,
this); this);
} else { } else {
keyUpdates += 2;
addPointWrite(root, begin, InternalVersionT(writeVersion), &tls, this); addPointWrite(root, begin, InternalVersionT(writeVersion), &tls, this);
} }
} }
// Run gc at least 150% the rate we're inserting entries
keyUpdates +=
(std::max<int64_t>(
tls.accum.entries_inserted - tls.accum.entries_erased, 0) *
3) >>
1;
memory_bytes.set(totalBytes); memory_bytes.set(totalBytes);
point_writes_total.add(tls.accum.point_writes); point_writes_total.add(tls.accum.point_writes);
range_writes_total.add(tls.accum.range_writes); range_writes_total.add(tls.accum.range_writes);
+8
View File
@@ -76,6 +76,14 @@ void workload(weaselab::ConflictSet *cs) {
} else { } else {
w.begin.len = k.size(); w.begin.len = k.size();
cs->addWrites(&w, 1, version); cs->addWrites(&w, 1, version);
int64_t beginN = version - kWindowSize + rand() % kWindowSize;
auto b = numToKey(beginN);
auto e = numToKey(beginN + 1000);
w.begin.p = b.data();
w.begin.len = b.size();
w.end.p = e.data();
w.end.len = e.size();
cs->addWrites(&w, 1, version);
} }
} }
// GC // GC
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1 @@
:ゥゥゥゥゥゥゥゥゥゥ:::::
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More