15 Commits

Author SHA1 Message Date
ed274c24d7 Just output metrics in the order declared
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1531, passed: 1531
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.71% (1677/1699) * Branch Coverage: 64.55% (1504/2330) * Complexity Density: 0.00 * Lines of Code: 1699 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-12 16:10:28 -07:00
cecfcc0da7 Fix build
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1531, passed: 1531
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.71% (1680/1702) * Branch Coverage: 64.50% (1508/2338) * Complexity Density: 0.00 * Lines of Code: 1702 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-12 14:52:42 -07:00
f6edde0e50 Fix data race
Some checks failed
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 14:49:46 -07:00
04ac41a7e7 Add memory usage and some "check" metrics
Some checks failed
Tests / Clang total: 1533, failed: 361, passed: 1172
Tests / Debug total: 1531, failed: 362, passed: 1169
Tests / SIMD fallback total: 1533, failed: 359, passed: 1174
Tests / Release [gcc] total: 1533, failed: 361, passed: 1172
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 14:22:16 -07:00
354920f86f Add newly-imported symbol for arm64 build
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1531, passed: 1531
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.69% (1661/1683) * Branch Coverage: 64.78% (1499/2314) * Complexity Density: 0.00 * Lines of Code: 1683 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-12 13:55:41 -07:00
bfd02503e7 Remove duplicate clang compiler warning action
Some checks failed
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1531, passed: 1531
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, failed: 1, passed: 1143
Tests / Coverage total: 1151, passed: 1151
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 13:44:51 -07:00
d0bd293f8d Try to fix Jenkins
Some checks failed
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Debug total: 1531, passed: 1531
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 13:38:11 -07:00
41e887c358 Try to fix symbol tests in CI
Some checks failed
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Clang - debug total: 1531, passed: 1531
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 13:34:00 -07:00
e394e3d96a Adapt comments from prometheus data model better 2024-07-12 13:29:10 -07:00
3288c583e4 Make metrics thread-safe
Some checks failed
Tests / Clang total: 1533, failed: 1, passed: 1532
Tests / Clang - debug total: 1531, passed: 1531
Tests / SIMD fallback total: 1533, failed: 1, passed: 1532
Tests / Release [gcc] total: 1533, failed: 1, passed: 1532
Tests / Release [gcc,aarch64] total: 1144, failed: 2, passed: 1142
Tests / Coverage total: 1151, passed: 1151
weaselab/conflict-set/pipeline/head There was a failure building this commit
Even concurrently with calling non-const methods on the associated
ConflictSet
2024-07-12 13:22:02 -07:00
ef14003781 Try to fix symbol tests in CI
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.75% (1653/1674) * Branch Coverage: 64.81% (1497/2310) * Complexity Density: 0.00 * Lines of Code: 1674 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-12 12:24:23 -07:00
3ac16bc966 Propose a metrics interface
Some checks failed
Tests / Clang total: 1533, failed: 2, passed: 1531
Tests / SIMD fallback total: 1533, failed: 2, passed: 1531
Tests / Release [gcc] total: 1533, failed: 2, passed: 1531
Tests / Release [gcc,aarch64] total: 1144, failed: 2, passed: 1142
Tests / Coverage total: 1151, passed: 1151
weaselab/conflict-set/pipeline/head There was a failure building this commit
2024-07-12 12:00:46 -07:00
1e82f7fe22 Fix compiler warnings
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.71% (1607/1628) * Branch Coverage: 65.61% (1473/2245) * Complexity Density: 0.00 * Lines of Code: 1628 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-11 19:33:57 -07:00
4182d904c5 Fix an issue where gc wasn't outpacing writes
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |2|0|2|0|:zzz:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.71% (1607/1628) * Branch Coverage: 65.61% (1473/2245) * Complexity Density: 0.00 * Lines of Code: 1628 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-11 16:40:18 -07:00
bd8ed4e7bd Bump version
All checks were successful
Tests / Clang total: 1533, passed: 1533
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Tests / SIMD fallback total: 1533, passed: 1533
Tests / Release [gcc] total: 1533, passed: 1533
GNU C Compiler (gcc) |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |2|0|2|0|:zzz:
Tests / Release [gcc,aarch64] total: 1144, passed: 1144
Tests / Coverage total: 1151, passed: 1151
Code Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 98.71% (1606/1627) * Branch Coverage: 65.63% (1472/2243) * Complexity Density: 0.00 * Lines of Code: 1627 #### Quality Gates Summary Output truncated.
weaselab/conflict-set/pipeline/head This commit looks good
2024-07-10 18:57:44 -07:00
13 changed files with 327 additions and 14 deletions

View File

@@ -358,7 +358,30 @@ void benchWorstCaseForRadixRangeRead() {
// }
}
void benchMetrics() {
ankerl::nanobench::Bench bench;
ConflictSet cs{0};
int count;
ConflictSet::MetricsV1 *m;
cs.getMetricsV1(&m, &count);
bench.batch(count);
bench.run("fetch metric", [&]() {
for (int i = 0; i < count; ++i) {
m[i].getValue();
}
});
}
void benchCreateAndDestroy() {
ankerl::nanobench::Bench bench;
bench.run("create and destroy", [&]() { ConflictSet cs{0}; });
}
int main(void) {
benchConflictSet();
benchWorstCaseForRadixRangeRead();
benchMetrics();
benchCreateAndDestroy();
}

View File

@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.18)
project(
conflict-set
VERSION 0.0.8
VERSION 0.0.9
DESCRIPTION
"A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys."
HOMEPAGE_URL "https://git.weaselab.dev/weaselab/conflict-set"
@@ -60,6 +60,8 @@ cmake_pop_check_state()
option(USE_SIMD_FALLBACK
"Use fallback implementations of functions that use SIMD" OFF)
option(DISABLE_TSAN "Disable TSAN" OFF)
# This is encouraged according to
# https://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.clientreq
include_directories(SYSTEM ${CMAKE_CURRENT_SOURCE_DIR}/third_party/valgrind)
@@ -245,7 +247,7 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR AND BUILD_TESTING)
endforeach()
# tsan tests
if(NOT CMAKE_CROSSCOMPILING AND NOT CMAKE_BUILD_TYPE STREQUAL Debug)
if(NOT CMAKE_CROSSCOMPILING AND NOT DISABLE_TSAN)
add_executable(tsan_driver ConflictSet.cpp FuzzTestDriver.cpp)
target_compile_options(tsan_driver PRIVATE ${TEST_FLAGS} -fsanitize=thread)
target_link_options(tsan_driver PRIVATE -fsanitize=thread)

View File

@@ -546,7 +546,50 @@ static_assert(kBytesPerKey - sizeof(Node0) >= kMinNodeSurplus);
constexpr int64_t kFreeListMaxMemory = 1 << 20;
struct Metric {
Metric *prev;
const char *name;
const char *help;
ConflictSet::MetricsV1::Type type;
std::atomic<double> value;
protected:
Metric(ConflictSet::Impl *impl, const char *name, const char *help,
ConflictSet::MetricsV1::Type type);
};
struct Gauge : private Metric {
Gauge(ConflictSet::Impl *impl, const char *name, const char *help)
: Metric(impl, name, help, ConflictSet::MetricsV1::Gauge) {}
void set(double value) {
this->value.store(value, std::memory_order_relaxed);
}
};
struct Counter : private Metric {
Counter(ConflictSet::Impl *impl, const char *name, const char *help)
: Metric(impl, name, help, ConflictSet::MetricsV1::Counter) {}
// Expensive. Accumulate locally and then call add instead of repeatedly
// calling add.
void add(double value) {
assert(value >= 0);
static_assert(std::atomic<double>::is_always_lock_free);
double old = this->value.load(std::memory_order_relaxed);
for (;;) {
double newVal = old + value;
if (this->value.compare_exchange_weak(old, newVal,
std::memory_order_relaxed)) {
break;
}
}
}
};
template <class T> struct BoundedFreeListAllocator {
Counter *allocated_total = nullptr;
Counter *released_total = nullptr;
static_assert(sizeof(T) >= sizeof(void *));
static_assert(std::derived_from<T, Node>);
static_assert(std::is_trivial_v<T>);
@@ -580,6 +623,7 @@ template <class T> struct BoundedFreeListAllocator {
}
T *allocate(int partialKeyCapacity) {
allocated_total->add(1);
T *result = allocate_helper(partialKeyCapacity);
if constexpr (!std::is_same_v<T, Node0>) {
memset(result->children, 0, sizeof(result->children));
@@ -596,6 +640,7 @@ template <class T> struct BoundedFreeListAllocator {
}
void release(T *p) {
released_total->add(1);
if (freeListBytes >= kFreeListMaxMemory) {
removeNode(p);
return safe_free(p, sizeof(T) + p->partialKeyCapacity);
@@ -1670,11 +1715,17 @@ struct SearchStepWise {
}
};
thread_local double accume_point_read = 0;
thread_local double accume_prefix_read = 0;
thread_local double accume_range_read = 0;
thread_local double accume_max_between = 0;
// Logically this is the same as performing firstGeq and then checking against
// point or range version according to cmp, but this version short circuits as
// soon as it can prove that there's no conflict.
bool checkPointRead(Node *n, const std::span<const uint8_t> key,
InternalVersionT readVersion, ConflictSet::Impl *impl) {
++accume_point_read;
#if DEBUG_VERBOSE && !defined(NDEBUG)
fprintf(stderr, "Check point read: %s\n", printable(key).c_str());
#endif
@@ -1752,6 +1803,7 @@ downLeftSpine:
// short circuits as soon as it can prove that there's no conflict.
bool checkPrefixRead(Node *n, const std::span<const uint8_t> key,
InternalVersionT readVersion, ConflictSet::Impl *impl) {
++accume_prefix_read;
#if DEBUG_VERBOSE && !defined(NDEBUG)
fprintf(stderr, "Check prefix read: %s\n", printable(key).c_str());
#endif
@@ -1993,6 +2045,7 @@ scan16(const InternalVersionT *vs, int begin, int end,
template <bool kAVX512>
bool checkMaxBetweenExclusive(Node *n, int begin, int end,
InternalVersionT readVersion) {
++accume_max_between;
assume(-1 <= begin);
assume(begin <= 256);
assume(-1 <= end);
@@ -2590,6 +2643,7 @@ template <bool kAVX512>
bool checkRangeReadImpl(Node *n, std::span<const uint8_t> begin,
std::span<const uint8_t> end,
InternalVersionT readVersion, ConflictSet::Impl *impl) {
++accume_range_read;
int lcp = longestCommonPrefix(begin.data(), end.data(),
std::min(begin.size(), end.size()));
if (lcp == int(begin.size()) && end.size() == begin.size() + 1 &&
@@ -3007,6 +3061,10 @@ Node *firstGeqPhysical(Node *n, const std::span<const uint8_t> key) {
struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
void check(const ReadRange *reads, Result *result, int count) {
assert(accume_point_read == 0);
assert(accume_prefix_read == 0);
assert(accume_range_read == 0);
assert(accume_max_between == 0);
for (int i = 0; i < count; ++i) {
const auto &r = reads[i];
auto begin = std::span<const uint8_t>(r.begin.p, r.begin.len);
@@ -3023,6 +3081,10 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
? Commit
: Conflict;
}
read_point_total.add(std::exchange(accume_point_read, 0));
read_prefix_total.add(std::exchange(accume_prefix_read, 0));
read_range_total.add(std::exchange(accume_range_read, 0));
read_range_node_total.add(std::exchange(accume_max_between, 0));
}
void addWrites(const WriteRange *writes, int count, int64_t writeVersion) {
@@ -3056,6 +3118,8 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
this);
}
}
memory_bytes.set(totalBytes);
}
// Spends up to `fuel` gc'ing, and returns its unused fuel. Reclaims memory
@@ -3070,8 +3134,12 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
rootMaxVersion = std::max(rootMaxVersion, oldestVersion);
n = nextPhysical(n);
}
for (; fuel > 0 && n != nullptr; --fuel) {
for (; fuel > 0 && n != nullptr;) {
rezero(n, oldestVersion);
// The "make sure gc keeps up with writes" calculations assume that we're
// scanning key by key, not node by node. Make sure we only spend fuel
// when there's a logical entry.
fuel -= n->entryPresent;
if (n->entryPresent && std::max(n->entry.pointVersion,
n->entry.rangeVersion) <= oldestVersion) {
// Any transaction n would have prevented from committing is
@@ -3121,6 +3189,8 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
}
#endif
keyUpdates = gcScanStep(keyUpdates);
memory_bytes.set(totalBytes);
}
int64_t getBytes() const { return totalBytes; }
@@ -3132,6 +3202,16 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
allocators.~NodeAllocators();
new (&allocators) NodeAllocators();
allocators.node0.allocated_total = &node0_allocated_total;
allocators.node3.allocated_total = &node3_allocated_total;
allocators.node16.allocated_total = &node16_allocated_total;
allocators.node48.allocated_total = &node48_allocated_total;
allocators.node256.allocated_total = &node256_allocated_total;
allocators.node0.released_total = &node0_released_total;
allocators.node3.released_total = &node3_released_total;
allocators.node16.released_total = &node16_released_total;
allocators.node48.released_total = &node48_released_total;
allocators.node256.released_total = &node256_released_total;
removalKeyArena = Arena{};
removalKey = {};
@@ -3156,8 +3236,14 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
// Intentionally not resetting totalBytes
}
explicit Impl(int64_t oldestVersion) { init(oldestVersion); }
~Impl() { destroyTree(root); }
explicit Impl(int64_t oldestVersion) {
init(oldestVersion);
initMetrics();
}
~Impl() {
destroyTree(root);
safe_free(metrics, metricsCount * sizeof(metrics[0]));
}
NodeAllocators allocators;
@@ -3173,8 +3259,71 @@ struct __attribute__((visibility("hidden"))) ConflictSet::Impl {
int64_t oldestVersionAtGcBegin;
int64_t newestVersionFullPrecision;
int64_t totalBytes = 0;
MetricsV1 *metrics;
int metricsCount = 0;
void initMetrics() {
metrics = (MetricsV1 *)safe_malloc(metricsCount * sizeof(metrics[0]));
for (auto [i, m] = std::make_tuple(metricsCount - 1, metricList); i >= 0;
--i, m = m->prev) {
metrics[i].name = m->name;
metrics[i].help = m->help;
metrics[i].p = m;
metrics[i].type = m->type;
}
}
Metric *metricList = nullptr;
#define GAUGE(name, help) \
Gauge name { this, #name, help }
#define COUNTER(name, help) \
Counter name { this, #name, help }
// ==================== METRICS DEFINITIONS ====================
COUNTER(node0_allocated_total,
"Total number of nodes of type \"Node0\" that have been allocated");
COUNTER(node3_allocated_total,
"Total number of nodes of type \"Node3\" that have been allocated");
COUNTER(node16_allocated_total,
"Total number of nodes of type \"Node16\" that have been allocated");
COUNTER(node48_allocated_total,
"Total number of nodes of type \"Node48\" that have been allocated");
COUNTER(node256_allocated_total,
"Total number of nodes of type \"Node256\" that have been allocated");
COUNTER(node0_released_total,
"Total number of nodes of type \"Node0\" that have been released");
COUNTER(node3_released_total,
"Total number of nodes of type \"Node3\" that have been released");
COUNTER(node16_released_total,
"Total number of nodes of type \"Node16\" that have been released");
COUNTER(node48_released_total,
"Total number of nodes of type \"Node48\" that have been released");
COUNTER(node256_released_total,
"Total number of nodes of type \"Node256\" that have been released");
GAUGE(memory_bytes, "Total number of bytes in use.");
COUNTER(read_point_total, "Total number of point reads checked.");
COUNTER(read_prefix_total, "Total number of prefix reads checked.");
COUNTER(read_range_total, "Total number of range reads checked.");
COUNTER(read_range_node_total,
"Total number of range checks of individual nodes while "
"checking a range read.");
// ==================== END METRICS DEFINITIONS ====================
#undef GAUGE
#undef COUNTER
void getMetricsV1(MetricsV1 **metrics, int *count) {
*metrics = this->metrics;
*count = metricsCount;
}
};
Metric::Metric(ConflictSet::Impl *impl, const char *name, const char *help,
ConflictSet::MetricsV1::Type type)
: prev(std::exchange(impl->metricList, this)), name(name), help(help),
type(type), value(0) {
++impl->metricsCount;
}
InternalVersionT maxVersion(Node *n, ConflictSet::Impl *impl) {
int index = n->parentsIndex;
n = n->parent;
@@ -3304,6 +3453,15 @@ void internal_destroy(ConflictSet::Impl *impl) {
int64_t internal_getBytes(ConflictSet::Impl *impl) { return impl->getBytes(); }
void internal_getMetricsV1(ConflictSet::Impl *impl,
ConflictSet::MetricsV1 **metrics, int *count) {
impl->getMetricsV1(metrics, count);
}
double internal_getMetricValue(const ConflictSet::MetricsV1 *metric) {
return ((Metric *)metric->p)->value.load(std::memory_order_relaxed);
}
// ==================== END IMPLEMENTATION ====================
// GCOVR_EXCL_START
@@ -3388,6 +3546,14 @@ void ConflictSet::setOldestVersion(int64_t oldestVersion) {
int64_t ConflictSet::getBytes() const { return internal_getBytes(impl); }
void ConflictSet::getMetricsV1(MetricsV1 **metrics, int *count) const {
return internal_getMetricsV1(impl, metrics, count);
}
double ConflictSet::MetricsV1::getValue() const {
return internal_getMetricValue(this);
}
ConflictSet::ConflictSet(int64_t oldestVersion)
: impl(internal_create(oldestVersion)) {}
@@ -3443,6 +3609,7 @@ static_assert(std::is_standard_layout_v<ConflictSet::Result>);
static_assert(std::is_standard_layout_v<ConflictSet::Key>);
static_assert(std::is_standard_layout_v<ConflictSet::ReadRange>);
static_assert(std::is_standard_layout_v<ConflictSet::WriteRange>);
static_assert(std::is_standard_layout_v<ConflictSet::MetricsV1>);
namespace {

View File

@@ -702,17 +702,36 @@ template <class ConflictSetImpl> struct TestDriver {
}
}
oldestVersion +=
arbitrary.bounded(10) ? arbitrary.bounded(10) : arbitrary.next();
oldestVersion = std::min(oldestVersion, writeVersion);
#ifdef THREAD_TEST
std::latch ready{1};
std::thread thread2{[&]() {
ready.count_down();
ConflictSet::MetricsV1 *m;
int count;
cs.getMetricsV1(&m, &count);
for (int i = 0; i < count; ++i) {
m[i].getValue();
}
}};
ready.wait();
#endif
CALLGRIND_START_INSTRUMENTATION;
cs.addWrites(writes, numPointWrites + numRangeWrites, v);
CALLGRIND_STOP_INSTRUMENTATION;
refImpl.addWrites(writes, numPointWrites + numRangeWrites, v);
oldestVersion +=
arbitrary.bounded(10) ? arbitrary.bounded(10) : arbitrary.next();
oldestVersion = std::min(oldestVersion, writeVersion);
cs.setOldestVersion(oldestVersion);
refImpl.setOldestVersion(oldestVersion);
#ifdef THREAD_TEST
thread2.join();
#endif
}
{
int numPointReads = arbitrary.bounded(100);
@@ -785,7 +804,15 @@ template <class ConflictSetImpl> struct TestDriver {
std::latch ready{1};
std::thread thread2{[&]() {
ready.count_down();
// Call all const methods
cs.check(reads, results3, numPointReads + numRangeReads);
cs.getBytes();
ConflictSet::MetricsV1 *m;
int count;
cs.getMetricsV1(&m, &count);
for (int i = 0; i < count; ++i) {
m[i].getValue();
}
}};
ready.wait();
#endif
@@ -794,6 +821,15 @@ template <class ConflictSetImpl> struct TestDriver {
cs.check(reads, results1, numPointReads + numRangeReads);
CALLGRIND_STOP_INSTRUMENTATION;
// Call remaining const methods
cs.getBytes();
ConflictSet::MetricsV1 *m;
int count;
cs.getMetricsV1(&m, &count);
for (int i = 0; i < count; ++i) {
m[i].getValue();
}
refImpl.check(reads, results2, numPointReads + numRangeReads);
auto compareResults = [reads](ConflictSet::Result *results1,

13
Jenkinsfile vendored
View File

@@ -48,6 +48,17 @@ pipeline {
recordIssues(tools: [clang()])
}
}
stage('Debug') {
agent {
dockerfile {
args '-v /home/jenkins/ccache:/ccache'
reuseNode true
}
}
steps {
CleanBuildAndTest("-DCMAKE_BUILD_TYPE=Debug")
}
}
stage('SIMD fallback') {
agent {
dockerfile {
@@ -106,7 +117,7 @@ pipeline {
}
}
steps {
CleanBuildAndTest("-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_FLAGS=--coverage -DCMAKE_CXX_FLAGS=--coverage -DCMAKE_BUILD_TYPE=Debug")
CleanBuildAndTest("-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_FLAGS=--coverage -DCMAKE_CXX_FLAGS=--coverage -DCMAKE_BUILD_TYPE=Debug -DDISABLE_TSAN=ON")
sh '''
gcovr -f ConflictSet.cpp --cobertura > build/coverage.xml
'''

View File

@@ -129,6 +129,16 @@ int main(int argc, const char **argv) {
close(fd);
}
ConflictSet::MetricsV1 *metrics;
int metricsCount;
cs.getMetricsV1(&metrics, &metricsCount);
for (int i = 0; i < metricsCount; ++i) {
printf("# HELP %s %s\n", metrics[i].name, metrics[i].help);
printf("# TYPE %s %s\n", metrics[i].name,
metrics[i].type == metrics[i].Counter ? "counter" : "gauge");
printf("%s %g\n", metrics[i].name, metrics[i].getValue());
}
printf("Check: %g seconds, %g MB/s, Add: %g seconds, %g MB/s, Gc ratio: "
"%g%%, Peak idle memory: %g\n",
checkTime, checkBytes / checkTime * 1e-6, addTime,

View File

@@ -42,7 +42,7 @@ std::span<const uint8_t> copyToArena(Arena &arena,
}
using Version = int64_t;
#define force_inline __attribute__((always_inline))
#define force_inline inline __attribute__((always_inline))
using StringRef = std::span<const uint8_t>;
struct KeyRangeRef {
@@ -861,6 +861,16 @@ void internal_destroy(ConflictSet::Impl *impl) {
int64_t internal_getBytes(ConflictSet::Impl *impl) { return impl->totalBytes; }
void internal_getMetricsV1(ConflictSet::Impl *impl,
ConflictSet::MetricsV1 **metrics, int *count) {
*metrics = nullptr;
*count = 0;
}
double internal_getMetricValue(const ConflictSet::MetricsV1 *metric) {
return 0;
}
void ConflictSet::check(const ReadRange *reads, Result *results,
int count) const {
internal_check(impl, reads, results, count);
@@ -877,6 +887,14 @@ void ConflictSet::setOldestVersion(int64_t oldestVersion) {
int64_t ConflictSet::getBytes() const { return internal_getBytes(impl); }
void ConflictSet::getMetricsV1(MetricsV1 **metrics, int *count) const {
return internal_getMetricsV1(impl, metrics, count);
}
double ConflictSet::MetricsV1::getValue() const {
return internal_getMetricValue(this);
}
ConflictSet::ConflictSet(int64_t oldestVersion)
: impl(internal_create(oldestVersion)) {}

View File

@@ -1,3 +1,5 @@
__aarch64_cas8_relax
__getauxval@GLIBC_2.17
__stack_chk_fail@GLIBC_2.17
__stack_chk_guard@GLIBC_2.17
abort@GLIBC_2.17

View File

@@ -13,5 +13,7 @@ __ZN8weaselab11ConflictSetC2Ex
__ZN8weaselab11ConflictSetD1Ev
__ZN8weaselab11ConflictSetD2Ev
__ZN8weaselab11ConflictSetaSEOS0_
__ZNK8weaselab11ConflictSet12getMetricsV1EPPNS0_9MetricsV1EPi
__ZNK8weaselab11ConflictSet5checkEPKNS0_9ReadRangeEPNS0_6ResultEi
__ZNK8weaselab11ConflictSet8getBytesEv
__ZNK8weaselab11ConflictSet9MetricsV18getValueEv

View File

@@ -1,6 +1,7 @@
#include "ConflictSet.h"
#include <cassert>
#include <cstdio>
using namespace weaselab;
@@ -21,4 +22,14 @@ int main(void) {
assert(result == ConflictSet::Conflict);
int64_t bytes = cs.getBytes();
assert(bytes > 0);
ConflictSet::MetricsV1 *metrics;
int metricsCount;
cs.getMetricsV1(&metrics, &metricsCount);
for (int i = 0; i < metricsCount; ++i) {
printf("# HELP %s %s\n", metrics[i].name, metrics[i].help);
printf("# TYPE %s %s\n", metrics[i].name,
metrics[i].type == metrics[i].Counter ? "counter" : "gauge");
printf("%s %g\n", metrics[i].name, metrics[i].getValue());
}
}

View File

@@ -88,6 +88,35 @@ struct __attribute__((__visibility__("default"))) ConflictSet {
/** Returns the total bytes in use by this ConflictSet */
int64_t getBytes() const;
/** Experimental! */
struct MetricsV1 {
/** A null-terminated string with static lifetime. Identifies this metric.
* Matches the regex [a-zA-Z_:][a-zA-Z0-9_:]*
*/
const char *name;
/** A null-terminated string with static lifetime. Describes this metric.
* May contain any sequence of UTF-8 characters, but the backslash and the
* line feed characters are escaped as \\ and \n, respectively.
*/
const char *help;
/** Counters are >= 0 and non-decreasing. Gauges are any value. */
enum Type { Counter, Gauge } type;
/** Get the most up-to-date (best effort) value for this metric.
* Thread-safe. */
double getValue() const;
/** @private */
void *p;
};
/** Experimental! Store a pointer to an array of MetricsV1 (owned by the
* ConflictSet) to `*metrics`, and its length to `*count`. This function makes
* no guarantees about the contents of the metrics (e.g. names, help text, and
* the meaning of values). A correct implementation is free to return nonsense
* or nothing at all. Not intended to be inspected programmatically. Only
* intended to be plumbed along to e.g. Prometheus. Callers may repeatedly
* call `getValue` on the metrics they're interested in. */
void getMetricsV1(MetricsV1 **metrics, int *count) const;
#if __cplusplus > 199711L
ConflictSet(ConflictSet &&) noexcept;
ConflictSet &operator=(ConflictSet &&) noexcept;

View File

@@ -6,12 +6,14 @@ ConflictSet_getBytes
ConflictSet_setOldestVersion
_ZN8weaselab11ConflictSet16setOldestVersionEl
_ZN8weaselab11ConflictSet9addWritesEPKNS0_10WriteRangeEil
_ZN8weaselab11ConflictSetaSEOS0_
_ZN8weaselab11ConflictSetC1El
_ZN8weaselab11ConflictSetC1EOS0_
_ZN8weaselab11ConflictSetC2El
_ZN8weaselab11ConflictSetC1El
_ZN8weaselab11ConflictSetC2EOS0_
_ZN8weaselab11ConflictSetC2El
_ZN8weaselab11ConflictSetD1Ev
_ZN8weaselab11ConflictSetD2Ev
_ZN8weaselab11ConflictSetaSEOS0_
_ZNK8weaselab11ConflictSet12getMetricsV1EPPNS0_9MetricsV1EPi
_ZNK8weaselab11ConflictSet5checkEPKNS0_9ReadRangeEPNS0_6ResultEi
_ZNK8weaselab11ConflictSet8getBytesEv
_ZNK8weaselab11ConflictSet9MetricsV18getValueEv

View File

@@ -8,4 +8,4 @@ free@GLIBC_2.2.5
malloc@GLIBC_2.2.5
memcpy@GLIBC_2.14
memmove@GLIBC_2.2.5
memset@GLIBC_2.2.5
memset@GLIBC_2.2.5