weaselab/conflict-set
2
0
Fork 0
Code Issues 4 Packages Releases 13 Activity

Switch to new machine for benchmarks
Some checks failed
Tests / Clang total: 1479, passed: 1479
Details
Clang |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |0|0|0|0|:clap:
Details
Tests / Debug total: 1477, passed: 1477
Details
weaselab/conflict-set/pipeline/head There was a failure building this commit
Details

Browse Source
This commit is contained in:
Andrew Noyes
2024-08-02 18:18:36 -07:00
parent f1ad68109a
commit d1dc1247e1
1 changed files with 28 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
README.md
View File

@@ -1,86 +1,38 @@
A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys. A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys.
Intended to replace FoundationDB's skip list. Intended as an alternative to FoundationDB's skip list.
Hardware for all benchmarks is a mac m1 2020. Hardware for all benchmarks is an AMD Ryzen 9 7900 with (2x32GB) 5600MT/s CL28-34-34-89 1.35V RAM
# FoundationDB's benchmark # Microbenchmark
## Skip list ## Skip list
``` | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
New conflict set: 1.957 sec |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
0.639 Mtransactions/sec | 172.03 | 5,812,791.77 | 0.4% | 3,130.62 | 879.00 | 3.562 | 509.23 | 0.0% | 0.01 | `point reads`
2.555 Mkeys/sec | 167.44 | 5,972,130.71 | 0.2% | 3,065.14 | 862.27 | 3.555 | 494.30 | 0.0% | 0.01 | `prefix reads`
Detect only: 1.845 sec | 238.77 | 4,188,130.84 | 0.9% | 3,589.93 | 1,259.30 | 2.851 | 637.12 | 0.0% | 0.01 | `range reads`
0.678 Mtransactions/sec | 424.01 | 2,358,426.70 | 0.2% | 5,620.05 | 2,242.35 | 2.506 | 854.80 | 1.7% | 0.01 | `point writes`
2.710 Mkeys/sec | 418.45 | 2,389,780.56 | 0.4% | 5,525.07 | 2,211.05 | 2.499 | 831.71 | 1.7% | 0.01 | `prefix writes`
Skiplist only: 1.263 sec | 254.87 | 3,923,568.88 | 2.6% | 3,187.01 | 1,366.50 | 2.332 | 529.11 | 2.7% | 0.02 | `range writes`
0.990 Mtransactions/sec | 675.96 | 1,479,374.50 | 3.3% | 7,735.41 | 3,468.60 | 2.230 | 1,386.02 | 1.8% | 0.01 | `monotonic increasing point writes`
3.960 Mkeys/sec | 137,986.20 | 7,247.10 | 0.6% | 789,752.33 | 699,462.00 | 1.129 | 144,824.14 | 0.0% | 0.01 | `worst case for radix tree`
Performance counters: | 21.63 | 46,231,564.03 | 1.0% | 448.00 | 107.14 | 4.181 | 84.00 | 0.0% | 0.01 | `create and destroy`
Build: 0.0546
Add: 0.0563
Detect: 1.84
D.Sort: 0.412
D.Combine: 0.0141
D.CheckRead: 0.671
D.CheckIntraBatch: 0.0068
D.MergeWrite: 0.592
D.RemoveBefore: 0.146
```
## Radix tree (this implementation) ## Radix tree (this implementation)
``` | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
New conflict set: 1.366 sec |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
0.915 Mtransactions/sec | 17.03 | 58,732,967.93 | 0.6% | 276.28 | 87.96 | 3.141 | 52.15 | 0.4% | 0.01 | `point reads`
3.660 Mkeys/sec | 19.52 | 51,239,158.04 | 0.3% | 367.16 | 101.50 | 3.617 | 61.92 | 0.3% | 0.01 | `prefix reads`
Detect only: 1.248 sec | 47.74 | 20,947,676.63 | 0.5% | 998.16 | 247.43 | 4.034 | 161.64 | 0.2% | 0.01 | `range reads`
1.002 Mtransactions/sec | 23.14 | 43,207,824.89 | 0.4% | 408.18 | 121.64 | 3.356 | 70.20 | 0.3% | 0.01 | `point writes`
4.007 Mkeys/sec | 38.02 | 26,302,115.66 | 0.1% | 709.72 | 199.70 | 3.554 | 134.26 | 0.3% | 0.01 | `prefix writes`
Skiplist only: 0.573 sec | 44.28 | 22,583,559.17 | 0.9% | 825.19 | 233.10 | 3.540 | 141.48 | 0.2% | 0.01 | `range writes`
2.182 Mtransactions/sec | 85.50 | 11,695,990.63 | 0.5% | 1,488.16 | 455.68 | 3.266 | 289.22 | 0.1% | 0.01 | `monotonic increasing point writes`
8.730 Mkeys/sec | 338,388.50 | 2,955.18 | 3.3% | 4,097,087.00 | 1,809,996.00 | 2.264 | 759,645.00 | 0.0% | 0.01 | `worst case for radix tree`
Performance counters: | 84.84 | 11,787,313.59 | 1.4% | 1,716.02 | 440.50 | 3.896 | 271.00 | 0.0% | 0.01 | `create and destroy`
Build: 0.0594
Add: 0.0572
Detect: 1.25
D.Sort: 0.418
D.Combine: 0.0149
D.CheckRead: 0.232
D.CheckIntraBatch: 0.0067
D.MergeWrite: 0.341
D.RemoveBefore: 0.232
```
# Our benchmark
## Skip list
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 245.99 | 4,065,232.81 | 0.3% | 0.01 | `point reads`
| 265.93 | 3,760,430.49 | 0.2% | 0.01 | `prefix reads`
| 485.30 | 2,060,569.50 | 0.2% | 0.01 | `range reads`
| 449.60 | 2,224,195.17 | 0.4% | 0.01 | `point writes`
| 441.76 | 2,263,688.18 | 1.1% | 0.01 | `prefix writes`
| 245.42 | 4,074,647.54 | 2.4% | 0.02 | `range writes`
| 572.80 | 1,745,810.06 | 1.3% | 0.01 | `monotonic increasing point writes`
| 154,819.33 | 6,459.14 | 0.9% | 0.01 | `worst case for radix tree`
## Radix tree (this implementation)
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 20.25 | 49,372,759.86 | 0.3% | 0.01 | `point reads`
| 23.58 | 42,401,298.00 | 0.3% | 0.01 | `prefix reads`
| 64.12 | 15,595,463.14 | 0.8% | 0.01 | `range reads`
| 29.50 | 33,903,101.20 | 0.7% | 0.01 | `point writes`
| 46.76 | 21,384,036.19 | 1.2% | 0.01 | `prefix writes`
| 51.25 | 19,512,195.12 | 0.0% | 0.01 | `range writes`
| 109.51 | 9,131,469.31 | 3.6% | 0.01 | `monotonic increasing point writes`
| 1,153,875.00 | 866.65 | 1.6% | 0.01 | `worst case for radix tree`
# "Real data" test # "Real data" test
@@ -89,13 +41,13 @@ Point queries only, best of three runs. Gc ratio is the ratio of time spent doin
## skip list ## skip list
``` ```
Check: 11.3385 seconds, 329.718 MB/s, Add: 5.35612 seconds, 131.072 MB/s, Gc ratio: 45.7173% Check: 4.47891 seconds, 364.05 MB/s, Add: 4.55599 seconds, 123.058 MB/s, Gc ratio: 37.1145%
``` ```
## radix tree ## radix tree
``` ```
Check: 2.60639 seconds, 1434.36 MB/s, Add: 2.10911 seconds, 332.86 MB/s, Gc ratio: 46.3071% Check: 1.05813 seconds, 1540.97 MB/s, Add: 1.32071 seconds, 424.508 MB/s, Gc ratio: 42.2067%
``` ```
## hash table ## hash table
@@ -103,5 +55,5 @@ Check: 2.60639 seconds, 1434.36 MB/s, Add: 2.10911 seconds, 332.86 MB/s, Gc rati
(The hash table implementation doesn't work on range queries, and its purpose is to provide an idea of how fast point queries can be) (The hash table implementation doesn't work on range queries, and its purpose is to provide an idea of how fast point queries can be)
``` ```
Check: 1.83386 seconds, 2038.6 MB/s, Add: 0.601411 seconds, 1167.32 MB/s, Gc ratio: 48.9776% Check: 0.804094 seconds, 2027.81 MB/s, Add: 0.652952 seconds, 858.645 MB/s, Gc ratio: 35.3885%
``` ```