Andrew Noyes
76a7e17b29
Some checks failed
Tests / Clang total: 1096, failed: 2, passed: 1094
Tests / Release [gcc] total: 1096, failed: 2, passed: 1094
Tests / Release [gcc,aarch64] total: 824, failed: 2, passed: 822
Tests / Coverage total: 823, passed: 823
weaselab/conflict-set/pipeline/head There was a failure building this commit
I was incorrectly linking to an old build of conflict set, before the conflict-set name change. I don't know what the regression is from, but update the README for transparency now.
A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys.
Intended to replace FoundationDB's skip list.
Hardware for all benchmarks is a mac m1 2020.
FoundationDB's benchmark
Skip list
New conflict set: 1.962 sec
0.637 Mtransactions/sec
2.548 Mkeys/sec
Detect only: 1.842 sec
0.679 Mtransactions/sec
2.714 Mkeys/sec
Skiplist only: 1.261 sec
0.991 Mtransactions/sec
3.964 Mkeys/sec
Performance counters:
Build: 0.0597
Add: 0.0587
Detect: 1.84
D.Sort: 0.411
D.Combine: 0.0136
D.CheckRead: 0.67
D.CheckIntraBatch: 0.00671
D.MergeWrite: 0.592
D.RemoveBefore: 0.146
Radix tree (this implementation)
New conflict set: 1.660 sec
0.753 Mtransactions/sec
3.012 Mkeys/sec
Detect only: 1.524 sec
0.820 Mtransactions/sec
3.280 Mkeys/sec
Skiplist only: 0.844 sec
1.481 Mtransactions/sec
5.926 Mkeys/sec
Performance counters:
Build: 0.0699
Add: 0.0628
Detect: 1.52
D.Sort: 0.435
D.Combine: 0.0183
D.CheckRead: 0.37
D.CheckIntraBatch: 0.00801
D.MergeWrite: 0.473
D.RemoveBefore: 0.215
Our benchmark
Skip list
| ns/op | op/s | err% | total | benchmark |
|---|---|---|---|---|
| 246.99 | 4,048,700.59 | 0.2% | 0.01 | point reads |
| 260.16 | 3,843,784.65 | 0.1% | 0.01 | prefix reads |
| 493.35 | 2,026,953.19 | 0.1% | 0.01 | range reads |
| 462.05 | 2,164,289.23 | 0.6% | 0.01 | point writes |
| 448.19 | 2,231,205.25 | 0.9% | 0.01 | prefix writes |
| 255.83 | 3,908,845.72 | 1.5% | 0.02 | range writes |
| 582.63 | 1,716,349.02 | 1.3% | 0.01 | monotonic increasing point writes |
Radix tree (this implementation)
| ns/op | op/s | err% | total | benchmark |
|---|---|---|---|---|
| 19.42 | 51,483,206.67 | 0.3% | 0.01 | point reads |
| 58.43 | 17,115,612.57 | 0.1% | 0.01 | prefix reads |
| 216.09 | 4,627,766.60 | 0.2% | 0.01 | range reads |
| 28.35 | 35,267,567.72 | 0.2% | 0.01 | point writes |
| 43.43 | 23,026,226.17 | 0.2% | 0.01 | prefix writes |
| 50.00 | 20,000,000.00 | 0.0% | 0.01 | range writes |
| 92.38 | 10,824,863.69 | 4.1% | 0.01 | monotonic increasing point writes |
"Real data" test
Point queries only, best of three runs. Gc ratio is the ratio of time spent doing garbage collection to time spent adding writes or doing garbage collection. Lower is better.
skip list
Check: 11.3385 seconds, 329.718 MB/s, Add: 5.35612 seconds, 131.072 MB/s, Gc ratio: 45.7173%
radix tree
Check: 2.48583 seconds, 1503.93 MB/s, Add: 2.12768 seconds, 329.954 MB/s, Gc ratio: 41.7943%
hash table
(The hash table implementation doesn't work on range queries, and its purpose is to provide an idea of how fast point queries can be)
Check: 1.83386 seconds, 2038.6 MB/s, Add: 0.601411 seconds, 1167.32 MB/s, Gc ratio: 48.9776%