A data structure for optimistic concurrency control on ranges of bitwise-lexicographically-ordered keys.

Intended to replace FoundationDB's skip list.

Hardware for all benchmarks is a mac m1 2020.

FoundationDB's benchmark

Skip list

New conflict set: 1.957 sec
                  0.639 Mtransactions/sec
                  2.555 Mkeys/sec
Detect only:      1.845 sec
                  0.678 Mtransactions/sec
                  2.710 Mkeys/sec
Skiplist only:    1.263 sec
                  0.990 Mtransactions/sec
                  3.960 Mkeys/sec
Performance counters:
               Build: 0.0546
                 Add: 0.0563
              Detect: 1.84
              D.Sort: 0.412
           D.Combine: 0.0141
         D.CheckRead: 0.671
   D.CheckIntraBatch: 0.0068
        D.MergeWrite: 0.592
      D.RemoveBefore: 0.146

Radix tree (this implementation)

New conflict set: 1.366 sec
                  0.915 Mtransactions/sec
                  3.660 Mkeys/sec
Detect only:      1.248 sec
                  1.002 Mtransactions/sec
                  4.007 Mkeys/sec
Skiplist only:    0.573 sec
                  2.182 Mtransactions/sec
                  8.730 Mkeys/sec
Performance counters:
               Build: 0.0594
                 Add: 0.0572
              Detect: 1.25
              D.Sort: 0.418
           D.Combine: 0.0149
         D.CheckRead: 0.232
   D.CheckIntraBatch: 0.0067
        D.MergeWrite: 0.341
      D.RemoveBefore: 0.232

Our benchmark

Skip list

ns/op	op/s	err%	total	benchmark
245.99	4,065,232.81	0.3%	0.01	point reads
265.93	3,760,430.49	0.2%	0.01	prefix reads
485.30	2,060,569.50	0.2%	0.01	range reads
449.60	2,224,195.17	0.4%	0.01	point writes
441.76	2,263,688.18	1.1%	0.01	prefix writes
245.42	4,074,647.54	2.4%	0.02	range writes
572.80	1,745,810.06	1.3%	0.01	monotonic increasing point writes
154,819.33	6,459.14	0.9%	0.01	worst case for radix tree

Radix tree (this implementation)

ns/op	op/s	err%	total	benchmark
19.17	52,163,930.66	0.1%	0.01	point reads
23.68	42,224,388.21	0.7%	0.01	prefix reads
63.30	15,797,506.06	0.9%	0.01	range reads
29.66	33,720,994.74	0.3%	0.01	point writes
43.50	22,987,781.25	1.0%	0.01	prefix writes
50.00	20,000,000.00	0.8%	0.01	range writes
103.25	9,684,786.47	2.9%	0.01	monotonic increasing point writes
1,181,500.00	846.38	2.3%	0.01	worst case for radix tree

"Real data" test

Point queries only, best of three runs. Gc ratio is the ratio of time spent doing garbage collection to time spent adding writes or doing garbage collection. Lower is better.

skip list

Check: 11.3385 seconds, 329.718 MB/s, Add: 5.35612 seconds, 131.072 MB/s, Gc ratio: 45.7173%

radix tree

Check: 2.48583 seconds, 1503.93 MB/s, Add: 2.12768 seconds, 329.954 MB/s, Gc ratio: 41.7943%

hash table

(The hash table implementation doesn't work on range queries, and its purpose is to provide an idea of how fast point queries can be)

Check: 1.83386 seconds, 2038.6 MB/s, Add: 0.601411 seconds, 1167.32 MB/s, Gc ratio: 48.9776%