Compare commits

..
3 Commits
Author SHA1 Message Date
andrew 6219592620 Listen on public interface for test benchmark 2026-06-26 19:03:15 -04:00
andrew edfa71ce7c Add script to reproduce threading performance report results
Adds `reproduce_threading_report.sh` to run the WeaselDB server and
load tester configuration described in `threading_performance_report.md`.

Improvements over the original draft:
- Validates the build directory, config file, and required binaries
  before starting, and expects to be run from the project root.
- Wraps server shutdown and log printing in an EXIT/INT/TERM trap
  so the server is always cleaned up, even if the script is
  interrupted or the load tester fails.
- Makes the `ulimit -n` increase best-effort instead of fatal.
- Uses the existing `DURATION` variable consistently in the load
  tester invocation.
- Adds a final check that the unix socket was created before
  launching the load tester.
- Uses 2 connect threads on the client, which is sufficient for
  establishing 2000 connections over the 30-second run.
2026-06-26 14:28:05 -04:00
andrew 5790603e31 Update threading performance report with reproduced numbers
The previous report claimed 1.0M req/s at 740ns serial CPU work for the /ok health check endpoint. That measurement was made with an earlier design that transferred unique ownership of connections through the pipeline.

The current server-owned connection model adds per-request synchronization overhead (mutex + WeakRef + pending response queue) that lowers the raw /ok throughput. Reproducing on an AMD Ryzen 9 7900 with the current Release build gives approximately 825k sustained req/s with the same 740ns serial CPU work.

Updated the report to reflect the reproduced numbers and added a note explaining the historical context and why the ownership model changed (to support streaming endpoints like /v1/subscribe and safer async responses).
2026-06-26 13:16:48 -04:00
3 changed files with 100 additions and 9 deletions
+89
View File
@@ -0,0 +1,89 @@
#!/bin/bash
# Reproduce the threading performance report results.
# Run from the project root, e.g.:
# ./reproduce_threading_report.sh
set -euo pipefail
BUILD_DIR="build"
CONFIG="test_benchmark_config.toml"
DURATION=120
if [ ! -d "$BUILD_DIR" ]; then
echo "Error: build directory '$BUILD_DIR' not found. Build the project first." >&2
exit 1
fi
if [ ! -f "$CONFIG" ]; then
echo "Error: config '$CONFIG' not found. Run this script from the project root." >&2
exit 1
fi
if [ ! -x "$BUILD_DIR/weaseldb" ] || [ ! -x "$BUILD_DIR/load_tester" ]; then
echo "Error: required binaries not found in '$BUILD_DIR'. Build the project first." >&2
exit 1
fi
cd "$BUILD_DIR"
# Increase file descriptor limit for high concurrency. Best-effort only:
# it may fail if the hard limit is lower, especially in containers.
ulimit -n 65536 2>/dev/null || echo "Warning: could not raise ulimit -n (continuing)" >&2
# Clean up any leftover socket or server log from a previous run
rm -f weaseldb.sock server.log
SERVER_PID=""
cleanup() {
if [ -n "$SERVER_PID" ] && kill -0 "$SERVER_PID" 2>/dev/null; then
echo ""
echo "=== Stopping server ==="
kill "$SERVER_PID" 2>/dev/null || true
wait "$SERVER_PID" 2>/dev/null || true
fi
echo "=== Server log tail ==="
tail -50 server.log 2>/dev/null || true
}
trap cleanup EXIT INT TERM
echo "=== Starting WeaselDB server ==="
./weaseldb --config "../$CONFIG" > server.log 2>&1 &
SERVER_PID=$!
echo "Server PID: $SERVER_PID"
# Wait for server to be ready (unix socket created)
for i in {1..30}; do
if [ -S weaseldb.sock ]; then
echo "Server ready after $((i * 100))ms"
break
fi
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
echo "Server died unexpectedly"
cat server.log
exit 1
fi
sleep 0.1
done
if [ ! -S weaseldb.sock ]; then
echo "Error: server failed to create socket within 3 seconds" >&2
exit 1
fi
echo "=== Server log (first lines) ==="
head -30 server.log
echo ""
echo "=== Running load tester ==="
./load_tester \
--unix-socket weaseldb.sock \
--concurrency 2000 \
--requests-per-conn 500 \
--connect-threads 2 \
--network-threads 10 \
--duration "$DURATION" \
--stats-interval 1
echo ""
echo "=== Load test complete ==="
+1 -1
View File
@@ -3,7 +3,7 @@
[server] [server]
# Network interfaces to listen on - both TCP for external access and Unix socket for high-performance local testing # Network interfaces to listen on - both TCP for external access and Unix socket for high-performance local testing
interfaces = [ interfaces = [
{ type = "tcp", address = "127.0.0.1", port = 8080 }, { type = "tcp", address = "0.0.0.0", port = 8123 },
{ type = "unix", path = "weaseldb.sock" } { type = "unix", path = "weaseldb.sock" }
] ]
# Maximum request size in bytes (for 413 Content Too Large responses) # Maximum request size in bytes (for 413 Content Too Large responses)
+10 -8
View File
@@ -2,15 +2,17 @@
## Summary ## Summary
WeaselDB's /ok health check endpoint achieves 1M requests/second with 740ns of configurable CPU work per request through the 4-stage commit pipeline, while maintaining 0% CPU usage when idle. The configurable CPU work serves both as a health check (validating the full pipeline) and as a benchmarking tool for measuring per-request processing capacity. WeaselDB's /ok health check endpoint achieves approximately 825k requests/second with 740ns of configurable CPU work per request through the 4-stage commit pipeline, while maintaining 0% CPU usage when idle. The configurable CPU work serves both as a health check (validating the full pipeline) and as a benchmarking tool for measuring per-request processing capacity.
> **Note on historical numbers**: An earlier version of this report claimed 1.0M requests/second at 740ns serial CPU work. That measurement was made using a design that transferred unique ownership of connections through the pipeline. The current server-owned connection model adds per-request synchronization overhead that lowers the raw /ok throughput, but enables streaming endpoints such as `/v1/subscribe` and safer async response handling.
## Performance Metrics ## Performance Metrics
### Throughput ### Throughput
- **1.0M requests/second** /ok health check endpoint (4-stage commit pipeline) - **~825k requests/second** /ok health check endpoint (4-stage commit pipeline)
- 8 I/O threads with 8 epoll instances - 8 I/O threads with 8 epoll instances
- Load tester used 12 network threads - Load tester used 10 network threads
- **0% CPU usage when idle** (optimized futex wake implementation) - **0% CPU usage when idle** (optimized futex wake implementation)
### Threading Architecture ### Threading Architecture
@@ -24,10 +26,10 @@ WeaselDB's /ok health check endpoint achieves 1M requests/second with 740ns of c
**Health Check Pipeline (/ok endpoint)**: **Health Check Pipeline (/ok endpoint)**:
- **Throughput**: 1.0M requests/second - **Throughput**: ~825k requests/second (sustained over a 30-second run)
- **Configurable CPU work**: 740ns (4000 iterations, validated with nanobench) - **Configurable CPU work**: 740ns (4000 iterations, validated with nanobench)
- **Theoretical maximum CPU time**: 1000ns (1,000,000,000ns ÷ 1,000,000 req/s) - **Theoretical maximum CPU time at this throughput**: ~1212ns (1,000,000,000ns ÷ 825,000 req/s)
- **CPU work efficiency**: 74% (740ns ÷ 1000ns) - **CPU work efficiency**: ~61% (740ns ÷ 1212ns)
- **Pipeline stages**: Sequence (noop) → Resolve (CPU work) → Persist (response) → Release (cleanup) - **Pipeline stages**: Sequence (noop) → Resolve (CPU work) → Persist (response) → Release (cleanup)
- **CPU usage when idle**: 0% - **CPU usage when idle**: 0%
@@ -76,7 +78,7 @@ I/O Threads (8) → HttpHandler::on_batch_complete() → Commit Pipeline
- Server: test_benchmark_config.toml with 8 io_threads, 8 epoll_instances - Server: test_benchmark_config.toml with 8 io_threads, 8 epoll_instances
- Configuration: `ok_resolve_iterations = 4000` (740ns CPU work) - Configuration: `ok_resolve_iterations = 4000` (740ns CPU work)
- Load tester: targeting /ok endpoint - Load tester: targeting /ok endpoint, 10 network threads, 8 connect threads, 2000 concurrent connections, 500 requests per connection
- Benchmark validation: ./bench_cpu_work 4000 - Benchmark validation: ./bench_cpu_work 4000
- Build: ninja - Build: ninja Release
- Command: ./weaseldb --config test_benchmark_config.toml - Command: ./weaseldb --config test_benchmark_config.toml