weaseldb/design.md

WeaselDB Development Guide

Table of Contents

1. Project Overview
2. Quick Start
3. Architecture
4. Development Guidelines
5. Common Patterns
6. Reference

IMPORTANT: Read style.md first - contains mandatory C++ coding standards, threading rules, and testing guidelines that must be followed for all code changes.

---

Project Overview

WeaselDB is a high-performance write-side database component designed for systems where reading and writing are decoupled. The system focuses exclusively on handling transactional commits with optimistic concurrency control, while readers are expected to maintain their own queryable representations by subscribing to change streams.

Key Features

• Ultra-fast arena allocation (~1ns vs ~20-270ns for malloc)
• High-performance JSON parsing with streaming support and SIMD optimization
• Multi-threaded networking using multiple epoll instances with unified I/O thread pool
• Configurable epoll instances to eliminate kernel-level contention
• Optimized memory management with arena allocation and efficient copying
• Factory pattern safety ensuring correct object lifecycle management

---

Quick Start

Build System

Use CMake with C++20 and always use ninja (see style.md for build details):

mkdir -p build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
ninja

Testing & Development

Run all tests:

ninja test    # or ctest

Individual targets:

• ./test_arena - Arena allocator unit tests
• ./test_commit_request - JSON parsing and validation tests
• ./test_http_handler - HTTP protocol handling tests
• ./test_metric - Metrics system tests
• ./test_api_url_parser - API URL parsing tests
• ./test_reference - Reference counting system tests
• ./test_server_connection_return - Connection lifecycle tests

Benchmarking:

• ./bench_arena - Memory allocation performance
• ./bench_commit_request - JSON parsing performance
• ./bench_cpu_work - CPU work benchmarking utility
• ./bench_format_comparison - String formatting performance
• ./bench_metric - Metrics system performance
• ./bench_parser_comparison - Compare vs nlohmann::json and RapidJSON
• ./bench_reference - Reference counting performance
• ./bench_thread_pipeline - Lock-free pipeline performance

Debug tools:

• ./debug_arena - Analyze arena allocator behavior

Load Testing:

• ./load_tester - A tool to generate load against the server for performance and stability analysis.

Dependencies

System requirements:

• weaseljson - Must be installed system-wide (high-performance JSON parser)
• gperf - System requirement for perfect hash generation

Auto-fetched:

• simdutf - SIMD base64 encoding/decoding
• toml11 - TOML configuration parsing
• doctest - Testing framework
• nanobench - Benchmarking library
• nlohmann/json - JSON library (used in benchmarks)
• RapidJSON - High-performance JSON library (used in benchmarks)
• llhttp - Fast HTTP parser

---

Architecture

Core Components

Arena Allocator (src/arena.hpp)

Ultra-fast memory allocator optimized for request/response patterns:

• ~1ns allocation time vs ~20-270ns for malloc
• Lazy initialization with geometric block growth (doubling strategy)
• Intrusive linked list design for minimal memory overhead
• Memory-efficient reset that keeps the first block and frees others
• STL-compatible interface via ArenaStlAllocator
• O(1) amortized allocation with proper alignment handling
• Move semantics for efficient transfers
• Thread-safe per-connection usage via exclusive ownership model

Networking Layer

Server (src/server.{hpp,cpp}):

• High-performance multi-threaded networking using multiple epoll instances with unified I/O thread pool
• Configurable epoll instances to eliminate kernel-level epoll_ctl contention (default: 2, max: io_threads)
• Round-robin thread-to-epoll assignment distributes I/O threads across epoll instances
• Connection distribution keeps accepted connections on same epoll, returns via round-robin
• Factory pattern construction via Server::create() ensures you can only get a Ref<Server>
• Safe shutdown mechanism with async-signal-safe shutdown() method
• Connection ownership management with automatic cleanup on server destruction
• Pluggable protocol handlers via ConnectionHandler interface
• EPOLL_EXCLUSIVE on listen socket across all epoll instances prevents thundering herd

Connection (src/connection.{hpp,cpp}):

• Efficient per-connection state management with arena-based memory allocation
• Safe ownership transfer between server threads and protocol handlers
• Automatic cleanup on connection closure or server shutdown
• Handler interface isolation - only exposes necessary methods to protocol handlers
• Protocol-specific data: user_data void* for custom handler data

ConnectionHandler Interface (src/connection_handler.hpp):

• Abstract protocol interface decoupling networking from application logic
• Ownership transfer support allowing handlers to take connections for async processing
• Streaming data processing with partial message handling
• Connection lifecycle hooks for initialization and cleanup

Thread Pipeline (src/thread_pipeline.hpp)

A high-performance, multi-stage, lock-free pipeline for inter-thread communication.

• Lock-Free Design: Uses a shared ring buffer with atomic counters for coordination, avoiding locks for maximum throughput.
• Multi-Stage Processing: Allows items (like connections or data packets) to flow through a series of processing stages (e.g., from I/O threads to worker threads).
• Batching Support: Enables efficient batch processing of items to reduce overhead.
• RAII Guards: Utilizes RAII (StageGuard, ProducerGuard) to ensure thread-safe publishing and consumption of items in the pipeline, even in the presence of exceptions.

Parsing Layer

JSON Commit Request Parser (src/json_commit_request_parser.{hpp,cpp}):

• High-performance JSON parser using weaseljson library
• Streaming parser support for incremental parsing of network data
• gperf-optimized token recognition for fast JSON key parsing
• Base64 decoding using SIMD-accelerated simdutf
• Comprehensive validation of transaction structure
• Perfect hash table lookup for JSON keys using gperf
• Zero hash collisions for known JSON tokens eliminates branching

Parser Interface (src/commit_request_parser.hpp):

• Abstract base class for commit request parsers
• Format-agnostic parsing interface supporting multiple serialization formats
• Streaming and one-shot parsing modes
• Standardized error handling across parser implementations

Data Model

Commit Request Data Model (src/commit_request.hpp):

• Format-agnostic data structure for representing transactional commits
• Arena-backed string storage with efficient memory management
• Move-only semantics for optimal performance
• Builder pattern for constructing commit requests
• String views pointing to arena-allocated memory to avoid unnecessary copying

Metrics System (src/metric.{hpp,cpp})

High-Performance Metrics Implementation:

• Thread-local counters/histograms with single writer for performance
• Global gauges with lock-free atomic CAS operations for multi-writer scenarios
• SIMD-optimized histogram bucket updates using AVX instructions for high throughput
• Arena allocator integration for efficient memory management during rendering

Threading Model:

• Counters: Per-thread storage, single writer, atomic write in Counter::inc(), atomic read in render thread
• Histograms: Per-thread storage, single writer, per-histogram mutex serializes all access (observe and render)
• Gauges: Lock-free atomic operations using std::bit_cast for double precision
• Thread cleanup: Automatic accumulation of thread-local state into global state on destruction

Prometheus Compatibility:

• Standard metric types with proper label handling and validation
• Bucket generation helpers for linear/exponential histogram distributions
• Callback-based metrics for dynamic values
• UTF-8 validation using simdutf for label values

Configuration & Optimization

Configuration System (src/config.{hpp,cpp}):

• TOML-based configuration using toml11 library
• Structured configuration with server, commit, and subscription sections
• Default fallback values for all configuration options
• Type-safe parsing with validation and bounds checking
• See config.md for complete configuration documentation

JSON Token Optimization (src/json_tokens.gperf, src/json_token_enum.hpp):

• Perfect hash table generated by gperf for O(1) JSON key lookup
• Compile-time token enumeration for type-safe key identification
• Minimal perfect hash reduces memory overhead and improves cache locality
• Build-time code generation ensures optimal performance

Transaction Data Model

CommitRequest Structure

CommitRequest {
  - request_id: Optional unique identifier
  - leader_id: Expected leader for consistency
  - read_version: Snapshot version for preconditions
  - preconditions[]: Optimistic concurrency checks
    - point_read: Single key existence/content validation
    - range_read: Range-based consistency validation
  - operations[]: Ordered mutation operations
    - write: Set key-value pair
    - delete: Remove single key
    - range_delete: Remove key range
}

Memory Management Model

Connection Ownership Lifecycle

1. Creation: Server creates connections and stores them in registry
2. Processing: I/O threads access connections via registry lookup
3. Handler Access: Handlers receive Connection& references, server retains ownership
4. Async Processing: Handlers use WeakRef for safe async access
5. Safety: Connection mutex synchronizes concurrent access between threads
6. Cleanup: RAII ensures proper resource cleanup when connections are destroyed

Arena Memory Lifecycle

1. Request Processing: Handler creates request-scoped arena for parsing request data
2. Response Generation: Handler uses same arena for response construction (headers, JSON, etc.)
3. Response Queuing: Handler calls conn->append_message() passing span + arena ownership
4. Response Writing: I/O thread writes messages to socket, arena freed after completion

Note

: Call conn->reset() periodically to reclaim arena memory. Best practice is after all outgoing bytes have been written.

Threading Model and EPOLLONESHOT

EPOLLONESHOT Design Rationale: WeaselDB uses EPOLLONESHOT for all connection file descriptors to enable safe multi-threaded ownership transfer without complex synchronization:

Key Benefits:

1. Automatic fd disarming - When epoll triggers an event, the fd is automatically removed from epoll monitoring
2. Race-free ownership transfer - Handlers can safely take connection ownership and move to other threads
3. Zero-coordination async processing - No manual synchronization needed between network threads and handler threads

Threading Flow:

1. Event Trigger: Network thread gets epoll event → connection auto-disarmed via ONESHOT
2. Safe Transfer: Handler can take ownership (std::move(conn_ptr)) with no epoll interference
3. Async Processing: Connection processed on handler thread while epoll cannot trigger spurious events
4. Return & Re-arm: Internal server method re-arms fd with epoll_ctl(EPOLL_CTL_MOD) via Server::release_back_to_server()

Performance Trade-off:

• Cost: One epoll_ctl(MOD) syscall per connection return (~100-200ns)
• Benefit: Eliminates complex thread synchronization and prevents race conditions
• Alternative cost: Manual EPOLL_CTL_DEL/ADD + locking would be significantly higher

Without EPOLLONESHOT risks:

• Multiple threads processing same fd simultaneously
• Use-after-move when network thread accesses transferred connection
• Complex synchronization between epoll events and ownership transfers

This design enables the async handler pattern where connections can be safely moved between threads for background processing while maintaining high performance and thread safety.

API Endpoints

The system implements a RESTful API. See api.md for comprehensive API documentation including request/response formats, examples, and error codes.

Design Principles

1. Performance-first - Every component optimized for high throughput
2. Scalable concurrency - Multiple epoll instances eliminate kernel contention
3. Memory efficiency - Arena allocation eliminates fragmentation
4. Efficient copying - Minimize unnecessary copies while accepting required ones
5. Streaming-ready - Support incremental processing
6. Type safety - Compile-time validation where possible
7. Resource management - RAII and move semantics throughout

---

Development Guidelines

Code Style & Conventions

See style.md for comprehensive C++ coding standards and conventions.

Critical Implementation Rules

• Server Creation: Always use Server::create() factory method - direct construction is impossible
• Connection Creation: Only the Server can create connections - no public constructor or factory method
• Connection Ownership: Use unique_ptr semantics for safe ownership transfer between components
• Arena Allocator Pattern: Always use Arena for temporary allocations within request processing
• String View Usage: Prefer std::string_view over std::string when pointing to arena-allocated memory
• Ownership Transfer: Use Server::release_back_to_server() for returning connections to server from handlers
• JSON Token Lookup: Use the gperf-generated perfect hash table in json_tokens.hpp for O(1) key recognition
• Base64 Handling: Always use simdutf for base64 encoding/decoding for performance
• Thread Safety: Connection ownership transfers are designed to be thread-safe with proper RAII cleanup

Project Structure

• src/ - Core headers and implementation files
• tests/ - doctest-based unit tests
• benchmarks/ - nanobench performance tests
• tools/ - Debugging and analysis utilities
• build/ - CMake-generated files including json_tokens.cpp

Extension Points

Adding New Protocol Handlers

1. Inherit from ConnectionHandler in src/connection_handler.hpp
2. Implement on_data_arrived() with proper ownership semantics
3. Use connection's arena allocator for temporary allocations: conn->get_arena()
4. Handle partial messages and streaming protocols appropriately
5. Use Server::release_back_to_server() if taking ownership for async processing
6. Add corresponding test cases and integration tests
7. Consider performance implications of ownership transfers

Adding New Parsers

1. Inherit from CommitRequestParser in src/commit_request_parser.hpp
2. Implement both streaming and one-shot parsing modes
3. Use arena allocation for all temporary string storage
4. Add corresponding test cases in tests/
5. Add benchmark comparisons in benchmarks/

Performance Guidelines

• Memory: Arena allocation eliminates fragmentation - use it for all request-scoped data
• CPU: Perfect hashing and SIMD operations are critical paths - avoid alternatives
• I/O: Streaming parser design supports incremental network data processing
• Cache: String views avoid copying, keeping data cache-friendly

Configuration & Testing

• Configuration: All configuration is TOML-based using config.toml (see config.md)
• Testing Strategy: Run unit tests, benchmarks, and debug tools before submitting changes
• Build System: CMake generates gperf hash tables at build time
• Testing Guidelines: See style.md for comprehensive testing standards including synchronization rules

---

Common Patterns

Factory Method Patterns

Server Creation

// Server must be created via factory method
auto server = Server::create(config, handler);

// Never create on stack or with make_shared (won't compile):
// Server server(config, handler);  // Compiler error - constructor private
// auto server = std::make_shared<Server>(config, handler);  // Compiler error

Connection Creation (Server-Only)

Only Server can create connections (using private constructor via friend access)

ConnectionHandler Implementation Patterns

Simple Synchronous Handler

class HttpHandler : public ConnectionHandler {
public:
  void on_data_arrived(std::string_view data, std::unique_ptr<Connection>& conn_ptr) override {
    // Parse HTTP request using connection's arena
    Arena& arena = conn_ptr->get_arena();

    // Generate response
    conn_ptr->append_message("HTTP/1.1 200 OK\r\n\r\nHello World");

    // Server retains ownership
  }
};

Async Handler with Ownership Transfer

class AsyncHandler : public ConnectionHandler {
public:
  void on_data_arrived(std::string_view data, std::unique_ptr<Connection>& conn_ptr) override {
    // Take ownership for async processing
    auto connection = std::move(conn_ptr); // conn_ptr is now null

    work_queue.push([connection = std::move(connection)](std::string_view data) mutable {
      // Process asynchronously
      connection->append_message("Async response");

      // Return ownership to server when done
      Server::release_back_to_server(std::move(connection));
    });
  }
};

Batching Handler with User Data

class BatchingHandler : public ConnectionHandler {
public:
  void on_connection_established(Connection &conn) override {
    // Allocate some protocol-specific data and attach it to the connection
    conn.user_data = new MyProtocolData();
  }

  void on_connection_closed(Connection &conn) override {
    // Free the protocol-specific data
    delete static_cast<MyProtocolData*>(conn.user_data);
  }

  void on_data_arrived(std::string_view data,
                       std::unique_ptr<Connection> &conn_ptr) override {
    // Process data and maybe store some results in the user_data
    auto* proto_data = static_cast<MyProtocolData*>(conn_ptr->user_data);
    proto_data->process(data);
  }

  void on_batch_complete(std::span<std::unique_ptr<Connection>> batch) override {
    // Process a batch of connections
    for (auto& conn_ptr : batch) {
      if (conn_ptr) {
        auto* proto_data = static_cast<MyProtocolData*>(conn_ptr->user_data);
        if (proto_data->is_ready()) {
          // This connection is ready for the next stage, move it to the pipeline
          pipeline_.push(std::move(conn_ptr));
        }
      }
    }
  }

private:
  MyProcessingPipeline pipeline_;
};

Streaming "yes" Handler

class YesHandler : public ConnectionHandler {
public:
  void on_connection_established(Connection &conn) override {
    // Write an initial "y\n"
    conn.append_message("y\n");
  }

  void on_write_progress(std::unique_ptr<Connection> &conn) override {
    if (conn->outgoing_bytes_queued() == 0) {
      // Don't use an unbounded amount of memory
      conn->reset();
      // Write "y\n" repeatedly
      conn->append_message("y\n");
    }
  }
};

Memory Management Patterns

Arena-Based String Handling

// Preferred: String view with arena allocation to minimize copying
std::string_view process_json_key(const char* data, Arena& arena);

// Avoid: Unnecessary string copies
std::string process_json_key(const char* data);

Safe Connection Ownership Transfer

// In handler - take ownership for background processing
Connection* raw_conn = conn_ptr.release();

// Process on worker thread
background_processor.submit([raw_conn]() {
  // Do work...
  raw_conn->append_message("Background result");

  // Return to server safely (handles server destruction)
  Server::release_back_to_server(std::unique_ptr<Connection>(raw_conn));
});

Data Construction Patterns

Builder Pattern Usage

CommitRequest request = CommitRequestBuilder(arena)
    .request_id("example-id")
    .leader_id("leader-123")
    .read_version(42)
    .build();

Error Handling Pattern

enum class ParseResult { Success, InvalidJson, MissingField };
ParseResult parse_commit_request(const char* json, CommitRequest& out);

---

Reference

Build Targets

Test Executables:

• test_arena - Arena allocator functionality tests
• test_commit_request - JSON parsing and validation tests
• test_metric - Metrics system functionality tests
• Main server executable (compiled from src/main.cpp)

Benchmark Executables:

• bench_arena - Arena allocator performance benchmarks
• bench_commit_request - JSON parsing performance benchmarks
• bench_parser_comparison - Comparison benchmarks vs nlohmann::json and RapidJSON
• bench_metric - Metrics system performance benchmarks

Debug Tools:

• debug_arena - Debug tool for arena allocator analysis

Performance Characteristics

Memory Allocation:

• ~1ns allocation time vs standard allocators
• Bulk deallocation eliminates individual free() calls
• Optimized geometric growth uses current block size for doubling strategy
• Alignment-aware allocation prevents performance penalties

JSON Parsing:

• Streaming parser handles large payloads efficiently
• Incremental processing suitable for network protocols
• Arena storage eliminates string allocation overhead
• SIMD-accelerated base64 decoding using simdutf for maximum performance
• Perfect hash table provides O(1) JSON key lookup via gperf
• Zero hash collisions for known JSON tokens eliminates branching

Build Notes

See style.md for comprehensive build configuration and CMake integration details.