diff --git a/design.md b/design.md index 99fc8d6..d8a42fa 100644 --- a/design.md +++ b/design.md @@ -99,6 +99,47 @@ See `config.md` for complete configuration documentation. - **Minimal perfect hash** reduces memory overhead and improves cache locality - **Build-time code generation** ensures optimal performance +#### 7. **Server** (`src/server.{hpp,cpp}`) +- **High-performance multi-threaded networking** using epoll with thread pools +- **Factory pattern construction** via `Server::create()` ensures proper shared_ptr semantics +- **Safe shutdown mechanism** with async-signal-safe shutdown() method +- **Connection ownership management** with automatic cleanup on server destruction +- **Pluggable protocol handlers** via ConnectionHandler interface + +Key features: +- Multi-threaded architecture: separate accept and network thread pools +- EPOLL_EXCLUSIVE load balancing across accept threads +- Connection lifecycle safety with weak_ptr references +- Graceful shutdown with proper resource cleanup +- RAII-based connection management with unique_ptr ownership + +#### 8. **Connection** (`src/connection.{hpp,cpp}`) +- **Efficient per-connection state management** with arena-based memory allocation +- **Safe ownership transfer** between server threads and protocol handlers +- **Automatic cleanup** on connection closure or server shutdown +- **Handler interface isolation** - only exposes necessary methods to protocol handlers + +Key features: +- Arena allocator per connection for efficient memory management +- **Request/Response arena lifecycle**: Arena resets after each complete request/response cycle +- Weak reference to server for safe cleanup after server destruction +- Private networking details accessible only to Server via friend relationship +- Public handler interface: appendMessage(), closeAfterSend(), getArena(), getId() +- Thread-safe ownership transfer with Server::releaseBackToServer() + +#### 9. **ConnectionHandler Interface** (`src/connection_handler.hpp`) +- **Abstract protocol interface** decoupling networking from application logic +- **Ownership transfer support** allowing handlers to take connections for async processing +- **Streaming data processing** with partial message handling +- **Connection lifecycle hooks** for initialization and cleanup + +Key features: +- process_data() with unique_ptr& for ownership transfer +- ProcessResult enum for connection lifecycle control (Continue/CloseAfterSend/CloseNow) +- on_connection_established/closed() hooks for protocol state management +- Zero-copy data processing with arena allocator integration +- Thread-safe ownership transfer via Server::releaseBackToServer() + ### Data Model #### Transaction Structure @@ -118,10 +159,29 @@ CommitRequest { ``` #### Memory Management -- **Arena-based allocation** ensures efficient bulk memory management +- **Arena-based allocation** ensures efficient bulk memory management per connection - **String views** eliminate unnecessary copying of JSON data - **Zero-copy design** for binary data handling -- **Automatic memory cleanup** on transaction completion +- **RAII-based connection lifecycle** with automatic cleanup on destruction +- **Safe ownership transfer** between server threads and protocol handlers +- **Weak reference safety** prevents crashes when connections outlive server + +Connection Ownership Model: +1. **Creation**: Accept threads create connections, transfer to epoll as raw pointers +2. **Processing**: Network threads claim ownership by wrapping in unique_ptr +3. **Handler Transfer**: Handlers can take ownership for async processing via unique_ptr.release() +4. **Return Path**: Handlers use Server::releaseBackToServer() to return connections +5. **Safety**: All transfers use weak_ptr to server for safe cleanup +6. **Cleanup**: RAII ensures proper resource cleanup in all scenarios + +Arena Memory Lifecycle: +1. **Request Processing**: Handler uses `conn->getArena()` to allocate memory for parsing request data +2. **Response Generation**: Handler uses arena for temporary response construction (headers, JSON, etc.) +3. **Response Queuing**: Handler calls `conn->appendMessage()` which copies data to arena-backed message queue +4. **Response Writing**: Server writes all queued messages to socket via `writeBytes()` +5. **Arena Reset**: After successful write completion, arena resets to reclaim all memory from the request/response cycle + +This design assumes request/response pairs (HTTP-like protocols) but works for any protocol where there's a clear completion point for memory reclamation. ### API Design @@ -198,11 +258,15 @@ The modular design allows each component to be optimized independently while mai ## Development Guidelines ### Important Implementation Details +- **Server Creation**: Always use `Server::create()` factory method - direct construction is impossible +- **Connection Ownership**: Use unique_ptr semantics for safe ownership transfer between components - **Arena Allocator Pattern**: Always use `ArenaAllocator` for temporary allocations within request processing - **String View Usage**: Prefer `std::string_view` over `std::string` when pointing to arena-allocated memory +- **Ownership Transfer**: Use `Server::releaseBackToServer()` for returning connections to server from handlers - **JSON Token Lookup**: Use the gperf-generated perfect hash table in `json_tokens.hpp` for O(1) key recognition - **Base64 Handling**: Always use simdutf for base64 encoding/decoding for performance - **Error Propagation**: Use structured error types that can be efficiently returned up the call stack +- **Thread Safety**: Connection ownership transfers are designed to be thread-safe with proper RAII cleanup ### File Organization - **Core Headers**: `src/` contains all primary implementation files @@ -211,6 +275,16 @@ The modular design allows each component to be optimized independently while mai - **Tools**: `tools/` contains debugging and analysis utilities - **Build-Generated**: `build/` contains CMake-generated files including `json_tokens.cpp` +### Adding New Protocol Handlers +- Inherit from `ConnectionHandler` in `src/connection_handler.hpp` +- Implement `process_data()` with proper ownership semantics +- Use connection's arena allocator for temporary allocations: `conn->getArena()` +- Handle partial messages and streaming protocols appropriately +- Return appropriate `ProcessResult` for connection lifecycle management +- Use `Server::releaseBackToServer()` if taking ownership for async processing +- Add corresponding test cases and integration tests +- Consider performance implications of ownership transfers + ### Adding New Parsers - Inherit from `CommitRequestParser` in `src/commit_request_parser.hpp` - Implement both streaming and one-shot parsing modes @@ -244,9 +318,59 @@ The modular design allows each component to be optimized independently while mai ## Common Patterns +### Server Creation Pattern +```cpp +// Server must be created via factory method +auto server = Server::create(config, handler); + +// Never create on stack or with make_shared (won't compile): +// Server server(config, handler); // Compiler error - constructor private +// auto server = std::make_shared(config, handler); // Compiler error +``` + +### ConnectionHandler Implementation Patterns + +#### Simple Synchronous Handler +```cpp +class HttpHandler : public ConnectionHandler { +public: + ProcessResult process_data(std::string_view data, std::unique_ptr& conn_ptr) override { + // Parse HTTP request using connection's arena + ArenaAllocator& arena = conn_ptr->getArena(); + + // Generate response + conn_ptr->appendMessage("HTTP/1.1 200 OK\r\n\r\nHello World"); + + // Server retains ownership + return ProcessResult::CloseAfterSend; + } +}; +``` + +#### Async Handler with Ownership Transfer +```cpp +class AsyncHandler : public ConnectionHandler { +public: + ProcessResult process_data(std::string_view data, std::unique_ptr& conn_ptr) override { + // Take ownership for async processing + auto connection = std::move(conn_ptr); // conn_ptr is now null + + work_queue.push([connection = std::move(connection)](std::string_view data) mutable { + // Process asynchronously + connection->appendMessage("Async response"); + + // Return ownership to server when done + Server::releaseBackToServer(std::move(connection)); + }); + + return ProcessResult::Continue; // Server won't continue processing (conn_ptr is null) + } +}; +``` + ### Arena-Based String Handling ```cpp -// Preferred: Zero-copy string view +// Preferred: Zero-copy string view with arena allocation std::string_view process_json_key(const char* data, ArenaAllocator& arena); // Avoid: Unnecessary string copies @@ -267,3 +391,18 @@ CommitRequest request = CommitRequestBuilder(arena) .read_version(42) .build(); ``` + +### Safe Connection Ownership Transfer +```cpp +// In handler - take ownership for background processing +Connection* raw_conn = conn_ptr.release(); + +// Process on worker thread +background_processor.submit([raw_conn]() { + // Do work... + raw_conn->appendMessage("Background result"); + + // Return to server safely (handles server destruction) + Server::releaseBackToServer(std::unique_ptr(raw_conn)); +}); +``` diff --git a/src/connection.cpp b/src/connection.cpp index 292b25c..3267717 100644 --- a/src/connection.cpp +++ b/src/connection.cpp @@ -113,7 +113,11 @@ bool Connection::writeBytes() { } } assert(messages_.empty()); + + // Reset arena after completing request/response cycle + // This reclaims memory from request parsing and response generation arena_.reset(); + return closeConnection_; } diff --git a/src/connection_handler.hpp b/src/connection_handler.hpp index e8b2d0c..131112b 100644 --- a/src/connection_handler.hpp +++ b/src/connection_handler.hpp @@ -40,8 +40,8 @@ public: * - Return appropriate ProcessResult for connection management * - Handle partial messages and streaming protocols appropriately * - Can take ownership by calling conn_ptr.release() to pass to other threads - * - If ownership is taken, handler must call conn->releaseBackToServer() when - * done + * - If ownership is taken, handler must call Server::releaseBackToServer() + * when done */ virtual ProcessResult process_data(std::string_view data, std::unique_ptr &conn_ptr) = 0;