Update design.md

2025-08-19 13:33:31 -04:00
parent cb322bbb2b
commit 5bfa20643a
3 changed files with 148 additions and 5 deletions
--- a/design.md
+++ b/design.md
@@ -99,6 +99,47 @@ See `config.md` for complete configuration documentation.
 - **Minimal perfect hash** reduces memory overhead and improves cache locality
 - **Build-time code generation** ensures optimal performance

+#### 7. **Server** (`src/server.{hpp,cpp}`)
+- **High-performance multi-threaded networking** using epoll with thread pools
+- **Factory pattern construction** via `Server::create()` ensures proper shared_ptr semantics
+- **Safe shutdown mechanism** with async-signal-safe shutdown() method
+- **Connection ownership management** with automatic cleanup on server destruction
+- **Pluggable protocol handlers** via ConnectionHandler interface
+
+Key features:
+- Multi-threaded architecture: separate accept and network thread pools
+- EPOLL_EXCLUSIVE load balancing across accept threads
+- Connection lifecycle safety with weak_ptr references
+- Graceful shutdown with proper resource cleanup
+- RAII-based connection management with unique_ptr ownership
+
+#### 8. **Connection** (`src/connection.{hpp,cpp}`)
+- **Efficient per-connection state management** with arena-based memory allocation
+- **Safe ownership transfer** between server threads and protocol handlers
+- **Automatic cleanup** on connection closure or server shutdown
+- **Handler interface isolation** - only exposes necessary methods to protocol handlers
+
+Key features:
+- Arena allocator per connection for efficient memory management
+- **Request/Response arena lifecycle**: Arena resets after each complete request/response cycle
+- Weak reference to server for safe cleanup after server destruction
+- Private networking details accessible only to Server via friend relationship
+- Public handler interface: appendMessage(), closeAfterSend(), getArena(), getId()
+- Thread-safe ownership transfer with Server::releaseBackToServer()
+
+#### 9. **ConnectionHandler Interface** (`src/connection_handler.hpp`)
+- **Abstract protocol interface** decoupling networking from application logic
+- **Ownership transfer support** allowing handlers to take connections for async processing
+- **Streaming data processing** with partial message handling
+- **Connection lifecycle hooks** for initialization and cleanup
+
+Key features:
+- process_data() with unique_ptr<Connection>& for ownership transfer
+- ProcessResult enum for connection lifecycle control (Continue/CloseAfterSend/CloseNow)
+- on_connection_established/closed() hooks for protocol state management
+- Zero-copy data processing with arena allocator integration
+- Thread-safe ownership transfer via Server::releaseBackToServer()
+
 ### Data Model

 #### Transaction Structure
@@ -118,10 +159,29 @@ CommitRequest {
 ```

 #### Memory Management
- **Arena-based allocation** ensures efficient bulk memory management
+- **Arena-based allocation** ensures efficient bulk memory management per connection
 - **String views** eliminate unnecessary copying of JSON data
 - **Zero-copy design** for binary data handling
- **Automatic memory cleanup** on transaction completion
+- **RAII-based connection lifecycle** with automatic cleanup on destruction
+- **Safe ownership transfer** between server threads and protocol handlers
+- **Weak reference safety** prevents crashes when connections outlive server
+
+Connection Ownership Model:
+1. **Creation**: Accept threads create connections, transfer to epoll as raw pointers
+2. **Processing**: Network threads claim ownership by wrapping in unique_ptr
+3. **Handler Transfer**: Handlers can take ownership for async processing via unique_ptr.release()
+4. **Return Path**: Handlers use Server::releaseBackToServer() to return connections
+5. **Safety**: All transfers use weak_ptr to server for safe cleanup
+6. **Cleanup**: RAII ensures proper resource cleanup in all scenarios
+
+Arena Memory Lifecycle:
+1. **Request Processing**: Handler uses `conn->getArena()` to allocate memory for parsing request data
+2. **Response Generation**: Handler uses arena for temporary response construction (headers, JSON, etc.)
+3. **Response Queuing**: Handler calls `conn->appendMessage()` which copies data to arena-backed message queue
+4. **Response Writing**: Server writes all queued messages to socket via `writeBytes()`
+5. **Arena Reset**: After successful write completion, arena resets to reclaim all memory from the request/response cycle
+
+This design assumes request/response pairs (HTTP-like protocols) but works for any protocol where there's a clear completion point for memory reclamation.

 ### API Design

@@ -198,11 +258,15 @@ The modular design allows each component to be optimized independently while mai
 ## Development Guidelines

 ### Important Implementation Details
+- **Server Creation**: Always use `Server::create()` factory method - direct construction is impossible
+- **Connection Ownership**: Use unique_ptr semantics for safe ownership transfer between components
 - **Arena Allocator Pattern**: Always use `ArenaAllocator` for temporary allocations within request processing
 - **String View Usage**: Prefer `std::string_view` over `std::string` when pointing to arena-allocated memory
+- **Ownership Transfer**: Use `Server::releaseBackToServer()` for returning connections to server from handlers
 - **JSON Token Lookup**: Use the gperf-generated perfect hash table in `json_tokens.hpp` for O(1) key recognition
 - **Base64 Handling**: Always use simdutf for base64 encoding/decoding for performance
 - **Error Propagation**: Use structured error types that can be efficiently returned up the call stack
+- **Thread Safety**: Connection ownership transfers are designed to be thread-safe with proper RAII cleanup

 ### File Organization
 - **Core Headers**: `src/` contains all primary implementation files
@@ -211,6 +275,16 @@ The modular design allows each component to be optimized independently while mai
 - **Tools**: `tools/` contains debugging and analysis utilities
 - **Build-Generated**: `build/` contains CMake-generated files including `json_tokens.cpp`

+### Adding New Protocol Handlers
+- Inherit from `ConnectionHandler` in `src/connection_handler.hpp`
+- Implement `process_data()` with proper ownership semantics
+- Use connection's arena allocator for temporary allocations: `conn->getArena()`
+- Handle partial messages and streaming protocols appropriately
+- Return appropriate `ProcessResult` for connection lifecycle management
+- Use `Server::releaseBackToServer()` if taking ownership for async processing
+- Add corresponding test cases and integration tests
+- Consider performance implications of ownership transfers
+
 ### Adding New Parsers
 - Inherit from `CommitRequestParser` in `src/commit_request_parser.hpp`
 - Implement both streaming and one-shot parsing modes
@@ -244,9 +318,59 @@ The modular design allows each component to be optimized independently while mai

 ## Common Patterns

+### Server Creation Pattern
+```cpp
+// Server must be created via factory method
+auto server = Server::create(config, handler);
+
+// Never create on stack or with make_shared (won't compile):
+// Server server(config, handler);  // Compiler error - constructor private
+// auto server = std::make_shared<Server>(config, handler);  // Compiler error
+```
+
+### ConnectionHandler Implementation Patterns
+
+#### Simple Synchronous Handler
+```cpp
+class HttpHandler : public ConnectionHandler {
+public:
+  ProcessResult process_data(std::string_view data, std::unique_ptr<Connection>& conn_ptr) override {
+    // Parse HTTP request using connection's arena
+    ArenaAllocator& arena = conn_ptr->getArena();
+
+    // Generate response
+    conn_ptr->appendMessage("HTTP/1.1 200 OK\r\n\r\nHello World");
+
+    // Server retains ownership
+    return ProcessResult::CloseAfterSend;
+  }
+};
+```
+
+#### Async Handler with Ownership Transfer
+```cpp
+class AsyncHandler : public ConnectionHandler {
+public:
+  ProcessResult process_data(std::string_view data, std::unique_ptr<Connection>& conn_ptr) override {
+    // Take ownership for async processing
+    auto connection = std::move(conn_ptr); // conn_ptr is now null
+
+    work_queue.push([connection = std::move(connection)](std::string_view data) mutable {
+      // Process asynchronously
+      connection->appendMessage("Async response");
+
+      // Return ownership to server when done
+      Server::releaseBackToServer(std::move(connection));
+    });
+
+    return ProcessResult::Continue; // Server won't continue processing (conn_ptr is null)
+  }
+};
+```
+
 ### Arena-Based String Handling
 ```cpp
-// Preferred: Zero-copy string view
+// Preferred: Zero-copy string view with arena allocation
 std::string_view process_json_key(const char* data, ArenaAllocator& arena);

 // Avoid: Unnecessary string copies
@@ -267,3 +391,18 @@ CommitRequest request = CommitRequestBuilder(arena)
    .read_version(42)
    .build();
 ```
+
+### Safe Connection Ownership Transfer
+```cpp
+// In handler - take ownership for background processing
+Connection* raw_conn = conn_ptr.release();
+
+// Process on worker thread
+background_processor.submit([raw_conn]() {
+  // Do work...
+  raw_conn->appendMessage("Background result");
+
+  // Return to server safely (handles server destruction)
+  Server::releaseBackToServer(std::unique_ptr<Connection>(raw_conn));
+});
+```