Add initial http server design

http.md

@@ -0,0 +1,154 @@
+# HTTP Server Design
+
+## Overview
+
+High-performance HTTP server implementation for WeaselDB using epoll-based event loop architecture with configurable threading model and lock-free communication between components.
+
+## Architecture
+
+### Threading Model
+
+#### Accept Threads
+- **Configurable number** of dedicated accept threads
+- Each thread runs **synchronous accept() loop** on listening socket
+- Accept returns file descriptor used to construct connection object
+- Connection posted to epoll with **EPOLLIN | EPOLLONESHOT** interest
+
+#### Network Threads
+- **Configurable number** of network I/O threads
+- Each thread calls **epoll_wait()** to receive connection ownership
+- **EPOLLONESHOT** ensures clean ownership transfer without thundering herd
+- Handle connection state transitions and HTTP protocol processing
+
+#### Service Threads
+- **Configurable service pipelines** (e.g., commit and status can share pipeline)
+- **Dequeue connections** from shared LMAX Disruptor inspired ring buffer
+- Process business logic and generate responses
+- **Post completed connections** back to epoll for response writing
+
+### Connection Lifecycle
+
+1. **Accept**: Accept thread creates connection, posts to epoll with read interest
+2. **Read**: Network thread reads data, pumps llhttp streaming parser
+3. **Route**: Complete request path triggers handler association
+4. **Process**: Handler either:
+   - Processes in network thread (fast path)
+   - Posts to service ring buffer (slow path)
+5. **Respond**: Connection posted to epoll with write interest
+6. **Write**: Network thread writes response data
+7. **Keep-alive**: Connection posted back with read interest for next request
+
+### HTTP Protocol Support
+
+#### Request Processing
+- **llhttp streaming parser** for incremental request parsing
+- **Content-Type based routing** to appropriate service handlers
+- **HTTP/1.1 keep-alive support** for connection reuse
+- **Malformed request handling** with proper HTTP error responses
+
+#### Response Handling
+- **Direct response writing** by network threads
+- **Partial write support** with epoll re-registration
+- **Connection reuse** after complete response transmission
+
+### Memory Management
+
+#### Arena Allocation
+- **Per-connection arena allocator** for request-scoped memory
+- **Arena reset after each response** to prepare for next request
+- **Zero-copy string views** pointing to arena-allocated memory
+- **Efficient bulk deallocation** when connection closes
+
+#### Ring Buffer Communication
+- **Ring buffer** for high-performance inter-thread communication
+- **Shared ring buffers** between related services (e.g., commit and status share pipeline)
+- **Backpressure handling** via ring buffer capacity limits
+
+### Error Handling
+
+#### Backpressure Response
+- **Ring buffer full**: Network thread writes immediate error response
+- **Partial write**: Connection re-posted to epoll with write interest
+- **Graceful degradation** under high load conditions
+
+#### Protocol Errors
+- **Malformed HTTP**: Proper HTTP error responses (400 Bad Request, etc.)
+- **Partial I/O**: Re-registration with epoll for completion
+- **Client disconnection**: Clean connection state cleanup
+
+## Integration Points
+
+### Parser Selection
+- **Content-Type header inspection** for parser routing
+- **ParserInterface implementation** for format-agnostic parsing
+- **CommitRequestParser extensibility** for future format support (protobuf, msgpack)
+
+### Configuration System
+- **TOML configuration** using existing config system
+- **Configurable parameters**:
+  - Accept thread count
+  - Network thread count
+  - Ring buffer sizes
+  - Connection timeouts
+  - Keep-alive settings
+
+### Request Routing
+- **Path-based routing** to service handlers
+- **Shared ring buffers** for related services with flexible pipeline configuration
+- **Handler registration** for extensible endpoint support
+
+## Performance Characteristics
+
+### Scalability
+- **Configurable parallelism** for accept and network operations
+- **Lock-free communication** eliminates contention bottlenecks
+- **EPOLLONESHOT semantics** prevent thundering herd effects and manages connection ownership
+
+### Efficiency
+- **Zero-copy I/O** where possible using string views
+- **Arena allocation** eliminates per-request malloc overhead
+- **Connection reuse** reduces accept/close syscall overhead
+- **Streaming parser** handles large requests incrementally
+
+### Resource Management
+- **Bounded memory usage** via arena reset and ring buffer limits
+- **Graceful backpressure** prevents resource exhaustion
+- **Clean connection lifecycle** with deterministic cleanup
+
+## Implementation Phases
+
+### Phase 1: Core Infrastructure
+- Basic epoll event loop with accept/network threads
+- Connection state management and lifecycle
+- llhttp integration for request parsing
+
+### Phase 2: Service Integration
+- Ring buffer implementation for service communication
+- Request routing and handler registration
+- Arena-based memory management
+
+### Phase 3: Protocol Features
+- HTTP/1.1 keep-alive support
+- Error handling and backpressure responses
+- Configuration system integration
+
+### Phase 4: Content Handling
+- Content-Type based parser selection
+- CommitRequest processing integration
+- Response serialization and transmission
+
+## Configuration Schema
+
+```toml
+[http]
+accept_threads = 2
+network_threads = 8
+bind_address = "0.0.0.0"
+port = 8080
+keepalive_timeout_sec = 30
+request_timeout_sec = 10
+
+[http.pipelines.commit_status]
+ring_buffer_size = 1024
+threads = 4
+```