Make pipeline policy/topology configurable

config.md

@@ -28,13 +28,15 @@ Controls server networking, threading, and request handling behavior.
 
 ### Commit Configuration (`[commit]`)
 
-Controls behavior of the `/v1/commit` endpoint and request ID management.
+Controls behavior of the `/v1/commit` endpoint, request ID management, and commit pipeline threading.
 
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
 | `min_request_id_length` | integer | `20` | Minimum length required for client-provided `request_id` fields to ensure sufficient entropy for collision avoidance |
 | `request_id_retention_hours` | integer | `24` | How long to retain request IDs in memory for `/v1/status` queries. Longer retention reduces the chance of `log_truncated` responses |
 | `request_id_retention_versions` | integer | `100000000` | Minimum number of versions to retain request IDs for, regardless of time. Provides additional protection against `log_truncated` responses |
+| `pipeline_wait_strategy` | string | `"WaitIfUpstreamIdle"` | Wait strategy for the commit pipeline. `"WaitIfStageEmpty"` = block when individual stages are empty (safe for shared CPUs), `"WaitIfUpstreamIdle"` = block only when all upstream stages are idle (requires dedicated cores, highest throughput), `"Never"` = never block, busy-wait continuously (requires dedicated cores, lowest latency) |
+| `pipeline_release_threads` | integer | `1` | Number of threads in the release stage (final stage of commit pipeline). Higher values increase parallelism for connection release and response transmission |
 
 ### Subscription Configuration (`[subscription]`)
 
@@ -77,6 +79,8 @@ read_buffer_size = 32768  # 32KB
 min_request_id_length = 32
 request_id_retention_hours = 48
 request_id_retention_versions = 50000
+pipeline_wait_strategy = "WaitIfUpstreamIdle"  # Options: "WaitIfStageEmpty", "WaitIfUpstreamIdle", "Never"
+pipeline_release_threads = 4  # Default: 1, increase for higher throughput
 
 [subscription]
 max_buffer_size_bytes = 52428800  # 50MB
@@ -115,6 +119,11 @@ These configuration parameters directly affect server and API behavior:
 
 - **`request_id_retention_*`**: Affects availability of data for `/v1/status` queries and likelihood of `log_truncated` responses
 
+**Commit Pipeline Performance:**
+
+- **`pipeline_wait_strategy`**: Controls CPU usage vs latency tradeoff in commit processing. `WaitIfStageEmpty` is safest for shared CPUs, `WaitIfUpstreamIdle` provides highest throughput with dedicated cores, `Never` provides lowest latency but uses 100% CPU
+- **`pipeline_release_threads`**: Determines parallelism in the final stage of commit processing. More threads can improve throughput when processing many concurrent requests
+
 **Subscription Streaming:**
 
 - **`max_buffer_size_bytes`**: Controls when `/v1/subscribe` connections are terminated due to slow consumption
@@ -137,6 +146,8 @@ The configuration system includes comprehensive validation with specific bounds
 - **`min_request_id_length`**: Must be between 8 and 256 characters
 - **`request_id_retention_hours`**: Must be between 1 and 8760 hours (1 year)
 - **`request_id_retention_versions`**: Must be > 0
+- **`pipeline_wait_strategy`**: Must be one of: `"WaitIfStageEmpty"`, `"WaitIfUpstreamIdle"`, or `"Never"`
+- **`pipeline_release_threads`**: Must be between 1 and 64
 
 ### Subscription Configuration Limits