weaselab/weaseldb
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve precision

Browse Source
This commit is contained in:
Andrew Noyes
2025-08-25 06:49:26 -04:00
parent ce794f8a0f
commit c83fda5877
1 changed files with 13 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
persistence.md
View File

@@ -102,7 +102,7 @@ The persistence thread collects commits into batches using two trigger condition
**Required Constraints**:
- `batch_size_threshold` > 0 (must process at least one commit per batch)
- `max_in_flight_requests` > 0 (must allow at least one concurrent request)
- `max_in_flight_requests` < 1000 (required for single-call recovery guarantee)
- `max_in_flight_requests` <= 1000 (required for single-call recovery guarantee)
- `target_pool_size` >= `max_in_flight_requests` (pool must accommodate all in-flight requests)
- `batch_timeout_ms` > 0 (timeout must be positive)
- `max_retry_attempts` >= 0 (zero disables retries)
@@ -119,9 +119,9 @@ WeaselDB's batched persistence design enables efficient recovery while maintaini
#### **Batch Ordering and Durability**
**Ordered Acknowledgment Property**: Batches may be retried out-of-order for performance, but acknowledgment to the next pipeline stage maintains strict ordering. This ensures that if batch N is acknowledged as durable, all batches 0 through N-1 are also guaranteed durable.
**Ordered Acknowledgment Property**: Batches may be retried out-of-order for performance, but acknowledgment to the next pipeline stage maintains strict ordering. This ensures that if batch N is acknowledged as durable, all earlier batches (higher numbers N+1, N+2, etc.) are also guaranteed durable.
**Durability Watermark**: The system maintains a durable watermark indicating the highest consecutively durable batch ID. This watermark advances only when all preceding batches are confirmed persistent.
**Durability Watermark**: The system maintains a durable watermark indicating the latest consecutively durable batch (lowest batch number in the consecutive sequence). This watermark advances only when all preceding batches (higher numbers) are confirmed persistent.
#### **Recovery Protocol**
@@ -147,10 +147,16 @@ WeaselDB uses a **sequential batch numbering** scheme with **S3 atomic operation
**Crash Recovery**:
1. **S3 Scan with Bounded Cost**: List S3 objects with prefix `batches/` and limit of 1000 objects
2. **Gap Detection**: Check for missing sequential batch numbers. WeaselDB never puts 1000 batches in flight concurrently, so a limit of 1000 is sufficient.
3. **Watermark Reconstruction**: Set durability watermark to highest consecutive batch number found
2. **Gap Detection**: Check for missing sequential batch numbers. WeaselDB never puts more than 1000 batches in flight concurrently, so a limit of 1000 is sufficient.
3. **Watermark Reconstruction**: Set durability watermark to the latest consecutive batch (lowest batch number in consecutive sequence)
4. **Leadership Transition**: Begin writing batches starting from next available batch number. Skip past any batch numbers claimed in the durability watermark scan.
**Bounded Recovery Guarantee**: Since at most 999 batches can be in-flight during a crash, the durability watermark is guaranteed to be found within the first 1000 objects in S3. This ensures **O(1) recovery time** regardless of database size, with at most **one S3 LIST operation** required.
**Bounded Recovery Guarantee**: Since at most 1000 batches can be in-flight during a crash, the durability watermark is guaranteed to be found within the first 1000 objects in S3. This ensures **O(1) recovery time** regardless of database size, with at most **one S3 LIST operation** required.
**Recovery Performance Limits**: To maintain single-call recovery guarantees, `max_in_flight_requests` is limited to **1000**, matching S3's maximum objects per LIST operation. This ensures recovery a single S3 API call is sufficient for recovery.
**Recovery Protocol Detail**: Even with exactly 1000 batches in-flight, recovery works correctly:
- Worst case: earliest batch (highest number) fails while remaining 999 batches succeed
- S3 LIST returns 1000 objects: the 999 successful batches plus one previously written batch
- Gap detection identifies the missing batch and sets watermark to latest consecutive batch (lowest number in sequence)
- Since batches count downward with zero-padded names, lexicographic ordering ensures proper sequence detection
**Recovery Performance Limits**: To maintain single-call recovery guarantees, `max_in_flight_requests` is limited to **1000**, matching S3's maximum objects per LIST operation. This ensures a single S3 API call is sufficient for recovery.