weaselab/weaseldb
2
Fork 1
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
400 Commits 1 Branch 0 Tags
main
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
andrew 3b7cc2a70f Don't count_down past 0
CI / pre-commit (push) Successful in 59s
Details
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64) (push) Successful in 2m22s
Details
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64) (push) Successful in 2m57s
Details
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64) (push) Successful in 2m44s
Details
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64) (push) Successful in 3m10s
Details
2026-07-17 21:15:23 -04:00
andrew a427278cc0 Read until EAGAIN 
To prepare for EPOLLET
 2026-07-17 15:39:56 -04:00
andrew 8056da856f Replace protocol_context void* with ProtocolHandle int64_t
CI / pre-commit (push) Successful in 58s
Details
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-arm64, ubuntu-latest-arm64) (push) Successful in 2m23s
Details
CI / build (-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, clang-amd64, ubuntu-latest-amd64) (push) Successful in 2m59s
Details
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-arm64, ubuntu-latest-arm64) (push) Successful in 2m44s
Details
CI / build (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, gcc-amd64, ubuntu-latest-amd64) (push) Successful in 3m12s
Details 
Use an opaque int64_t handle for correlating async responses instead of raw pointers. HTTP uses sequence_id as the handle and stores response data in a unique_ptr<HttpResponseContext> keyed by sequence_id.
 2026-07-17 15:11:02 -04:00
andrew 0c0f154390 Rename append_message to send_response and update Connection construction docs 2026-07-17 15:11:02 -04:00
andrew addef07866 Prepare for more per io thread/epoll instance state 2026-07-17 15:11:02 -04:00
andrew 36a50dfdde Remove dead code 2026-07-17 15:11:02 -04:00
andrew 6219592620 Listen on public interface for test benchmark 2026-06-26 19:03:15 -04:00
andrew edfa71ce7c Add script to reproduce threading performance report results 
Adds `reproduce_threading_report.sh` to run the WeaselDB server and
load tester configuration described in `threading_performance_report.md`.

Improvements over the original draft:
- Validates the build directory, config file, and required binaries
  before starting, and expects to be run from the project root.
- Wraps server shutdown and log printing in an EXIT/INT/TERM trap
  so the server is always cleaned up, even if the script is
  interrupted or the load tester fails.
- Makes the `ulimit -n` increase best-effort instead of fatal.
- Uses the existing `DURATION` variable consistently in the load
  tester invocation.
- Adds a final check that the unix socket was created before
  launching the load tester.
- Uses 2 connect threads on the client, which is sufficient for
  establishing 2000 connections over the 30-second run.
 2026-06-26 14:28:05 -04:00
andrew 5790603e31 Update threading performance report with reproduced numbers 
The previous report claimed 1.0M req/s at 740ns serial CPU work for the /ok health check endpoint. That measurement was made with an earlier design that transferred unique ownership of connections through the pipeline.

The current server-owned connection model adds per-request synchronization overhead (mutex + WeakRef + pending response queue) that lowers the raw /ok throughput. Reproducing on an AMD Ryzen 9 7900 with the current Release build gives approximately 825k sustained req/s with the same 740ns serial CPU work.

Updated the report to reflect the reproduced numbers and added a note explaining the historical context and why the ownership model changed (to support streaming endpoints like /v1/subscribe and safer async responses).
 2026-06-26 13:16:48 -04:00
andrew 273f288020 Merge pull request 'Support building on ARM with NEON histogram intrinsics' (#4) from weaselbot/weaseldb:weaselbot/issue-3 into main 
Reviewed-on: #4
Reviewed-by: andrew <andrew@weaselab.dev>
 2026-06-26 16:13:00 +00:00
andrew f458c6b249 Make pipeline policy/topology configurable 2025-11-06 15:55:27 -05:00
andrew 9f8562e30f Fix histogram thread death bug 2025-09-18 12:39:47 -04:00
andrew cb66d65479 Add a bit more precision to docs, plus philosophy 2025-09-16 00:57:20 -04:00
andrew 4d015fa3dc Make recording metrics never block 2025-09-15 23:34:30 -04:00
andrew 0659319906 Prepare for try_lock optimization 
So histogram observations never block
 2025-09-15 23:04:54 -04:00
andrew 6f421629aa Add comments to avoid blocking in sequence and resolve stages 2025-09-15 22:51:41 -04:00
andrew ba59a992dd Document per-connection locking strategy 2025-09-15 22:33:52 -04:00
andrew 0d76c73077 Only shut down write side for http 2025-09-15 21:42:36 -04:00
andrew 4ecbc07367 Fix flaky connection shutdown test 2025-09-15 21:39:48 -04:00
andrew 5e625197aa Fix url accumulation bug 2025-09-15 21:36:49 -04:00
andrew 5dda7353fa Add test for url accumulation bug 2025-09-15 21:36:49 -04:00
andrew 345d8e21b2 Use WaitIfUpstreamIdle 2025-09-15 20:48:12 -04:00
andrew 917066d8c0 Move thread local state into stack 2025-09-15 20:33:45 -04:00
andrew 5a88047b9f Two release threads 2025-09-15 20:09:15 -04:00
andrew 1acdc1e753 Put Arena destructor and move constructors in header 2025-09-15 17:01:38 -04:00
andrew ae0c014298 Don't clear write interest if pending_response_queue_ non empty 2025-09-15 16:35:37 -04:00
andrew e2115152c8 Add tests for shutdown vs close 2025-09-15 15:48:10 -04:00
andrew 55f6ebc02b Implement shutting down the write-side only 2025-09-15 15:39:28 -04:00
andrew 6b52c4289c Prevent queueing of messages on connection after it will be closed 2025-09-15 15:25:40 -04:00
andrew 7ee5ca2a9b Remove dead code, use proper send_ordered_response 
And prepare to try to close a connection gracefully
 2025-09-15 15:08:19 -04:00
andrew 9120c05847 Use spend_cpu_cycles instead of volatile loop 2025-09-15 14:07:38 -04:00
andrew 528a467518 Make test names match binary names 2025-09-15 13:06:15 -04:00
andrew a67d7a8531 Consolidate into one send_ordered_response 2025-09-15 13:01:56 -04:00
andrew 6717b70772 Remove HttpConnectionState::response_queue_mutex 2025-09-15 12:32:19 -04:00
andrew 34accb9d80 Move GetVersion to commit pipeline 2025-09-15 12:30:02 -04:00
andrew f3c3f77a24 Extract commit pipeline to its own module 2025-09-15 11:51:01 -04:00
andrew afd240dba7 Remove vestigial "round-robin" code 2025-09-15 11:22:14 -04:00
andrew 1cb7a4c301 Remove has_pending_responses_ 2025-09-15 11:10:47 -04:00
andrew 1b220d0d1c WIP 2025-09-15 10:28:17 -04:00
andrew ec2ad27e33 Add explanatory comments 2025-09-15 00:07:44 -04:00
andrew eb98e51867 We expect to get valid fds to close in ~Connection in ~Server 2025-09-15 00:07:09 -04:00
andrew 022a79bf5b Separate HttpRequestState and HttpConnectionState 
Now HttpConnectionState has a queue of HttpRequestState
 2025-09-14 23:49:32 -04:00
andrew fac6b8de88 Add test that shows parsing issue 
It's meant to show the pipelining issue. I guess we'll solve the
newly-discovered parsing issue first.
 2025-09-14 22:24:02 -04:00
andrew 1f61f91bf5 Reset connection state after finishing with it in http_handler 2025-09-14 21:16:41 -04:00
andrew 632113f792 Add test for pipeline request parsing bug 2025-09-14 20:53:19 -04:00
andrew f62770c4ab Add copying utility methods to Arena 2025-09-14 20:38:54 -04:00
andrew 147edf5c93 More cleanup 2025-09-14 20:27:14 -04:00
andrew f39149d516 Update documentation with new networking model 2025-09-14 19:03:56 -04:00
andrew 0389fd2c9f Consistently use state->arena for http handling 2025-09-14 17:16:05 -04:00
andrew 7ef54a2d08 Call epoll_ctl in release stage 2025-09-14 16:28:12 -04:00
andrew 16c7ee0408 Separate Connection and Request lifetimes 2025-09-14 15:04:37 -04:00
andrew cf0c1b7cc2 Add echo test for server 2025-09-14 12:56:22 -04:00
andrew bd06798fd3 Remove test_http_handler and test_server_connection_return 2025-09-14 11:38:43 -04:00
andrew e96a493835 Remove release_back_to_server 2025-09-14 09:03:05 -04:00
andrew e887906da8 Remove some unused/indirectly used headers 2025-09-13 17:25:46 -04:00
andrew de6f38694f std::unique_ptr<Connection> -> Ref<Connection> 2025-09-13 17:25:46 -04:00
andrew 1fa3381e4b Use send/sendmsg and don't ignore SIGPIPE 2025-09-13 17:25:20 -04:00
andrew cd2e15677a Remove epoll instances config 2025-09-12 18:05:07 -04:00
andrew 2b8f095d27 Fix minor issues 2025-09-12 12:13:50 -04:00
andrew 543447971f Fix polymorphic WeakRef bug 2025-09-12 12:08:46 -04:00
andrew f89868058a Require explicit copies for Ref/WeakRef 2025-09-12 11:59:56 -04:00
andrew 674ff581e7 Update comments/docs to match code 2025-09-12 11:40:38 -04:00
andrew be5a0c6d8e Update some inaccuracies in markdown files 2025-09-12 11:31:22 -04:00
andrew bf90b8856a Add mdformat pre-commit hook 2025-09-12 11:24:16 -04:00
andrew 9d48caca76 add end-of-file-fixer 2025-09-12 11:21:00 -04:00
andrew 0561d951d4 Finish std::shared_ptr -> Ref migration 2025-09-11 15:06:04 -04:00
andrew a2da7fba84 Explicitly support having a WeakRef to self 2025-09-11 14:54:42 -04:00
andrew 5d932bf36c Add polymorphism support to Ref 2025-09-11 14:15:52 -04:00
andrew 9a8d4feedd Add documentation 2025-09-11 13:54:00 -04:00
andrew 9cd83fc426 Call ~ControlBlock 
It's trivially destructible, but just in case. Compiler should optimize it out
 2025-09-11 13:18:19 -04:00
andrew 10e382f633 Used biased weak count, cache T* pointer 
Logically, the strong pointer that destroys T owns +1 weak count too
 2025-09-11 13:15:03 -04:00
andrew f83e21b5a0 Defeat shared_ptr's single-threaded optimizations 
WeaselDB is always going to start multiple threads, so we don't care
about single-threaded performance
 2025-09-11 13:13:05 -04:00
andrew 5adbf8eee2 Organize bench_reference.cpp with doctest 2025-09-11 12:32:25 -04:00
andrew 2bc17cbfe6 Add bench_reference.cpp 
Also update snake case script for nanobench symbols
 2025-09-11 12:22:56 -04:00
andrew 89c5a2f165 Strengthen language instructing reading the style guide 2025-09-11 12:02:44 -04:00
andrew d35a4fa4db Update multi-threaded tests/benchmarks guidance 2025-09-11 12:01:18 -04:00
andrew 994e31032f Fix data race in freeing control block 2025-09-11 11:32:59 -04:00
andrew 0f179eed88 Switch to two separate atomic counters 
It's faster and still correct. I was confused remembering something
about atomic shared pointer ideas before.
 2025-09-11 10:53:25 -04:00
andrew b9106a0d3c Add test_reference.cpp 2025-09-10 22:05:31 -04:00
andrew 6aaca4c171 Finish reference.hpp 2025-09-10 21:58:08 -04:00
andrew 7c4d928807 Start on Ref/WeakRef 2025-09-10 20:04:32 -04:00
andrew 5d289ddd42 Add metric for write EAGAIN failures 2025-09-10 16:48:27 -04:00
andrew 962a010724 Simplify process_connection_writes condition 
And comment explaining that we there's something more precise but more
complex available.
 2025-09-10 16:45:04 -04:00
andrew f56ed2bfbe Rename ArenaAllocator -> Arena 2025-09-05 17:57:04 -04:00
andrew 46fe51c0bb Make config.toml comments more descriptive and accessible 2025-09-05 16:36:50 -04:00
andrew b93cc2072a Remove -Wno-vla-cxx-extension from .clangd 2025-09-05 16:29:12 -04:00
andrew 0357a41dd8 Implement spend_cpu_cycles in assembly 
The compiler was unrolling it previously, so we're doing assembly now for consistency.
 2025-09-05 15:16:49 -04:00
andrew ffe7ab0a3e Update default in config.toml 2025-09-05 13:06:10 -04:00
andrew ed3cf25936 Update stale documentation 2025-09-05 13:04:34 -04:00
andrew e67e4aee17 Update /ok to serve dual health check/benchmarking role 2025-09-05 12:39:10 -04:00
andrew 761eaa552b Add -Wno-deprecated-literal-operator for clang 2025-09-05 11:39:04 -04:00
andrew e846bc49f6 Set rapidjson docs + examples to off 2025-09-05 11:29:08 -04:00
andrew 72481be46d Consolidate into two static libs - one with assertions and one without 2025-09-05 11:22:04 -04:00
andrew d04705624a Handle percent encoding 2025-09-04 20:47:58 -04:00
andrew 2278694f4f Separate out api url parser 2025-09-04 16:39:19 -04:00
andrew 55069c0c79 Add counters for /v1/{commit,status,version} 2025-09-04 15:49:54 -04:00
andrew 96aae52853 Basic implementation of /commit, /version, and /status 
No precondition checking, persistence, or log scanning yet.
 2025-09-04 15:40:17 -04:00
andrew 8b6736127a Add commit pipeline design 2025-09-04 13:40:03 -04:00
andrew 9272048108 Outline commit pipeline 2025-09-03 23:43:03 -04:00
andrew b2ffe3bfab Refactor to use format for http responses 2025-09-03 22:45:59 -04:00