WeaselJSON: A Streaming JSON Parser Review
What is WeaselJSON?
WeaselJSON is a high-performance, streaming JSON parser that uses callbacks instead of building an object tree in memory. It's optimized with SIMD instructions and designed for situations where you either can't or don't want to load entire JSON files into memory.
Key Characteristics
What's good:
- Uses constant memory regardless of input size
- Fast parsing with SIMD optimizations
- Follows JSON spec properly with good security practices
- You can't accidentally make it O(n²) (unlike the simdjson ondemand api)
- Enables weird use cases like parsing just the beginning of huge JSON files
What's not good:
- The callback API is a pain to use correctly
- Requires a lot of boilerplate for simple tasks
- Most people don't actually need streaming JSON parsing
- Some features only work if you control how the JSON is structured
JSON Parser Decision Guide
flowchart TD
A[Need to parse JSON?] --> B{Do you want to parse<br/>partial JSON documents?}
B -->|Yes| C[WeaselJSON<br/>Streaming parser<br/>Constant memory usage]
B -->|No| D{Is maximum performance<br/>critical?}
D -->|No| E[SimdJSON DOM<br/>Easy to use<br/>Good performance]
D -->|Yes| F[SimdJSON On-Demand<br/>Fastest option<br/>Forward-only traversal]
When to Use What
Use SimdJSON DOM API when:
- You want to write
obj["key"]and have it just work - JSON files are reasonably sized (but you still want decent performance)
- You care about getting things done
Use SimdJSON On-Demand when:
- Performance is critical and your data fits in memory
- You can work with forward-only traversal (no random access or backtracking)
- You need maximum speed but still want a usable API
- Your JSON structure is consistent and you control both generation and parsing
Use WeaselJSON when:
- You absolutely cannot load the whole JSON into memory
- You're processing huge JSON files (multiple gigabytes)
- You want to parse just part of a JSON file without reading the rest
- You need to convert JSON into some other format as you parse it
- You can't risk accidentally slow performance
- Writing stateful callbacks is your idea of fun
The Reality
WeaselJSON solves real problems that other parsers can't handle, but most people don't have those problems. The callback API is legitimately difficult to use correctly, which pushes most developers toward easier alternatives.
The parser represents excellent engineering work and occupies a useful niche. It's the kind of tool you're very glad exists when you actually need it, but most people will never need it.
Note: WeaselJSON assumes modern CPU features (SIMD), so you probably can't use it for embedded development.
Technical Reference
Features
- SAX-style callback API
- No memory allocations during parsing
- O(1) memory usage regardless of input size
- Streaming API - no need to buffer the entire document in memory. Parsing is resumed when more data is available
- Strings are unescaped in place before they're presented. No unicode normalization is performed
- Robust to crashes with untrusted input
- SIMD optimizations for string scanning and validation
RFC 8259 Conformance
- There are no limits on number precision. Numbers are only validated syntactically and are presented as is
- Only UTF-8 is accepted
- Invalid UTF-8 is rejected
- Byte order markers are rejected
- Invalid escaped UTF-16 surrogate pairs are rejected
- Documents that are too deeply nested are rejected to control memory usage
- Duplicate keys are presented
Caveats
- Users should be prepared to discard work done during SAX callbacks if the document is ultimately rejected
- Requires manual state management in callback functions
- API is more complex than DOM-style parsers
Maintainer Notice
⚠️ Important: This is a hobby project by a single maintainer. Please be aware that:
- I'm doing this for fun and learning, not as a professional obligation
- Don't rely on this for mission-critical applications without understanding these limitations
- Feel free to email me, but I may not respond promptly
Note: This document was written by AI (Claude) in collaboration with the author.