The string unescape logic treats any code unit in the surrogate range (0xD800-0xDFFF) as the start of a surrogate pair, and only verifies that the second code unit is a low surrogate (0xDC00-0xDFFF). This allows two consecutive low surrogates (e.g. \uDC00\uDC00) to be accepted and decoded as a valid code point, which violates RFC 8259 and the project's stated RFC 8259 conformance ("Invalid escaped UTF-16 surrogate pairs are rejected" in README.md).
Affected code:
src/parser3.h lines 628-636 (fast path in n_string2):
A lone low surrogate followed by another low surrogate, or a high surrogate followed by another high surrogate, should be rejected. Only a high surrogate (0xD800-0xDBFF) followed by a low surrogate (0xDC00-0xDFFF) is valid.
The string unescape logic treats any code unit in the surrogate range (0xD800-0xDFFF) as the start of a surrogate pair, and only verifies that the second code unit is a low surrogate (0xDC00-0xDFFF). This allows two consecutive low surrogates (e.g. `\uDC00\uDC00`) to be accepted and decoded as a valid code point, which violates RFC 8259 and the project's stated RFC 8259 conformance ("Invalid escaped UTF-16 surrogate pairs are rejected" in README.md).
Affected code:
- `src/parser3.h` lines 628-636 (fast path in `n_string2`):
```cpp
if (0xd800 <= codepoint && codepoint <= 0xdfff) {
int32_t codepoint2 = read4_hex(buf + 2);
if (!(buf[0] == '\\' && buf[1] == 'u' && 0xdc00 <= codepoint2 &&
codepoint2 <= 0xdfff)) [[unlikely]] {
return WeaselJson_REJECT;
}
codepoint =
0x10000 + (codepoint - 0xd800) * 0x400 + (codepoint2 - 0xdc00);
```
The first check should be `0xd800 <= codepoint && codepoint <= 0xdbff`.
- `src/parser3.h` lines 796-798 and 855-862 (slow path in `t_hex2` / `t_hex3`):
```cpp
if (0xd800 <= self->utf8Codepoint && self->utf8Codepoint <= 0xdfff) {
// utf-16 surrogate
self->utf16Surrogate = self->utf8Codepoint;
```
and later in `t_hex3`:
```cpp
if (!(0xdc00 <= self->utf8Codepoint && self->utf8Codepoint <= 0xdfff))
[[unlikely]] {
return WeaselJson_REJECT;
}
```
Again, `t_hex2` should only treat high surrogates (0xD800-0xDBFF) as the start of a pair.
Concrete example:
- Input: `"\uDC00\uDC00"`
- Expected: `WeaselJson_REJECT` (invalid surrogate pair).
- Actual: accepted and decoded as U+10000.
A lone low surrogate followed by another low surrogate, or a high surrogate followed by another high surrogate, should be rejected. Only a high surrogate (0xD800-0xDBFF) followed by a low surrogate (0xDC00-0xDFFF) is valid.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The string unescape logic treats any code unit in the surrogate range (0xD800-0xDFFF) as the start of a surrogate pair, and only verifies that the second code unit is a low surrogate (0xDC00-0xDFFF). This allows two consecutive low surrogates (e.g.
\uDC00\uDC00) to be accepted and decoded as a valid code point, which violates RFC 8259 and the project's stated RFC 8259 conformance ("Invalid escaped UTF-16 surrogate pairs are rejected" in README.md).Affected code:
src/parser3.hlines 628-636 (fast path inn_string2):The first check should be
0xd800 <= codepoint && codepoint <= 0xdbff.src/parser3.hlines 796-798 and 855-862 (slow path int_hex2/t_hex3):and later in
t_hex3:Again,
t_hex2should only treat high surrogates (0xD800-0xDBFF) as the start of a pair.Concrete example:
"\uDC00\uDC00"WeaselJson_REJECT(invalid surrogate pair).A lone low surrogate followed by another low surrogate, or a high surrogate followed by another high surrogate, should be rejected. Only a high surrogate (0xD800-0xDBFF) followed by a low surrogate (0xDC00-0xDFFF) is valid.
The counterexample given doesn't actually work (it's correctly rejected), but we'll make the suggested changes anyway for clarity