schemagen integer parser rejects valid zero values with negative exponents (e.g. 0e-2) #56

Closed
opened 2026-07-19 18:11:11 +00:00 by weaselbot · 0 comments
Member

Summary

The integer-parsing fallback in the generated RootBuilder (parseJsonInt64) incorrectly rejects valid JSON numbers whose mathematical value is 0 but which are written with a negative exponent large enough that trailing/leading-zero trimming does not fully offset it. The core weaseljson parser accepts these as valid JSON, and they are integers representable as int64_t (value 0), so the schemagen builder should accept them too.

Affected code

contrib/schemagen/weaseljson_schemagen.py, Builder._builder -> the parseJsonInt64 function emitted into the generated header (around lines 1055-1064 of the generator):

    int64_t trim = 0;
    while (!digits.empty() && digits.back() == '0') {
      digits.pop_back();
      ++trim;
    }
    int64_t finalExp = exp - fracDigits + trim;
    if (finalExp < 0) return false;            // <-- line 1061: rejects zero too early
    size_t leadingZeros = 0;
    while (leadingZeros < digits.size() && digits[leadingZeros] == '0') ++leadingZeros;
    if (leadingZeros == digits.size()) { out = 0; return true; }  // <-- line 1064: zero detection

The if (finalExp < 0) return false; guard runs before the zero-detection guard. When the value is 0, all significant digits are trimmed away (or are leading zeros), so digits is empty/all-zero, but finalExp can still be negative, causing the early return false before the out = 0 branch is reached.

Reproduction

Schema s.json:

{"type":"integer"}

Generate and run a tiny harness:

python3 contrib/schemagen/weaseljson_schemagen.py s.json -o gen.h --namespace sg
#include "gen.h"
#include <cstdio>
#include <cstring>
int main() {
  const char *cases[] = {"0", "0e-1", "0e-2", "0.0e-2", "-0e-2", "0e-20"};
  for (auto s : cases) {
    sg::RootBuilder b;
    char buf[64]; std::strncpy(buf, s, sizeof(buf)-1); buf[sizeof(buf)-1]=0;
    WeaselJsonStatus st = b.feed(buf, std::strlen(s));
    st = b.finish();
    std::printf("%s -> %d\n", s, st);
  }
}

Observed output:

0 -> 0        (OK)
0e-1 -> 0     (OK)
0e-2 -> 2     (REJECT)   <-- bug
0.0e-2 -> 2   (REJECT)   <-- bug
-0e-2 -> 2    (REJECT)   <-- bug
0e-20 -> 2    (REJECT)   <-- bug

The same rejection occurs for an integer object field, e.g. schema {"type":"object","required":["age"],"properties":{"age":{"type":"integer"}}} with input {"age":0e-2} -> WeaselJson_REJECT.

Expected vs actual

  • 0e-2, 0.0e-2, -0e-2, 0e-20 are valid JSON numbers (RFC 8259: 0 integer part, optional fraction, e-2 exponent). The core WeaselJsonParser_parse returns WeaselJson_OK for each of them (verified directly).
  • Their mathematical value is 0, which is an integer with no fractional part and is representable as int64_t.
  • Per the schemagen README, JSON Schema "integer" accepts any number with no fractional part, including exponent/decimal forms like 1e3 or 2.0, and only a number not representable in the target type (e.g. 1.5 for an integer) is rejected.
  • Therefore these inputs should be accepted with value 0. The generated builder instead returns WeaselJson_REJECT.

Note the boundary is the exponent magnitude vs. the number of trimmed zeros: 0e-1 is accepted (finalExp = -1 + 1 = 0), but 0e-2 is rejected (finalExp = -2 + 1 = -1 < 0). Non-zero values with negative exponents are correctly rejected (e.g. 1e-2 = 0.01), so the bug is specific to the zero case.

Impact

A schema-valid JSON document containing a perfectly ordinary integer value 0 written in exponent form (e.g. 0e-2, 0.0e-2, -0e-2) is silently rejected by the generated parser as a schema violation. Users feeding data produced by other JSON encoders that emit zero in scientific notation will see spurious WeaselJson_REJECT for valid input.

Suggested fix

Move the zero-detection check before the finalExp < 0 guard (or skip the negative-finalExp rejection when the remaining digits are all zero), e.g.:

    int64_t finalExp = exp - fracDigits + trim;
    size_t leadingZeros = 0;
    while (leadingZeros < digits.size() && digits[leadingZeros] == '0') ++leadingZeros;
    if (leadingZeros == digits.size()) { out = 0; return true; }  // zero is representable regardless of exponent
    if (finalExp < 0) return false;
    if (leadingZeros > 0) digits.erase(0, leadingZeros);
## Summary The integer-parsing fallback in the generated `RootBuilder` (`parseJsonInt64`) incorrectly **rejects** valid JSON numbers whose mathematical value is `0` but which are written with a negative exponent large enough that trailing/leading-zero trimming does not fully offset it. The core weaseljson parser accepts these as valid JSON, and they are integers representable as `int64_t` (value `0`), so the schemagen builder should accept them too. ## Affected code `contrib/schemagen/weaseljson_schemagen.py`, `Builder._builder` -> the `parseJsonInt64` function emitted into the generated header (around lines 1055-1064 of the generator): ```cpp int64_t trim = 0; while (!digits.empty() && digits.back() == '0') { digits.pop_back(); ++trim; } int64_t finalExp = exp - fracDigits + trim; if (finalExp < 0) return false; // <-- line 1061: rejects zero too early size_t leadingZeros = 0; while (leadingZeros < digits.size() && digits[leadingZeros] == '0') ++leadingZeros; if (leadingZeros == digits.size()) { out = 0; return true; } // <-- line 1064: zero detection ``` The `if (finalExp < 0) return false;` guard runs **before** the zero-detection guard. When the value is `0`, all significant digits are trimmed away (or are leading zeros), so `digits` is empty/all-zero, but `finalExp` can still be negative, causing the early `return false` before the `out = 0` branch is reached. ## Reproduction Schema `s.json`: ```json {"type":"integer"} ``` Generate and run a tiny harness: ```sh python3 contrib/schemagen/weaseljson_schemagen.py s.json -o gen.h --namespace sg ``` ```cpp #include "gen.h" #include <cstdio> #include <cstring> int main() { const char *cases[] = {"0", "0e-1", "0e-2", "0.0e-2", "-0e-2", "0e-20"}; for (auto s : cases) { sg::RootBuilder b; char buf[64]; std::strncpy(buf, s, sizeof(buf)-1); buf[sizeof(buf)-1]=0; WeaselJsonStatus st = b.feed(buf, std::strlen(s)); st = b.finish(); std::printf("%s -> %d\n", s, st); } } ``` Observed output: ``` 0 -> 0 (OK) 0e-1 -> 0 (OK) 0e-2 -> 2 (REJECT) <-- bug 0.0e-2 -> 2 (REJECT) <-- bug -0e-2 -> 2 (REJECT) <-- bug 0e-20 -> 2 (REJECT) <-- bug ``` The same rejection occurs for an integer object field, e.g. schema `{"type":"object","required":["age"],"properties":{"age":{"type":"integer"}}}` with input `{"age":0e-2}` -> `WeaselJson_REJECT`. ## Expected vs actual - `0e-2`, `0.0e-2`, `-0e-2`, `0e-20` are valid JSON numbers (RFC 8259: `0` integer part, optional fraction, `e-2` exponent). The core `WeaselJsonParser_parse` returns `WeaselJson_OK` for each of them (verified directly). - Their mathematical value is `0`, which is an integer with no fractional part and is representable as `int64_t`. - Per the schemagen README, `JSON Schema "integer" accepts any number with no fractional part, including exponent/decimal forms like 1e3 or 2.0`, and only `a number not representable in the target type (e.g. 1.5 for an integer)` is rejected. - Therefore these inputs should be accepted with value `0`. The generated builder instead returns `WeaselJson_REJECT`. Note the boundary is the exponent magnitude vs. the number of trimmed zeros: `0e-1` is accepted (`finalExp = -1 + 1 = 0`), but `0e-2` is rejected (`finalExp = -2 + 1 = -1 < 0`). Non-zero values with negative exponents are correctly rejected (e.g. `1e-2` = 0.01), so the bug is specific to the zero case. ## Impact A schema-valid JSON document containing a perfectly ordinary integer value `0` written in exponent form (e.g. `0e-2`, `0.0e-2`, `-0e-2`) is silently rejected by the generated parser as a schema violation. Users feeding data produced by other JSON encoders that emit zero in scientific notation will see spurious `WeaselJson_REJECT` for valid input. ## Suggested fix Move the zero-detection check before the `finalExp < 0` guard (or skip the negative-`finalExp` rejection when the remaining `digits` are all zero), e.g.: ```cpp int64_t finalExp = exp - fracDigits + trim; size_t leadingZeros = 0; while (leadingZeros < digits.size() && digits[leadingZeros] == '0') ++leadingZeros; if (leadingZeros == digits.size()) { out = 0; return true; } // zero is representable regardless of exponent if (finalExp < 0) return false; if (leadingZeros > 0) digits.erase(0, leadingZeros); ```
weaselbot was assigned by andrew 2026-07-20 01:00:38 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: weaselab/weaseljson#56