schemagen emits type names with non-identifier characters (e.g. '.', '+', '/') for $defs/property names containing them (generated code does not compile)
#60
weaseljson_schemagen.py generates C++ type names (enum classes and structs) that contain characters which are not valid in a C++ identifier — such as ., +, @, !, /, (, # — when a JSON Schema property key, $defs name, or array hint contains such a character. The generated header then fails to compile. This happens for perfectly valid JSON Schemas: JSON object keys (including $defs/definitions keys and properties keys) may be any string.
This is distinct from issue #59, which is about type names that start with a digit. The fix proposed in #59 (prefixing a digit-leading name with _) does not address names containing interior special characters such as foo.bar (which becomes Foo.bar).
Root cause
Type names are produced by camel() via unique_name() in contrib/schemagen/weaseljson_schemagen.py:
defcamel(name):# line 153parts=[pforpinname.replace("-"," ").replace("_"," ").split(" ")ifp]ifnotparts:return"T"return"".join(p[:1].upper()+p[1:]forpinparts)
camel() only splits on -, _, and space. It does not replace other non-alphanumeric characters, so camel("foo.bar") returns "Foo.bar", camel("foo+bar") returns "Foo+bar", etc.
The member-name helper sanitize() (line 111) does handle this correctly:
but type names go through camel() -> unique_name() (line 251-252, base = camel(hint)), which never applies that sanitization. So the special character is emitted verbatim into the enum/struct type name (and into the generated <Name>_names[] array and Kind enumerator).
Note the inconsistency visible in the output below: the member name is sanitized to foo_bar, but the type name is left as Foo.bar.
gen.h:17:12: error: use of enum 'Foo' without previous declaration
17 | enum class Foo.bar : int { a, b };
| ^~~
gen.h:17:15: error: expected unqualified-id before '.' token
The same defect affects generated struct names, e.g. for an inline object property whose key contains a special character:
Characters confirmed to produce non-compiling output: ., +, @, !, /, (, # (and any other non-alphanumeric character other than -, _, and space).
Expected vs actual
Expected: schemagen should produce a compilable header for any valid JSON Schema whose property/$defs names contain non-identifier characters, by replacing them the way sanitize() already does for member names (e.g. foo.bar -> Foo_bar).
Actual: schemagen exits 0 but emits type names containing the raw special characters, and the generated header does not compile.
Impact
Any schema using a property key or $defs name containing a character like . with an inline enum/object (or referenced via $ref) yields non-compiling generated code, with no error reported at generation time. Dotted names are common and valid (e.g. $defs keys like "Error.v2" or property keys like "foo.bar"), so this is reachable with ordinary schemas, not only adversarial ones.
Files / lines
contrib/schemagen/weaseljson_schemagen.py:153-157 — camel() keeps non-alphanumeric characters (only splits on -/_/space).
contrib/schemagen/weaseljson_schemagen.py:251-252 — unique_name() builds type names from camel(hint) without sanitizing.
contrib/schemagen/weaseljson_schemagen.py:111-121 — sanitize() shows the existing correct handling for member names (replace non-alphanumeric with _), which is not applied to generated type names.
## Summary
`weaseljson_schemagen.py` generates C++ type names (enum classes and structs) that contain characters which are not valid in a C++ identifier — such as `.`, `+`, `@`, `!`, `/`, `(`, `#` — when a JSON Schema property key, `$defs` name, or array `hint` contains such a character. The generated header then fails to compile. This happens for perfectly valid JSON Schemas: JSON object keys (including `$defs`/`definitions` keys and `properties` keys) may be any string.
This is distinct from issue #59, which is about type names that *start with a digit*. The fix proposed in #59 (prefixing a digit-leading name with `_`) does **not** address names containing interior special characters such as `foo.bar` (which becomes `Foo.bar`).
## Root cause
Type names are produced by `camel()` via `unique_name()` in `contrib/schemagen/weaseljson_schemagen.py`:
```python
def camel(name): # line 153
parts = [p for p in name.replace("-", " ").replace("_", " ").split(" ") if p]
if not parts:
return "T"
return "".join(p[:1].upper() + p[1:] for p in parts)
```
`camel()` only splits on `-`, `_`, and space. It does **not** replace other non-alphanumeric characters, so `camel("foo.bar")` returns `"Foo.bar"`, `camel("foo+bar")` returns `"Foo+bar"`, etc.
The member-name helper `sanitize()` (line 111) does handle this correctly:
```python
def sanitize(name, fallback="x"):
out = []
for ch in name:
out.append(ch if (ch.isalnum() or ch == "_") else "_")
...
```
but type names go through `camel()` -> `unique_name()` (line 251-252, `base = camel(hint)`), which never applies that sanitization. So the special character is emitted verbatim into the enum/struct type name (and into the generated `<Name>_names[]` array and `Kind` enumerator).
Note the inconsistency visible in the output below: the *member* name is sanitized to `foo_bar`, but the *type* name is left as `Foo.bar`.
## Reproduction
Minimal schema (`schema.json`):
```json
{"type":"object","additionalProperties":false,"properties":{"foo.bar":{"enum":["a","b"]}}}
```
```sh
python3 contrib/schemagen/weaseljson_schemagen.py schema.json -o gen.h --namespace ts
```
`schemagen` exits 0 and emits (note `enum class Foo.bar`, `std::optional<Foo.bar>`, `Foo.bar_names`):
```cpp
enum class Foo.bar : int { a, b };
struct Root1 {
std::optional<Foo.bar> foo_bar; // "foo.bar"
...
};
...
static constexpr const char *Foo.bar_names[] = { "a", "b" };
```
Compiling any TU that includes `gen.h` fails:
```
gen.h:17:12: error: use of enum 'Foo' without previous declaration
17 | enum class Foo.bar : int { a, b };
| ^~~
gen.h:17:15: error: expected unqualified-id before '.' token
```
The same defect affects generated **struct** names, e.g. for an inline object property whose key contains a special character:
```json
{"type":"object","additionalProperties":false,"properties":{"foo.bar":{"type":"object","additionalProperties":false,"properties":{}}}}
```
emits `struct Foo.bar;` / `struct Foo.bar { ... };` / `std::optional<Foo.bar> foo_bar;`, which also fails to compile.
It is also reachable via a `$ref` to a `$defs` entry whose name contains a special character:
```json
{"type":"object","additionalProperties":false,"properties":{"x":{"$ref":"#/$defs/foo.bar"}},"$defs":{"foo.bar":{"type":"object","additionalProperties":false,"properties":{}}}}
```
Characters confirmed to produce non-compiling output: `.`, `+`, `@`, `!`, `/`, `(`, `#` (and any other non-alphanumeric character other than `-`, `_`, and space).
## Expected vs actual
- **Expected:** schemagen should produce a compilable header for any valid JSON Schema whose property/`$defs` names contain non-identifier characters, by replacing them the way `sanitize()` already does for member names (e.g. `foo.bar` -> `Foo_bar`).
- **Actual:** schemagen exits 0 but emits type names containing the raw special characters, and the generated header does not compile.
## Impact
Any schema using a property key or `$defs` name containing a character like `.` with an inline enum/object (or referenced via `$ref`) yields non-compiling generated code, with no error reported at generation time. Dotted names are common and valid (e.g. `$defs` keys like `"Error.v2"` or property keys like `"foo.bar"`), so this is reachable with ordinary schemas, not only adversarial ones.
## Files / lines
- `contrib/schemagen/weaseljson_schemagen.py:153-157` — `camel()` keeps non-alphanumeric characters (only splits on `-`/`_`/space).
- `contrib/schemagen/weaseljson_schemagen.py:251-252` — `unique_name()` builds type names from `camel(hint)` without sanitizing.
- `contrib/schemagen/weaseljson_schemagen.py:111-121` — `sanitize()` shows the existing correct handling for member names (replace non-alphanumeric with `_`), which is not applied to generated type names.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
weaseljson_schemagen.pygenerates C++ type names (enum classes and structs) that contain characters which are not valid in a C++ identifier — such as.,+,@,!,/,(,#— when a JSON Schema property key,$defsname, or arrayhintcontains such a character. The generated header then fails to compile. This happens for perfectly valid JSON Schemas: JSON object keys (including$defs/definitionskeys andpropertieskeys) may be any string.This is distinct from issue #59, which is about type names that start with a digit. The fix proposed in #59 (prefixing a digit-leading name with
_) does not address names containing interior special characters such asfoo.bar(which becomesFoo.bar).Root cause
Type names are produced by
camel()viaunique_name()incontrib/schemagen/weaseljson_schemagen.py:camel()only splits on-,_, and space. It does not replace other non-alphanumeric characters, socamel("foo.bar")returns"Foo.bar",camel("foo+bar")returns"Foo+bar", etc.The member-name helper
sanitize()(line 111) does handle this correctly:but type names go through
camel()->unique_name()(line 251-252,base = camel(hint)), which never applies that sanitization. So the special character is emitted verbatim into the enum/struct type name (and into the generated<Name>_names[]array andKindenumerator).Note the inconsistency visible in the output below: the member name is sanitized to
foo_bar, but the type name is left asFoo.bar.Reproduction
Minimal schema (
schema.json):schemagenexits 0 and emits (noteenum class Foo.bar,std::optional<Foo.bar>,Foo.bar_names):Compiling any TU that includes
gen.hfails:The same defect affects generated struct names, e.g. for an inline object property whose key contains a special character:
emits
struct Foo.bar;/struct Foo.bar { ... };/std::optional<Foo.bar> foo_bar;, which also fails to compile.It is also reachable via a
$refto a$defsentry whose name contains a special character:Characters confirmed to produce non-compiling output:
.,+,@,!,/,(,#(and any other non-alphanumeric character other than-,_, and space).Expected vs actual
$defsnames contain non-identifier characters, by replacing them the waysanitize()already does for member names (e.g.foo.bar->Foo_bar).Impact
Any schema using a property key or
$defsname containing a character like.with an inline enum/object (or referenced via$ref) yields non-compiling generated code, with no error reported at generation time. Dotted names are common and valid (e.g.$defskeys like"Error.v2"or property keys like"foo.bar"), so this is reachable with ordinary schemas, not only adversarial ones.Files / lines
contrib/schemagen/weaseljson_schemagen.py:153-157—camel()keeps non-alphanumeric characters (only splits on-/_/space).contrib/schemagen/weaseljson_schemagen.py:251-252—unique_name()builds type names fromcamel(hint)without sanitizing.contrib/schemagen/weaseljson_schemagen.py:111-121—sanitize()shows the existing correct handling for member names (replace non-alphanumeric with_), which is not applied to generated type names.