Fix reading JSON typed paths whose name requires backtick quoting

[polyglotAI-bot]

Description

Fixes #502.

ClickHouse allows a JSON typed path name to contain characters that need backtick quoting — a space, a comma, a parenthesis. The server accepts, stores and round-trips such columns, and reports the column type as e.g. JSON(`a b` Int64). The driver could not parse that type name at all: because the failure happens while parsing the column type from the response header, the entire query failed with SerializationException: Unsupported path in JSON hint, so any table containing such a column was unreadable with no way to opt out.

Two independent causes:

1. Types/Grammar/Tokenizer.cs treated ' as the only quote character, so a comma or parenthesis inside a backtick-quoted identifier broke the token in half before it ever reached JsonType.Parse (JSON(`a,b` Int64) → hint `a).
2. JsonType.Parse split each hint on every space and required exactly two parts, so any path containing a space threw. It also only trimmed backticks off the path (Trim('')) rather than unescaping it — and the keys of HintedTypes` must match the raw path names that arrive on the wire, otherwise the hinted type is silently not applied to that path.

Verified against ClickHouse 26.5: the server normalizes such a type to a single fully-enclosing backtick pair around the whole path (JSON(a.b c Int64) is reported back as JSON(`a.b c` Int64)) and escapes with a backslash inside the quotes (a path containing a backtick is reported as `a\`b`), which is what the parser now expects.

Changes

• ClickHouse.Driver/Types/Grammar/Tokenizer.cs — track which quote character opened the current quoted run and accept ` alongside ', so , ( ) inside a quoted identifier no longer break the token. Escape handling is unchanged.
• ClickHouse.Driver/Types/JsonType.cs — replace Value.Split(' ') with IndexOfPathTypeSeparator, which skips a leading backtick-quoted identifier (spaces, commas and escaped characters included) before looking for the space that separates the path from its type. A hint with no type, or an unterminated quoted identifier, still throws the same SerializationException.
• ClickHouse.Driver/Utility/StringExtensions.cs — add DiscloseColumnName, which strips the enclosing backticks and unescapes the escape sequences ClickHouse uses inside a quoted identifier, so the resulting path matches the wire path name.
• CHANGELOG.md / RELEASENOTES.md — entry under Unreleased → Bug Fixes.

No public API change (Tokenizer's signature is untouched; StringExtensions is internal).

Test

• JsonTypeTests.JsonTypeTestCases — six new end-to-end cases (real SELECT against the server) for quoted paths containing a space, a comma, a parenthesis, an escaped backtick and an escaped single quote, plus a space with a parameterized type (Decimal(10, 2)).
• JsonTypeTests.ShouldSelectQuotedTypedPathWhenPathIsNested — end-to-end read of JSON(`a.b c` Int64), pinning that a quoted path containing a dot is still reconstructed as a nested object, matching the server's own JSON output.
• JsonTypeTests.ParseShouldUnquotePathWhenPathIsBacktickQuoted / ParseShouldMapQuotedPathToItsHintedType — parse-level cases for the same matrix plus \n / \\ escapes, a nested composite (Map(String, Array(Int32))) and coexistence with max_dynamic_paths= and SKIP `x,y` , asserting both the unescaped key and the resolved hinted type.
• JsonTypeTests.ParseShouldKeepPathWhenPathIsNotQuoted — contrast case: unquoted, dotted and settings/SKIP/SKIP REGEXP hints keep their existing behavior (passes both before and after this change).
• JsonTypeTests.ParseShouldThrowWhenHintHasNoType — a hint with no type still throws SerializationException.
• TypeGrammarParsingTests — round-trip cases for the shared Tokenizer change: single-quoted arguments containing spaces, commas, parentheses and escapes (Enum8, DateTime64(3, 'Europe/Amsterdam')) are unaffected, and backtick-quoted identifiers now round-trip.

All 15 originally-failing new cases fail on main (b24dc52) with the reported SerializationException and pass with the fix. Full unit suite green on net10.0: 9628 passed, 0 failed (baseline before the change: 9611 passed, 0 failed).

Pre-PR validation gate

• Deterministic repro confirmed (fails on main, passes on this branch)
• Root cause documented above
• Fix targets the root cause
• Test fails without fix, passes with fix
• No existing tests broken, weakened or edited
• CHANGELOG.md and RELEASENOTES.md updated
• Convention compliance verified per AGENTS.md

Notes

• JSON(`a b` Int64) also exercises Tokenizer, which is shared by all type-name parsing; the added TypeGrammarParsingTests cases pin that single-quoted type arguments are unaffected.
• Out of scope, filed separately for follow-up: quoted named tuple / Nested element names (e.g. Tuple(`a b` Int64)) are still mis-split by TypeConverter.ExtractTypeName / NestedType.ClearFieldName. That path is broken independently of this fix (the tokenizer yields the same token before and after this change) and touching it here would bundle an unrelated change.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c97e70c. Configure here.}

[Copilot]

Pull request overview

This PR fixes parsing of ClickHouse JSON(...) type hints when a typed path name requires backtick quoting (e.g. JSON(`a b` Int64) / JSON(`a,b` Int64)), preventing entire queries from failing during response-header type parsing and restoring readability of tables containing such columns.

Changes:

• Updated the type-name tokenizer to treat backticks as a valid quote character (in addition to single quotes), so commas/parentheses inside quoted identifiers don’t split tokens.
• Updated JsonType.Parse to locate the path/type separator space while correctly skipping a leading backtick-quoted path, and to unescape quoted path names so hinted-type keys match wire paths.
• Added end-to-end and parse-level tests for quoted paths (spaces, commas, parentheses, escaped characters) and added release-note/changelog entries.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
ClickHouse.Driver/Types/Grammar/Tokenizer.cs	Recognize backticks as quote delimiters so tokenization isn’t broken by , ( ) inside quoted identifiers.
ClickHouse.Driver/Types/JsonType.cs	Robustly split JSON hint path vs type and unescape quoted paths to correctly populate HintedTypes.
ClickHouse.Driver/Utility/StringExtensions.cs	Add DiscloseColumnName to unquote + unescape ClickHouse-style backtick-quoted identifiers.
ClickHouse.Driver.Tests/Types/SyntaxParsingTests.cs	Add grammar round-trip coverage for both single-quoted literals and backtick-quoted identifiers.
ClickHouse.Driver.Tests/Types/JsonTypeTests.cs	Add integration + parse-level test matrix for quoted JSON typed paths and edge-case escapes/settings.
CHANGELOG.md	Document the bug fix under Unreleased → Bug Fixes.
RELEASENOTES.md	Document the bug fix under Unreleased → Bug Fixes.