Skip to content

Fix named Tuple/Nested columns whose element name requires backtick quoting - #504

Open
polyglotAI-bot wants to merge 1 commit into
polyglot/json-quoted-typed-pathsfrom
polyglot/cs-quoted-tuple-element-names
Open

Fix named Tuple/Nested columns whose element name requires backtick quoting#504
polyglotAI-bot wants to merge 1 commit into
polyglot/json-quoted-typed-pathsfrom
polyglot/cs-quoted-tuple-element-names

Conversation

@polyglotAI-bot

Copy link
Copy Markdown
Collaborator

Description

Stacked on #503 (base is polyglot/json-quoted-typed-paths, which makes the type tokenizer backtick-aware and adds the identifier helpers). Review/merge #503 first; GitHub will retarget this PR to main automatically when #503 merges.

ClickHouse reports a named Tuple / Nested element name back-quoted whenever it needs quoting, so the type string on the wire can be Tuple(`p q` Int64, r String) (verified on 26.5). TypeConverter.ExtractTypeName strips the element name by splitting the declaration on its first space, which cuts such a name in half: `p q` Int64 becomes `p / q` Int64, the element type resolves to q` Int64, and the whole query fails with ArgumentException: Unknown type. Any SELECT of a column with such a type is unreadable, and an insert into one fails the same way (the insert path resolves the destination column types through the same parser).

The fix locates the name/type separator past the quoted identifier instead of at the first space. The scan for this already existed as a private helper in JsonType (added in #503 for the equivalent JSON typed-path bug, issue #502); it is factored out into StringExtensions.IndexOfNameTypeSeparator and reused by both call sites rather than duplicated.

Unquoted and unnamed elements keep resolving exactly as before, and the failure mode for a malformed declaration stays the same ArgumentException.

Changes

  • ClickHouse.Driver/Utility/StringExtensions.cs: new internal IndexOfNameTypeSeparator() — skips a leading backtick-quoted identifier (honouring ``` escapes, and accepting doubled backticks for hand-written type names) and returns the index of the separating space, or -1.
  • ClickHouse.Driver/Types/TypeConverter.cs: ExtractTypeName uses that helper instead of Split(" ", 2); the now-unused Separator field is removed.
  • ClickHouse.Driver/Types/JsonType.cs: uses the shared helper; its private IndexOfPathTypeSeparator copy is removed (no behavior change).
  • CHANGELOG.md / RELEASENOTES.md: entry under Unreleased → Bug Fixes.

Test

ClickHouse.Driver.Tests/Types/TupleTypeTests.cs:

  • Parse-level cases for quoted element names in Tuple and Nested: names containing a space, .+space, comma, parentheses, \``, ', \n, \, doubled backticks; scalar, Decimal(10, 2), Map(String, Array(Int32))andNullable(String)element types; quoted names wrapped inArray/Map/an outer Tuple; a single-element tuple; and a quoted name next to single-quoted element arguments (Enum8('x y' = 1, …), DateTime64(3, 'Europe/Amsterdam')`) so both quote kinds are honoured in one declaration.
  • Contrast cases pinning that unnamed and unquoted-named elements (Tuple(String, Int32), Tuple(name String, age Int32), Nested(Id Nullable(String), Comment Nullable(String)), …) resolve to exactly the same element types as before.
  • Malformed quoting (unterminated backtick, name with no type) still throws ArgumentException, for both Tuple and Nested.
  • Live-server round trips through the real entry points: reading Tuple(`p q` Int64, r String) and Nested(`a b` Decimal(10, 2), c String), and InsertBinaryAsync into a table column of type Tuple(`p q` Int64, r String) with a read-back.

15 of these fail on the base branch (ArgumentException: Unknown type: p q Int64) and all pass with the fix. Full ClickHouse.Driver.Tests suite on net10.0: 9659 passed, 0 failed, 142 skipped — no existing test changed.

Pre-PR validation gate

  • Deterministic repro confirmed (select cast(tuple(toInt64(1), 'a') as Tuple(p q Int64, r String)) throws on the base branch)
  • Root cause documented above
  • Fix targets the root cause (separator location, not a symptom guard)
  • Tests fail without the fix, pass with it
  • No existing tests broken or weakened
  • Convention compliance verified per AGENTS.md (integration tests preferred, TestCase parametrization, three-part test names, CHANGELOG + RELEASENOTES updated, no public API change — all touched members are internal)

…uoting

TypeConverter.ExtractTypeName stripped a named element's name by splitting
the declaration on its first space, which cuts a backtick-quoted name in
half: `p q` Int64 became "`p" / "q` Int64", so the element type resolved to
"q` Int64" and the whole query failed with ArgumentException: Unknown type.

The separator scan that the JSON typed-path fix introduced is factored out
of JsonType into StringExtensions.IndexOfNameTypeSeparator and reused here,
so the quoted identifier is skipped as a whole before the name/type
separator is located.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7581fc8. Configure here.

if (str[i] == '\\' || (str[i] == '`' && i + 1 < str.Length && str[i + 1] == '`'))
{
i++;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doubled backtick closes identifier early

Medium Severity

When a hand-written backtick-quoted element name uses SQL-style doubled backticks ( ) for an embedded backtick, `IndexOfNameTypeSeparator` advances only one character after detecting the pair. The next loop iteration treats the second backtick as the closing quote, so the name/type boundary can fall on a space still inside the identifier (e.g. a``b c Int64 ``), yielding a garbage type fragment and parse failure.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7581fc8. Configure here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — I checked this empirically and I don't think it holds: the loop already consumes both characters of a doubled backtick.

The i++ inside the branch is followed by the for statement's own i++, so detecting the pair at index i leaves the next iteration at i+2, i.e. past the second backtick — not on it. For the exact example in the comment ( `ab c Int64 ``): the pair is at indexes 2–3, so the next iteration starts at index 4 (b`), the closing backtick is found at 7, and the separator is the space at 8.

Verified by invoking the shipped method on this branch's build (ClickHouse.Driver.dll, net10.0) rather than by reading:

input=[`a b` Int64]      sep=5 name=[`a b`]      type=[Int64]
input=[`a\`b c` Int64]   sep=8 name=[`a\`b c`]   type=[Int64]
input=[`a``b c` Int64]   sep=8 name=[`a``b c`]   type=[Int64]   <- the reported case
input=[`a``b` Int64]     sep=6 name=[`a``b`]     type=[Int64]
input=[`a` Int64]        sep=3 name=[`a`]        type=[Int64]
input=[plain Int64]      sep=5 name=[plain]      type=[Int64]
input=[Int64]            sep=-1 (no separator)
input=[`unterminated Int64] sep=-1 (unterminated identifier)

And end-to-end through TypeConverter.ParseClickHouseType, both escape spellings resolve:

Tuple(`a``b c` Int64, r String)     -> Tuple(Int64,String)
Tuple(`a\`b c` Int64, r String)     -> Tuple(Int64,String)
Nested(`a b` Decimal(10, 2), c String) -> Nested(Decimal64(2),String)

For context on which spellings matter: on ClickHouse 26.5.1 the server accepts the SQL-style doubled backtick on input but always renders a type name with the backslash form, e.g.

SELECT toTypeName(CAST((1,2) AS Tuple(`a``b c` Int64, r Int64)))
-> Tuple(`a\`b c` Int64, r Int64)

so the ``` branch is the one that sees real wire data, and the doubled-backtick branch is the tolerant path for hand-written type strings — which, per the above, is handled correctly.

Leaving this thread unresolved so you can double-check the reasoning; no code change made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant