Skip to content

[CH] Record the real fallbackSummary node counts on ClickHouse so GlutenImplicitsTest can stop skipping three cases #12862

Description

@LuciferYang

What happens

Three of the six fallbackSummary cases in GlutenImplicitsTest are skipped on ClickHouse rather than asserted, because nobody has recorded what ClickHouse actually reports. gluten-ut/test/src/test/scala/org/apache/spark/sql/GlutenImplicitsTest.scala:103:

private def assumeVeloxOnly(): Unit = assume(BackendTestUtils.isVeloxBackendLoaded())

called from fallbackSummary with shuffle (:117), fallbackSummary with cache (:151) and fallbackSummary with cached data and shuffle (:163).

Why the numbers are unknown

The three cases assert exact numGlutenNodes counts, and ClickHouse produces different ones. Before #12840, gluten-ut/spark33's ClickHouseTestSettings handled this with three excludeGlutenTest entries. An exclude records that a case is off, not what it would have reported, so the real ClickHouse counts were never written down anywhere. #12840 moved the suite into the version-agnostic gluten-ut/test module and carried the same three exclusions over as assume, which keeps the behaviour identical and puts the reason next to the case, but does not recover the missing numbers.

Why it matters

GlutenImplicits.fallbackSummary has no ClickHouse coverage for shuffle or cached relations. The other three cases do run on ClickHouse, so the gap is specifically the plan shapes where the two backends differ, which is where a regression is most likely to hide.

Suggested fix

Run the suite once on a ClickHouse build, read the reported counts out of the assertion failure messages (each assert already passes df.fallbackSummary() as the clue), then replace assumeVeloxOnly() with a per-backend expected value. Something like val expected = if (BackendTestUtils.isCHBackendLoaded()) x else y keeps both backends asserting.

Related: #12845 is the other place where a migrated case is disabled because ClickHouse expectations were never verified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions