What
GlutenSQLQueryTestSuite.createScalaTestCase matches the supported list and the ignore list with contains on gluten-ut/spark34, and with == on gluten-ut/spark35, spark40 and spark41. The exact-match versions carry a comment saying it is deliberate:
// gluten-ut/spark35/.../GlutenSQLQueryTestSuite.scala:459
// Modified for Gluten to use exact name matching.
!supportedList.exists(t => testCase.name.toLowerCase(Locale.ROOT) == t.toLowerCase(Locale.ROOT))
// gluten-ut/spark34/.../GlutenSQLQueryTestSuite.scala:315
!supportedList.exists(t => testCase.name.toLowerCase(Locale.ROOT).contains(t.toLowerCase(Locale.ROOT)))
spark34 never got that change.
Why it matters
Two consequences, and the second one is the reason I am filing this.
spark34 runs more than its list says. A UDF .sql file expands into one test per UDF kind, named like udf/udf-count.sql - Scala UDF. Under contains, a short entry such as "count.sql" matches that name, so it silently enables all of that file's UDF variants. VeloxSQLQueryTestSettings on spark34 has nine such short entries: count.sql, cross-join.sql, having.sql, inner-join.sql, join-empty-relation.sql, natural-join.sql, outer-join.sql, datetime-special.sql, parse-schema-string.sql. I confirmed the effect by running the suite against Spark 3.4.4 with Velox: 222 tests run, and among them udf/udf-count.sql - Scala UDF and udf/udf-count.sql - Regular Python UDF, neither of which appears in the list in any form. On spark35 the same files do not run, because there the name has to match exactly.
The two sides cannot be compared by set arithmetic. Anyone auditing which .sql files run on which version, for example while checking what removing gluten-ut/spark33 would drop (#12863, #12807), will diff the lists and get a wrong answer for spark34. I did exactly that and it produced a nine-file "gap" that turned out not to exist: those files already run on spark34 through the short entries.
Two ways to fix it
Aligning spark34 with 3.5+ is the consistent option, but it is not free: the Regular Python UDF variants that run today on spark34 would stop, because the 3.5+ lists only enumerate ... - Scala UDF. So it is a real coverage decision, not a mechanical change, which is why this is an issue rather than a patch.
- Switch spark34 to
== and enumerate whatever should keep running, accepting that the Python UDF variants go away there as they already have on 3.5+.
- Keep
contains on spark34 and instead document it, so the next audit does not compare the lists directly.
Either way the fix should include a check that the set of tests actually run does not change unintentionally; the suite runs locally against a Spark source tree via -Dspark.test.home, so a before/after diff of the run names is cheap.
Related: #12807, #12863, #12890.
What
GlutenSQLQueryTestSuite.createScalaTestCasematches the supported list and the ignore list withcontainsongluten-ut/spark34, and with==ongluten-ut/spark35,spark40andspark41. The exact-match versions carry a comment saying it is deliberate:spark34 never got that change.
Why it matters
Two consequences, and the second one is the reason I am filing this.
spark34 runs more than its list says. A UDF
.sqlfile expands into one test per UDF kind, named likeudf/udf-count.sql - Scala UDF. Undercontains, a short entry such as"count.sql"matches that name, so it silently enables all of that file's UDF variants.VeloxSQLQueryTestSettingson spark34 has nine such short entries:count.sql,cross-join.sql,having.sql,inner-join.sql,join-empty-relation.sql,natural-join.sql,outer-join.sql,datetime-special.sql,parse-schema-string.sql. I confirmed the effect by running the suite against Spark 3.4.4 with Velox: 222 tests run, and among themudf/udf-count.sql - Scala UDFandudf/udf-count.sql - Regular Python UDF, neither of which appears in the list in any form. On spark35 the same files do not run, because there the name has to match exactly.The two sides cannot be compared by set arithmetic. Anyone auditing which
.sqlfiles run on which version, for example while checking what removinggluten-ut/spark33would drop (#12863, #12807), will diff the lists and get a wrong answer for spark34. I did exactly that and it produced a nine-file "gap" that turned out not to exist: those files already run on spark34 through the short entries.Two ways to fix it
Aligning spark34 with 3.5+ is the consistent option, but it is not free: the Regular Python UDF variants that run today on spark34 would stop, because the 3.5+ lists only enumerate
... - Scala UDF. So it is a real coverage decision, not a mechanical change, which is why this is an issue rather than a patch.==and enumerate whatever should keep running, accepting that the Python UDF variants go away there as they already have on 3.5+.containson spark34 and instead document it, so the next audit does not compare the lists directly.Either way the fix should include a check that the set of tests actually run does not change unintentionally; the suite runs locally against a Spark source tree via
-Dspark.test.home, so a before/after diff of the run names is cheap.Related: #12807, #12863, #12890.