Skip to content

[no-release-notes] Add skip for VECTOR index engine test#2827

Open
fulghum wants to merge 1 commit into
mainfrom
fulghum/skip-vector-test
Open

[no-release-notes] Add skip for VECTOR index engine test#2827
fulghum wants to merge 1 commit into
mainfrom
fulghum/skip-vector-test

Conversation

@fulghum

@fulghum fulghum commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor
Main PR
covering_index_scan_postgres 1886.66/s 1867.00/s -1.1%
groupby_scan_postgres 138.15/s 138.26/s 0.0%
index_join_postgres 654.59/s 653.69/s -0.2%
index_join_scan_postgres 801.89/s 804.28/s +0.2%
index_scan_postgres 24.89/s 24.81/s -0.4%
oltp_delete_insert_postgres 820.37/s 750.68/s -8.5%
oltp_insert 690.29/s 653.09/s -5.4%
oltp_point_select 2922.92/s 2891.88/s -1.1%
oltp_read_only 2930.39/s 2899.21/s -1.1%
oltp_read_write 2328.15/s 2165.57/s -7.0%
oltp_update_index 723.69/s 696.63/s -3.8%
oltp_update_non_index 752.30/s 742.75/s -1.3%
oltp_write_only 1747.00/s 1742.00/s -0.3%
select_random_points 1863.66/s 1847.56/s -0.9%
select_random_ranges 1113.81/s 1092.17/s -2.0%
table_scan_postgres 23.43/s 22.94/s -2.1%
types_delete_insert_postgres 776.23/s 779.34/s +0.4%
types_table_scan_postgres 8.24/s 7.94/s -3.7%

@fulghum fulghum force-pushed the fulghum/skip-vector-test branch from e4689f1 to 491d134 Compare June 8, 2026 23:27
@itoqa

itoqa Bot commented Jun 9, 2026

Copy link
Copy Markdown

Ito QA test results
Commit: 491d134: 8 test cases ran, 1 failed ❌, 7 passed ✅, 0 additional findings ⚠️.

Summary

Across 8 total cases, 7 passed and 1 failed: merge-harness and matcher-precision checks were largely stable, confirming the configured vector-index skip remained active, non-matching and near-match queries still executed, title drift behavior matched expectations, and repeated runs were deterministic. The key finding was a High-severity release-signal defect introduced by this PR: TestDoltMerge can report green while the vector-index merge scenario (“three-way merge of table with vector index”) is deterministically excluded via SkipQueryTest substring matching, so CI pass status can mask an untested compatibility gap.

Tests run by Ito

View full run

Result Severity Type Description
High severity Compatibility The suite can pass while excluding three-way merge of table with vector index, so the reported green signal does not prove vector-merge compatibility coverage.
Compatibility TestDoltMerge completed successfully while explicitly signaling that VECTOR merge coverage is intentionally skipped for dialect compatibility reasons.
Matcher Controlled broad-token probe showed overlap risk only when an intentionally generic skip fragment was injected.
Matcher Non-matching query text remained executable with the current vector skip phrase configured.
Matcher Current vector-index skip phrase stayed specific in focused checks and did not trigger near-match suppression.
Merge TestDoltMerge completed successfully and the vector-index merge scenario remained excluded via the configured skip phrase.
Merge The harness behavior was verified: when upstream title text drifts, the literal skip phrase no longer matches and the renamed case stays executable.
Merge Two consecutive TestDoltMerge runs stayed stable, showing skip entries were preserved on the execution harness with no ordering loss.

Tip

Reply with @itoqa to send us feedback on this test run.

Comment thread testing/go/enginetest/doltgres_engine_test.go
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor
Main PR
Total 42090 42090
Successful 18259 18259
Failures 23831 23831
Partial Successes1 5332 5332
Main PR
Successful 43.3809% 43.3809%
Failures 56.6191% 56.6191%

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@elianddb elianddb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants