Fixed the Studio view test comparing goldens at the wrong offset - #184
Merged
fdesbiens merged 1 commit intoAug 27, 2026
Merged
Conversation
compare_file() locates the line where comparison should start, so that the
banner GUIX Studio writes into every generated file -- which carries a revision
string and a generation timestamp -- is skipped rather than compared. It looked
that line up correctly in the first file and then looked it up in the first file
again for the second:
for line in list_2:
if compare_start_string in line:
start_row_2 = list_1.index(line)
So the golden was sliced at the *generated* file's offset. While both files
happened to reach "#include" on the same line the mistake was invisible, which
is why it survived since the tests were added in eclipse-threadx#84.
The offsets diverged when 6ba13db added a ten-line MIT licence header to all
111 golden .c and .h files. GUIX Studio does not emit that header, so every
golden now reaches its first #include ten lines later than the file it is
compared against. The golden was therefore sliced ten lines early and compared
banner text against source.
That made every one of the 107 .c and .h comparisons in the suite fail -- all 29
reported test failures -- while the .csv, .xliff and .xml comparisons passed.
Those use skip_line instead of compare_start_string and never reach this code,
which is exactly the split the failure log shows.
The reported mismatch is the same in all 29: a generated "#include" line against
a golden banner comment. Confirmed arithmetically -- in every case the golden
line reported sits exactly ten lines before that golden's own first #include:
generic_16bpp_resources.c reported idx 13, #include at 23, delta 10
generic_16bpp_resources.h reported idx 16, #include at 26, delta 10
generic_16bpp_specifications.c reported idx 14, #include at 24, delta 10
generic_16bpp_specifications.h reported idx 16, #include at 26, delta 10
compare_output_file() is the only caller that passes compare_start_string, so
the change cannot affect the xliff, xml or plain-file comparisons.
Verified by running the suite locally against a Studio built from this tree.
Seven suites that fail in CI now pass with no mismatches at all, each gaining
exactly the tests it had been failing:
CI (broken) local (fixed)
Font 9 / 1 10 / 0
Multi-Themes 18 / 1 19 / 0
Project Import 10 / 2 12 / 0
Trigger Edit 6 / 1 7 / 0
Trigger Target Rename 1 / 1 2 / 0
Bidi Text 3 / 1 4 / 0
Widget Name 3 / 1 4 / 0
Project Import compares a generated specifications file, so this also shows that
the screen flow prototype block added by eclipse-threadx#173 does not disturb these goldens:
that block is only emitted for projects that use Screen Flow, and these do not.
No golden file is regenerated. The goldens still carry "GUIX Studio Revision
6.1.12.0" and a 2022 timestamp in their banners, and still name Azure RTOS
rather than Eclipse ThreadX, but the banner is what the comparison is supposed
to skip -- so none of that needs to be touched to make the suite correct.
This workflow had never actually executed before 2026-08-27. Every earlier run
was cancelled after queueing 24 hours for the retired windows-2019 image, or
sat awaiting approval on a fork pull request, including the run for the v6.5.1
release itself. eclipse-threadx#174 gave it a runner again; this is the first result it has
ever produced, and the defect it found is in the test harness rather than in
GUIX or in Studio.
Note that studio_view_test.yml triggers only on pull requests targeting master,
so this change is not exercised by its own pull request into dev. It is
validated by the release pull request that carries dev to master.
Assisted-by: Claude Code (Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the 29 failures in GUIX Studio View Test on #183.
What was wrong
compare_file()locates the line where comparison should start, so the banner GUIX Studio writes into every generated file — which carries a revision string and a generation timestamp — is skipped rather than compared. It looked that line up correctly in the first file, then looked it up in the first file again for the second:So the golden was sliced at the generated file's offset. While both files happened to reach
#includeon the same line the mistake was invisible, which is why it survived since the tests were added in #84.The offsets diverged when
6ba13dbd("Added missing copyright headers") added a ten-line MIT licence header to all 111 golden.cand.hfiles. GUIX Studio does not emit that header, so every golden now reaches its first#includeten lines later than the file it is compared against. The golden was therefore sliced ten lines early, and banner text got compared against source.Blast radius, and why it looked like scattered flakiness
Every one of the **107
.c/.hcomparisons failed — the whole set, not a sample. They roll up into the 29 reported test failures because a test stops at its first mismatching file.The
.csv,.xliffand.xmlcomparisons all passed. Those wrappers passskip_linerather thancompare_start_string, so they never reach this code. That split is exactly what the failure log shows.The reported mismatch is identical in all 29 — a generated
#includeline against a golden banner comment. Confirmed arithmetically: in every case the golden line reported sits exactly ten lines before that golden's own first#include.#includegeneric_16bpp_resources.cgeneric_16bpp_resources.hgeneric_16bpp_specifications.cgeneric_16bpp_specifications.hcompare_output_file()is the only caller that passescompare_start_string, so this change cannot affect the xliff, xml or plain-file comparisons.Verification
Run locally against a Studio built from this tree. Seven suites that fail in CI now pass with no mismatches at all, each gaining exactly the tests it had been failing:
Project Import compares a generated specifications file, so this also shows the screen flow prototype block added by #173 does not disturb these goldens — that block is only emitted for projects using Screen Flow, and these do not.
No golden file is regenerated
The goldens still carry
GUIX Studio Revision 6.1.12.0and a 2022 timestamp, and still name Azure RTOS rather than Eclipse ThreadX. But the banner is precisely what the comparison is meant to skip, so none of that needs touching to make the suite correct. Worth refreshing one day; not needed here, and not something to do while a release is waiting.Context
This workflow had never actually executed before 2026-08-27. Every earlier run was cancelled after queueing 24 hours for the retired
windows-2019image, or sat awaiting approval on a fork pull request — including the run for the v6.5.1 release itself. #174 gave it a runner again; this is the first result it has ever produced, and the defect it found is in the test harness rather than in GUIX or in Studio.The GUI automation itself is fine on a hosted runner: roughly 900 assertions passed, including the CRC32 canvas checks and 246 copy/paste-between-projects tests.
Note on validation
studio_view_test.ymltriggers only on pull requests targetingmaster, so this change is not exercised by its own pull request intodev. It is validated by the release pull request that carriesdevtomaster(#183), which re-runs the suite once this lands.