Skip to content

fix(ci): stop fanning out the whole cockpit e2e matrix for unrelated changes - #939

Merged
blove merged 1 commit into
mainfrom
blove/ci-cockpit-e2e-scoping
Sep 1, 2026
Merged

fix(ci): stop fanning out the whole cockpit e2e matrix for unrelated changes#939
blove merged 1 commit into
mainfrom
blove/ci-cockpit-e2e-scoping

Conversation

@blove

@blove blove commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

PR #932 changed exactly three files under apps/website/src and ran 41 Cockpit — e2e lanes. Two independent scoping defects stacked.

1. The cockpit shell claimed the e2e matrix

apps/cockpit/project.json carried scope:cockpit-e2e. But the matrix dispatches nx e2e for the standalone Angular cap apps under cockpit/**, and:

  • 0 of 40 caps depend on the apps/cockpit Next.js shell (checked against the nx project graph)
  • no workflow anywhere runs the shell's own e2e target

So the tag could never select real work — only over-select. And it did: apps/cockpit statically depends on apps/website, so a website-only PR made the shell nx-affected and flipped cockpit_e2e true. From that run's CI scope log:

Affected projects (3):
  website [scope:website, scope:website-e2e, type:app]
  cockpit [scope:cockpit, scope:cockpit-deploy-smoke, scope:cockpit-e2e, scope:cockpit-examples, type:app]
  scripts [scope:scripts-tests, type:tooling]
cockpit_e2e=true

2. A cap was only matchable by its Angular name

A cap is two independent nx projects — the Angular app that owns the e2e target and the python backend — with no edge between them. cockpit-matrix.mjs matched affected projects on cap.angular only, so a python-only cap change attributed nothing, which main() reads as "nx found no cap" and answers with the full fleet.

Measured, before → after

change lanes before lanes after
website-only (#932 base..head) 41 0 — the dispatcher's if: gate is now false
python-only cap change 40 1 — the cap that changed
libs/chat change 40 40 — unchanged, no coverage lost

(40 vs 41 is drift in the cap count since that run, not an effect of this change.)

Notes

  • The python sibling's nx name is read from its own project.json rather than derived from the path. A convention-derived guess would fail open to the full fleet without ever reporting that it guessed.
  • The emitted matrix JSON keeps its exact {angular, python} shape; pythonName is internal to attribution.
  • The shell still builds and tests on a website change — it consumes that code. Only the cap e2e matrix is descoped.

Tests

Both new spec groups were confirmed failing before the fix:

not ok 1 - apps/cockpit is not tagged scope:cockpit-e2e
not ok 2 - a website-only change leaves cockpit_e2e false
# SyntaxError: ... does not provide an export named 'isCapAffected'

After: node --test over all six suites 125/125, nx test scripts vitest 71/71 (baseline), nx lint scripts clean. No new spec file, so scripts/vite.config.mts exclude and the ci.yml node --test invocation need no change.

Left alone deliberately

scope:cockpit-examples and scope:cockpit-deploy-smoke on the shell are also broad, but those jobs are cheap and already self-scope (Cockpit — build all examples correctly logged "No affected cockpit angular projects" on #932). Separately: apps/cockpit has an e2e target that no workflow runs — worth a look, but not this PR.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
threadplane Canceled Canceled Sep 1, 2026 10:35pm UTC

Request Review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: this PR received an intelligent (AI) code review. See the review comments on this PR.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 0s —— View job


I'll analyze this and get back to you.

…changes

PR #932 changed three files under apps/website/src and ran 41 `Cockpit — e2e`
lanes. Two independent scoping defects stacked up.

1. apps/cockpit carried `scope:cockpit-e2e`. The cockpit-e2e matrix dispatches
   `nx e2e` for the standalone Angular cap apps under cockpit/**; none of the
   40 caps depends on the apps/cockpit Next.js shell, and no workflow runs the
   shell's own `e2e` target. The tag could never select real work — only
   over-select. And it did: apps/cockpit statically depends on apps/website, so
   a website-only PR made the shell nx-affected and flipped cockpit_e2e true.

2. cockpit-matrix matched affected projects on `cap.angular` only. A cap is two
   unlinked nx projects (Angular app + python backend), so a python-only cap
   change attributed nothing, which main() reads as "nx found no cap" and
   answers with the full fleet — 40 lanes to cover one cap.

Measured on this checkout, before → after:

  website-only (PR #932 base..head)   41 lanes → 0 (job gate is now false)
  python-only cap change              40 lanes → 1 (the cap that changed)
  libs/chat change                    40 lanes → 40 (unchanged; no coverage lost)

The python sibling's nx name is read from its own project.json rather than
derived from the path, because a wrong guess fails open to the full fleet
without ever reporting that it guessed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@blove
blove force-pushed the blove/ci-cockpit-e2e-scoping branch from 6bae069 to 753d40d Compare September 1, 2026 22:31
@blove
blove enabled auto-merge (squash) September 1, 2026 22:31
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 0s —— View job


I'll analyze this and get back to you.

@blove
blove merged commit 2ed9d11 into main Sep 1, 2026
28 checks passed
blove added a commit that referenced this pull request Sep 1, 2026
#921 added apps/cockpit/e2e/control-plane.spec.ts — 7 Playwright tests over
the operational sidebar: the real Angular handshake without blank or
unresponsive states, operational controls reachable at four viewports, forced
colors preserving control boundaries and keyboard focus, and reduced motion
disabling loader and drawer animation.

Nothing ever ran it. The suite sits behind `nx e2e cockpit`, and the
cockpit-e2e matrix only dispatches caps derived from walking cockpit/** —
apps/cockpit is never a candidate. No other job named the target either:
grepping every workflow for `nx e2e` returns examples-chat-angular,
examples-ag-ui-angular, the cap matrix, and website. So this has been dead
coverage for its whole life.

Verified it actually passes before wiring it up — 7/7 in 24s locally. It goes
in the existing `cockpit` job rather than a new one because it is cheap, it
reuses that job's `npm ci`, the `cockpit` scope already gates the shell, and
required-pr-checks already aggregates the job — so it becomes merge-blocking
with no gate change. The playwright config already sets retries: 2 on CI.

Found while auditing the tags I removed from apps/cockpit in #939.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
blove added a commit that referenced this pull request Sep 2, 2026
#921 added apps/cockpit/e2e/control-plane.spec.ts — 7 Playwright tests over
the operational sidebar: the real Angular handshake without blank or
unresponsive states, operational controls reachable at four viewports, forced
colors preserving control boundaries and keyboard focus, and reduced motion
disabling loader and drawer animation.

Nothing ever ran it. The suite sits behind `nx e2e cockpit`, and the
cockpit-e2e matrix only dispatches caps derived from walking cockpit/** —
apps/cockpit is never a candidate. No other job named the target either:
grepping every workflow for `nx e2e` returns examples-chat-angular,
examples-ag-ui-angular, the cap matrix, and website. So this has been dead
coverage for its whole life.

Verified it actually passes before wiring it up — 7/7 in 24s locally. It goes
in the existing `cockpit` job rather than a new one because it is cheap, it
reuses that job's `npm ci`, the `cockpit` scope already gates the shell, and
required-pr-checks already aggregates the job — so it becomes merge-blocking
with no gate change. The playwright config already sets retries: 2 on CI.

Found while auditing the tags I removed from apps/cockpit in #939.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
blove added a commit that referenced this pull request Sep 2, 2026
#921 added apps/cockpit/e2e/control-plane.spec.ts — 7 Playwright tests over
the operational sidebar: the real Angular handshake without blank or
unresponsive states, operational controls reachable at four viewports, forced
colors preserving control boundaries and keyboard focus, and reduced motion
disabling loader and drawer animation.

Nothing ever ran it. The suite sits behind `nx e2e cockpit`, and the
cockpit-e2e matrix only dispatches caps derived from walking cockpit/** —
apps/cockpit is never a candidate. No other job named the target either:
grepping every workflow for `nx e2e` returns examples-chat-angular,
examples-ag-ui-angular, the cap matrix, and website. So this has been dead
coverage for its whole life.

Verified it actually passes before wiring it up — 7/7 in 24s locally. It goes
in the existing `cockpit` job rather than a new one because it is cheap, it
reuses that job's `npm ci`, the `cockpit` scope already gates the shell, and
required-pr-checks already aggregates the job — so it becomes merge-blocking
with no gate change. The playwright config already sets retries: 2 on CI.

Found while auditing the tags I removed from apps/cockpit in #939.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
blove added a commit that referenced this pull request Sep 2, 2026
)

#921 added apps/cockpit/e2e/control-plane.spec.ts — 7 Playwright tests over
the operational sidebar: the real Angular handshake without blank or
unresponsive states, operational controls reachable at four viewports, forced
colors preserving control boundaries and keyboard focus, and reduced motion
disabling loader and drawer animation.

Nothing ever ran it. The suite sits behind `nx e2e cockpit`, and the
cockpit-e2e matrix only dispatches caps derived from walking cockpit/** —
apps/cockpit is never a candidate. No other job named the target either:
grepping every workflow for `nx e2e` returns examples-chat-angular,
examples-ag-ui-angular, the cap matrix, and website. So this has been dead
coverage for its whole life.

Verified it actually passes before wiring it up — 7/7 in 24s locally. It goes
in the existing `cockpit` job rather than a new one because it is cheap, it
reuses that job's `npm ci`, the `cockpit` scope already gates the shell, and
required-pr-checks already aggregates the job — so it becomes merge-blocking
with no gate change. The playwright config already sets retries: 2 on CI.

Found while auditing the tags I removed from apps/cockpit in #939.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant