Skip to content

test: make ESM hot-reload integration tests immune to lost file-watch events - #1194

Closed
maartenbreddels wants to merge 2 commits into
masterfrom
fix/harden-esm-hot-reload-test
Closed

test: make ESM hot-reload integration tests immune to lost file-watch events#1194
maartenbreddels wants to merge 2 commits into
masterfrom
fix/harden-esm-hot-reload-test

Conversation

@maartenbreddels

Copy link
Copy Markdown
Contributor

Why

test_ipyreact_module_hot_reload[flask-chromium] has been failing on the scheduled master runs of integration-test (ubuntu, 3.9, 8) since 2026-07-22, while code, resolved dependency versions (solara 1.60.3, ipyreact 0.6.0, playwright 1.50.0, watchdog 6.0.0, ...) and the runner image were all identical to the last green run. The same commit both passed and failed on different days, so this is a timing flake, not a regression.

The retained CI video of a failing run shows the page rendering "version 1" and staying there for the entire wait: the rewrite of the watched bundle never produced a reload. The test does a single write of "version 2" and then waits 30s, which makes it sensitive to two races:

  • the write can land before the file watch is fully armed
  • watch events can be lost or coalesced on loaded runners

Because pytest-retry reruns in the same process, one missed event failed all 4 attempts the same way.

What

Instead of one write + 30s wait, the test now writes a fresh "version {n}" every ~3s and waits briefly for it to appear, with an overall 30s deadline. A single lost event no longer matters, but the test still fails hard when hot reload is genuinely broken (no rewrite ever reaches the browser).

Verified locally by delaying the watcher.add_file arming by 5s so the first write's event is dropped: the old single-shot pattern times out, the retry loop recovers on a later write. tests/integration/esm_test.py (flask + starlette) passes 3/3 runs.

🤖 Generated with Claude Code

…events

The single rewrite of the watched bundle races the file watch being
armed, and watch events can be lost or coalesced on loaded CI runners:
the retained CI video shows the page stuck at "version 1" for the full
30s wait, i.e. the reload never fired at all. Because pytest-retry
reruns in the same process, one missed event also poisoned all retry
attempts, turning a timing hiccup into a hard job failure (flaky on
integration-test ubuntu/3.9/ipywidgets8 since 2026-07-22, with code,
dependencies and runner image all unchanged).

Writing a fresh version every few seconds until one propagates removes
the sensitivity to a single lost event, while the 30s deadline still
fails the test hard when hot reload is genuinely broken.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: text= is substring matching (version 2 would also
match version 20), and a 3s window narrowed the tolerated reload
latency from 30s to 3s on exactly the loaded runners this guards - a
slow but working reload could keep just missing the sliding target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@maartenbreddels
maartenbreddels temporarily deployed to fix/harden-esm-hot-reload-test - solara-stable PR #1194 July 28, 2026 13:33 — with Render Destroyed
@maartenbreddels

Copy link
Copy Markdown
Contributor Author

Closing: this was the wrong fix. The retry loop hid a real bug instead of guarding against a flaky watcher.

Root cause: a hot reload replaces the per-kernel ipyreact Module model (a trait update on a closed widget never reaches the browser), and until 0.6.1 a replacement model did not invalidate the page global module registry. Consumers rendered alongside the replacement could resolve the previous module and keep rendering it forever, so the update was lost permanently rather than delayed. Rewriting the file repeatedly just kept triggering reloads until one won the race.

Fixed upstream in widgetti/ipyreact#82 (released as ipyreact 0.6.1); the original single-shot test is the correct regression guard for it and passes first try with that release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant