test: make ESM hot-reload integration tests immune to lost file-watch events - #1194
test: make ESM hot-reload integration tests immune to lost file-watch events#1194maartenbreddels wants to merge 2 commits into
Conversation
…events The single rewrite of the watched bundle races the file watch being armed, and watch events can be lost or coalesced on loaded CI runners: the retained CI video shows the page stuck at "version 1" for the full 30s wait, i.e. the reload never fired at all. Because pytest-retry reruns in the same process, one missed event also poisoned all retry attempts, turning a timing hiccup into a hard job failure (flaky on integration-test ubuntu/3.9/ipywidgets8 since 2026-07-22, with code, dependencies and runner image all unchanged). Writing a fresh version every few seconds until one propagates removes the sensitivity to a single lost event, while the 30s deadline still fails the test hard when hot reload is genuinely broken. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: text= is substring matching (version 2 would also match version 20), and a 3s window narrowed the tolerated reload latency from 30s to 3s on exactly the loaded runners this guards - a slow but working reload could keep just missing the sliding target. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Closing: this was the wrong fix. The retry loop hid a real bug instead of guarding against a flaky watcher. Root cause: a hot reload replaces the per-kernel ipyreact Fixed upstream in widgetti/ipyreact#82 (released as ipyreact 0.6.1); the original single-shot test is the correct regression guard for it and passes first try with that release. |
Why
test_ipyreact_module_hot_reload[flask-chromium]has been failing on the scheduled master runs ofintegration-test (ubuntu, 3.9, 8)since 2026-07-22, while code, resolved dependency versions (solara 1.60.3, ipyreact 0.6.0, playwright 1.50.0, watchdog 6.0.0, ...) and the runner image were all identical to the last green run. The same commit both passed and failed on different days, so this is a timing flake, not a regression.The retained CI video of a failing run shows the page rendering "version 1" and staying there for the entire wait: the rewrite of the watched bundle never produced a reload. The test does a single write of "version 2" and then waits 30s, which makes it sensitive to two races:
Because pytest-retry reruns in the same process, one missed event failed all 4 attempts the same way.
What
Instead of one write + 30s wait, the test now writes a fresh "version {n}" every ~3s and waits briefly for it to appear, with an overall 30s deadline. A single lost event no longer matters, but the test still fails hard when hot reload is genuinely broken (no rewrite ever reaches the browser).
Verified locally by delaying the
watcher.add_filearming by 5s so the first write's event is dropped: the old single-shot pattern times out, the retry loop recovers on a later write.tests/integration/esm_test.py(flask + starlette) passes 3/3 runs.🤖 Generated with Claude Code