Part of #402 — hardening follow-up to the app-wiring PR. Pairs with the retry-wrapper issue.
Scope
Scheduler duty/slot subscribers ride a bounded tokio::sync::broadcast channel (buffer 100) and permanently terminate on RecvError::Lagged (crates/core/src/scheduler.rs:128-137,161-169). A subscriber stuck for ~25 slots (e.g. hung beacon calls at 4 broadcasts/slot) silently kills all duty dispatch for that subscriber until the node restarts.
This is a Pluto-only failure mode: Charon dispatches per-duty goroutines and cannot lag. It is made reachable by the current serial subscriber processing (see the retry-wrapper issue, which restores async dispatch); fix both directions — a lagged subscriber should skip missed messages and continue with a warning (or subscribers should be made non-blocking per duty), never stop permanently.
Dependencies
None (the retry-wrapper issue reduces the likelihood but does not remove the failure mode).
Acceptance
- A lagged subscriber logs a warning and continues receiving subsequent duties/slots.
- Regression test: a slow subscriber that lags the channel resumes processing instead of terminating.
Part of #402 — hardening follow-up to the app-wiring PR. Pairs with the retry-wrapper issue.
Scope
Scheduler duty/slot subscribers ride a bounded
tokio::sync::broadcastchannel (buffer 100) and permanently terminate onRecvError::Lagged(crates/core/src/scheduler.rs:128-137,161-169). A subscriber stuck for ~25 slots (e.g. hung beacon calls at 4 broadcasts/slot) silently kills all duty dispatch for that subscriber until the node restarts.This is a Pluto-only failure mode: Charon dispatches per-duty goroutines and cannot lag. It is made reachable by the current serial subscriber processing (see the retry-wrapper issue, which restores async dispatch); fix both directions — a lagged subscriber should skip missed messages and continue with a warning (or subscribers should be made non-blocking per duty), never stop permanently.
Dependencies
None (the retry-wrapper issue reduces the likelihood but does not remove the failure mode).
Acceptance