fix(om2w-eval): actor-aware settle window — peerd's 31.0% was undercounted

[jonybur]

The finding

peerd's Online-Mind2Web score (31.0% full-300, PR #193) was undercounted by a harness measurement bug, not a capability gap.

The eval scores a task only once the whole flow goes silent for a quiet-settle window (added originally to stop scoring the delegation ack instead of the answer — see §6 of the web-thread doc). That window was a fixed 15s. It covered the orchestrator's inter-event gaps but not the web actor's own reply-synthesis: after the actor finishes navigating, it goes silent for one more model call (~20–40s on a slow model) before its reply-wake re-enters the orchestrator with the real answer. The window closed in that gap and captured the orchestrator's intermediate "On it, I'll report back" ack as the final answer — on ~40% of the full-300 failures.

The proof

• Re-ran 42 promissory-ack failures with the window widened: 34/37 went from an ack to a substantive answer (run-variance cannot turn 0 substantive answers into 34).
• Full 300, changing only the settle window: 33.1% → 41.1% (Claude judge, McNemar p=0.011) — the first statistically significant result in the whole thread. o4-mini-equivalent ≈ 38–39%, at parity with browser-use's verified ~40%.

This is a legitimate correction, not a thumb on the scale — a real user does get the answer; 15s was simply too short a silence to wait for an async browser round-trip.

The fix

Not a bigger fixed timeout (that taxes every task +45s). The settle is now actor-aware: hold the window open while a web-actor delegation is in flight (turn/actor-start..turn/actor-done), keep it tight (15s) otherwise. Fast tasks settle in 15s; async tasks wait for the actor, not a padding constant. Still bounded by the task's own timeoutMs if an actor never reports done. Validated: reproduces the same 34/37-class substance recovery as a blanket 60s, without the per-task tax.

Also in this PR

• --task-ids=<file> on run-om2w.mjs — run an arbitrary failure cluster instead of a contiguous offset/count shard (the tool that made the failure diagnosis tractable).
• Documents the whole finding (run-variance noise floor → diagnosis → measurement bug → the +8pp correction) as §5 of docs/benchmarks/2026-07-web-thread.md.

Notes

• Corrected numbers are from the internal Claude judge (validated at 83% agreement / +2.3pp leniency vs o4-mini); the o4-mini re-score of the corrected run is the one remaining step and doesn't change the mechanism or the significance.
• A separate, smaller product fix (the reintegration reply-wake being treated as untrusted-inbound, which blocks ~7 tasks from re-delegating) is scoped as its own change since it touches the sender gate.
• Gates: bun run typecheck, eslint, bun test ./tests (2827 pass) all green. The settle logic is browser-context; it's validated empirically via the ack-cluster re-run rather than a unit test.