Skip to content

Deep fragmented backlog wedges drain at ~40% of ceiling with no recovery (single-generation claim filter) #418

Description

@hardbyte

Measured magnitudes (orphan-swept cells, W=128, 4CPU-capped postgres): at 8,000/s offered (past the ~6.5k drain knee), v0.6.0 drains 7,199/s (backlog →231k, p99 ~18.6s) and the 0.7 ledger branch drains 7,641/s (backlog →115k, p99 ~7.9s). Drain stays near the healthy ceiling; the dominant overload symptom is unbounded backlog + multi-second latency (the #341 backpressure gap). The single-generation yield ceiling below is real (deterministic seeder, identical on both versions) but requires deeper fragmentation than these shapes produce — it is a latent ceiling that mainly slows recovery from extreme backlogs.

Found while gating the 0.7-alpha perf stack; reproduces on v0.6.0 — pre-existing, not caused by the 0.7 branches.

Symptom

Once offered load exceeds the drain ceiling long enough for the ready backlog to fragment across many sealed ring generations, drain rate collapses and NEVER recovers, even though workers are idle-capable and the backlog is large:

  • v0.6.0 @ 8,000/s offered, W=128 (4CPU-capped postgres): enqueue holds 7,998/s, backlog grows 77k→350k, late-window drain wedges at 3,358/s (healthy ceiling is >5k at this shape).
  • 0.7 ledger branch @ 5,000/s: same shape (its ceiling is lower, so it enters the regime at lower offered load; bisect evidence on PR perf(queue-storage): ring cursors via append-only rotation ledgers (#371) #415).

Mechanism (verified with a deterministic seeder + A/B on both versions)

claim_ready_runtime's row-claim CTE filters to exactly ONE (ready_slot, ready_generation) per call — a claim cannot span generation boundaries. v_claim_limit (up to 512) is computed from the lane's whole outstanding, but the generation filter truncates the actual yield to the current generation's slice of the lane. With a backlog interleaved across N generations, yield-per-call ≈ slice size (measured: identical ~10/call on v0.6.0 and the ledger branch with burst-10 seeding across 15 generations). Under load, rotation stalls once all slots hold undrained rows (busy-check), pinning the fragmentation in place — the system stays in the low-yield regime until the entire backlog drains at the wedged rate.

There may additionally be a zero-yield call-storm component (measured mean yield ~10/call implies many empty routing/probe calls when slice arithmetic predicts ~500-row slices) — needs distribution-level measurement, not just pgss means.

Why it matters

A transient overload burst (or a slow consumer deploy) that lets backlog fragment turns into a persistent ~60% throughput loss with no self-healing. Backpressure (#341) reduces entry probability but doesn't remove the cliff; adaptive claimers (#380) don't either (yield is per-call, not per-claimer).

Candidate fix

Let one claim call span multiple CONTIGUOUS generations of the same lane up to v_claim_limit — raises the wedge floor to the true batch cap. This is a claim-CTE redesign (routing + attempt/receipt batch writes are keyed by (slot, generation)) and needs TLA+ model updates (AwaSegmentedStorage claim actions) — not a contained patch.

Evidence: bench repo results/2026-07-11-awa-07-alpha-gate + wedge cells (wedge_old/wedge_new); reproduction seeder described in PR #415's investigation thread.

Milestone: 0.7 (correctness/operational).

Metadata

Metadata

Assignees

No one assigned

    Labels

    operationalOperational tooling and configuration

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions