fix(atproto): stop a failed enable from trapping crossposting on - #334
Conversation
enable_user_atproto writes the opt-in before the control plane is called, and a failed trigger only downgraded the state to 'failed' — the enabled flag stayed true. The account then reads as publishing with no repo behind it, and there is no way back out: disabling calls the control plane first, so the same outage that refused the enable also refuses the disable. Roll the flag back when the trigger fails, and let a disable finish locally when there is no DID to publish from. A provisioned account still fails loudly, because reporting "off" while a live repo can keep posting would be the worse lie.
The disable escape hatch gates on `atproto_did` being null to decide that an account has no repo to publish from. Every lifecycle write passed `None` for that column, and `set_atproto_state` writes it unconditionally — so the enable path itself erased the signal the gate depends on. An enable on an already-provisioned account (or a control- plane sync landing mid-write) left `did = NULL` with a live repo behind it, and the next disable would then take the unprovisioned path and report "off" while the gateway kept publishing. Carry the current DID through `enable_user_atproto`, `reenable_user_atproto`, and the rollback instead. A stale DID cannot authorize anything on its own — `ready_atproto_identity` also requires `state = 'ready'` — and provisioning overwrites it through the internal sync once the control plane reports back. Also drop the redundant existence query in the disable path: the state read that the DID gate needs already yields the same `UserNotFound`. Tests: a provisioned enable failure keeps its DID and still refuses the disable; the reenable rollback (previously untested) is pinned.
The settings card keyed its `failed` phase on `enabled && state === "failed"`, a shape the API no longer emits now that a failed enable rolls the opt-in back. Such an account fell through to `ready_to_enable`, which still offers the retry button but drops the "last provisioning attempt failed" explanation and the error text. Key the phase on the state alone, and move the e2e fixture to the shape the API now returns.
dcadenas
left a comment
There was a problem hiding this comment.
✅ The rollback and the DID-gated local release are the right shape, and the interleaving tests pin them properly.
Three gaps remained around the pre-trigger write. I pushed fixes rather than handing them back; detail is inline.
Changes I pushed
- The opt-in write no longer reads
atproto_didand writes it back — it omits the column, so there is no window for a provisioning sync to be overwritten. - A confirmed
readyaccount is now excluded from that write, so a redundant enable during an outage cannot report a publishing account as switched off. - A rollback that lands now revokes ATProto OAuth sessions, matching the invariant the local-release test states.
The ready exclusion is the widest of the three: enabling an already-ready account now returns current status without calling the control plane. pending stays eligible, so a stuck opt-in can still be retried.
Checks
cargo fmt --all -- --checkandcargo clippy --workspace --all-targets --all-features -- -D warnings -A deprecatedcargo test --workspace, plus theintegration-testssuites for core, api and signercargo test -p keycast_api --test atproto_http_test: 35 passedcargo build --release --workspace- CI is green on the pushed head.
The two behaviour changes are pinned by tests that fail when the fix is removed — I removed each one and confirmed only the matching test failed. The two DID tests are regression guards, not race reproductions; the window they close has no seam to interleave into, and the fix removes it structurally rather than detecting it.
enable_failure_keeps_the_did_so_disable_still_refuses now seeds pending with a DID instead of ready, because the ready exclusion short-circuits before the trigger runs. Its intent and assertions are unchanged.
Merge stays with you.
Folding the username resolution into reenable_user_atproto_with_trigger left reenable_user_atproto with no callers in src or tests, and as a verbatim copy of the first half of the function that replaced it. It is pub, so nothing warned about it. The two doc links that pointed at it now name what actually does the work: begin_atproto_opt_in for the write, and the trigger wrapper for the re-enable path.
Summary
enable_user_atproto_with_trigger/reenable_user_atproto_with_triggernow roll the opt-in back toenabled=falsewhen the control-plane trigger fails, instead of leavingenabled=true, state='failed'.disable_user_atproto_with_triggercompletes the local disable when the trigger fails and the account has no DID. A provisioned account (DID present) still returns the error, unchanged.atproto_didforward instead of nulling it, so the DID that gates the disable above survives a failed enable.enabled=true.set_user_atproto_crosspost.Motivation
enable_user_atprotopersistsenabled=true, state='pending'before it calls the control plane, and on trigger failure the old code wroteenabled=true, state='failed'. That leaves an opt-in that exists nowhere except that row — no gateway record, no repo, no DID — while/api/user/atproto/statusreports crossposting as on.The second half is what makes it a trap rather than a cosmetic wrong flag: the disable path calls the control plane first and only writes
disabledafter it succeeds. So while the control plane is unavailable, the same outage that refused the enable also refuses the disable, and the account stays switched on with no way for the user to switch it off.This is reproducible in production right now. In the mobile client (
divinevideo/divine-mobile, Settings → Bluesky Publishing) the toggle reads as on with status "Account provisioning failed", and turning it off returns the scoped 503 from #281 ("Bluesky publishing is temporarily unavailable"). Verified on a real Samsung Galaxy against productionlogin.divine.video; the handle it advertises (<username>.divine.video) resolves nowhere — no/.well-known/atproto-did, no_atprotoTXT record,Profile not foundfrom the public Bluesky API.Design notes for review:
internal_sync_atprotoand the state converges there — so the rollback is self-healing in that race, whereas today's stuck flag is not.disable_trigger_failure_preserves_existing_statetest, which stays green.set_atproto_statewritesatproto_didunconditionally, and every lifecycle write passedNone— so the enable path erased the exact signal the gate above depends on. An enable against an already-provisioned account, or a control-plane sync landing mid-write, leftdid=NULLwith a live repo behind it, and the next disable would then take the unprovisioned path. Carrying the DID through is safe: a stale DID authorizes nothing on its own (ready_atproto_identityalso requiresstate='ready'), and provisioning overwrites it through the internal sync.internal_sync_atprotocan reportenabled=trueand switch the account back on after the user turned it off — the mirror image of the convergence that makes the enable rollback safe. Recovering from that needs the gateway to be reachable again, which is the same condition that makes the normal disable work.DIVINE_SKY_ATPROTO_CONTROL_PLANE_URL(divine-handle-gateway) fail in production is an operational issue this PR does not touch — it only stops that outage from leaving accounts in an unrecoverable state. A durable retry queue for remote disables is likewise out of scope; the gateway remains the source of truth for the remote link.Related Issue
Testing
cargo test --workspace --verbose— not run locally: this machine has neither Docker nor a local Postgres, and the ATProto suites needcommon::setup_test_db. Please let CI confirm the five touched tests inapi/tests/atproto_http_test.rs.e2e/tests/bluesky-settings.spec.ts— not run locally (needs the Playwright stack); the fixture change is mock-only.cargo clippy --workspace --all-targets --all-features -- -D warnings -A deprecated— cleancargo fmt --all -- --check— cleancargo test -p keycast_api --lib— 190 passedcargo test -p keycast_api --test atproto_http_test --no-run— the new tests compiletest_suggest_users_for_admin_uses_the_email_trigram_index(keycast_core, untouched here). Re-running the identical commit passed; the plan assertion depends on ambient row counts and is tracked in test(core): email-trigram plan assertion depends on ambient row counts #335. Every test in this PR passed in both runs.Visuals