Skip to content

[1/5][BREAKING][rollout][server] refactor: replace callbacks with leased long polling - #345

Draft
mathewjhan wants to merge 2 commits into
mainfrom
feat/long-polling-api
Draft

[1/5][BREAKING][rollout][server] refactor: replace callbacks with leased long polling#345
mathewjhan wants to merge 2 commits into
mainfrom
feat/long-polling-api

Conversation

@mathewjhan

@mathewjhan mathewjhan commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What

Replaces completion/grader callbacks with server-owned result futures and GET /rollout/{id}/result. Adds polling leases, result retention, and duplicate rollout ID rejection. Local and Harbor backends now return an ExecutionOutcome containing the workflow result and optional grader result.

Why

The caller can retrieve rollout outcomes without exposing callback endpoints. A polling lease also lets the server stop work when the caller stops polling.

Migration and dependencies

This is a breaking wire and backend API change: callback request/result models, ControllerAuth, controller_api_key, and GET /rollout/{id}/status are removed; callers must supply llm_api_key explicitly when the inference endpoint requires authentication. Custom backends must implement the new return-value contract, and grade controls whether grading runs.

This layer accepts a caller-provided polling lease; #346 changes admission to issue the lease on the server and adds the client. #347 migrates local eval. Stack: #345#346#347#348#349.

Existing TrainGate consumers still use callbacks. Osmosis-AI/osmosis-traingate#300 changes a MILES recipe, and Osmosis-AI/osmosis-traingate#301 adds recovery through the old status endpoint; neither migrates to this protocol. Rollout servers and their consumers need a coordinated migration before rollout.

How to Test

Run the lease, result registry, server admission, and Local/Harbor backend unit tests, then exercise admission → repeated polling → terminal result, lease expiry, and explicit cancellation.

Current CI is not green. Review of the complete stack also identified result serialization and workflow-error propagation issues in this layer; those fixes remain pending.

@mathewjhan mathewjhan changed the title feat/long polling api [feat] long polling api Sep 3, 2026
@mathewjhan mathewjhan changed the title [feat] long polling api [rollout] switch to long polling api Sep 4, 2026
@JoyboyBrian JoyboyBrian changed the title [rollout] switch to long polling api [1/5][BREAKING][rollout][server] refactor: replace callbacks with leased long polling Sep 5, 2026
@github-actions github-actions Bot added breaking Breaking changes refactor Code refactoring rollout Remote Rollout module server Rollout Server module stacked-pr Part of a stacked pull request series wip Work in progress (draft PR) labels Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaking changes refactor Code refactoring rollout Remote Rollout module server Rollout Server module stacked-pr Part of a stacked pull request series wip Work in progress (draft PR)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant