fix(server): do not append [DONE] after an upstream in-band stream error

[enwaiax]

Problem

When an upstream emits an SSE error event mid-stream, /v1/chat/completions
forwards that error frame and then still appends data: [DONE] — the OpenAI Chat
success sentinel. An SDK client that stops at [DONE] treats the truncated,
failed turn as a normally completed answer.

Captured against the release binary (loopback upstream, one content frame followed
by one error frame):

data: {"choices":[{"delta":{"content":"partial",...}}],...}

data: {"error":{"code":"stream_failed","message":"upstream failed after stream start",...}}

data: [DONE]      <-- the defect

/v1/messages and /v1/responses are unaffected — neither uses a [DONE]
sentinel, and both already terminate on the error event.

Root cause

crates/switchyard-server/src/sse.rs tracks a local failed flag, but only a
framing/JSON error or a transport-level stream error sets it. An upstream in-band
error event is a well-formed JSON event, so it flows through frame_event() as
ordinary data and leaves failed == false:

if !failed && target_format == WireFormat::OpenAiChat {
    yield Ok(Event::default().data("[DONE]"));   // runs even for a failed stream
}

The translation layer already knows the stream failed: encode_stream() checks
state.errored and returns early, deliberately skipping codec.finish(). That
outcome was simply never communicated to the serving layer, which cannot otherwise
tell a clean EOF from an early stop.

This also matches the contract already stated in
crates/switchyard-translation/src/helpers.rs: "An in-band error is terminal for
every target format: the encoder emits the pre-error content and the error, then
drops any later chunk."

Fix

Expose the outcome across the layer boundary and consult it before emitting the
sentinel. [DONE] stays in the serving layer — framing is the serving layer's job
per the RawEventStream doc contract, and [DONE] is not a JSON event object, so
it does not belong in a codec.

• new StreamOutcome (a shared AtomicBool) set where encode_stream already
  returns early on state.errored
• new encode_stream_with_outcome(); encode_stream() keeps its signature and
  delegates to it, so existing callers (e.g. libsy-llm-client) are untouched
• frame_stream() skips the sentinel when the outcome reports an in-band error

Tests

Two new unit tests in sse.rs:

• upstream_in_band_error_suppresses_the_done_marker — the reported defect
• clean_stream_still_emits_the_done_marker — guards the OpenAI Chat contract for
  the success path

The pre-existing stream_error_terminates_without_done_marker still passes.

Verification

cargo fmt --check                                       -> clean
cargo clippy --workspace --all-targets -- -D warnings   -> 0 warnings
cargo test --workspace                                  -> 21 test binaries, 0 failed

End-to-end A/B against the same loopback upstream, same test, only the binary
swapped:

patched binary  -> saw_done=False   (no [DONE] after the error frame)
v0.2.0-rc2      -> saw_done=True    (defect reproduces)

Note

The Python switchyard serve entry point has the same defect
(switchyard/lib/endpoints/sse_helpers.py yields [DONE] unless an exception is
raised, and an upstream error event is not an exception). It has no equivalent
errored signal to read, so it needs its own fix rather than a port of this one —
not included here to keep this PR to a single component.

Summary by CodeRabbit

• Bug Fixes
  • Improved streaming error handling so failed in-band responses no longer include a misleading completion marker.
  • Preserved correct completion markers for successfully completed streams.
  • Continued suppressing completion markers when stream framing encounters an error.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5cc60e2a-8c05-40a5-b7e2-df4132c37eb7

📥 Commits

Reviewing files that changed from the base of the PR and between f30498d and b64e2e9.

📒 Files selected for processing (3)

• crates/switchyard-server/src/response.rs
• crates/switchyard-server/src/sse.rs
• crates/switchyard-translation/src/helpers.rs

---

Walkthrough

The translation encoder now reports whether an in-band error ended the stream. The server passes this outcome to SSE framing. OpenAI Chat framing suppresses [DONE] after upstream errors and retains it for clean streams.

Changes

Stream outcome propagation

Layer / File(s)	Summary
Encoder termination outcome crates/switchyard-translation/src/helpers.rs	Added StreamOutcome and encode_stream_with_outcome. The encoder sets the shared flag before returning after an in-band error. encode_stream remains a stream-only wrapper.
Response framing and validation crates/switchyard-server/src/response.rs, crates/switchyard-server/src/sse.rs	The response path passes the outcome to frame_stream. OpenAI Chat framing suppresses [DONE] after upstream errors. Tests cover failed and clean streams.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a rabbit hopping through the stream,
Where errors now report their state.
No [DONE] carrot follows failure,
Clean paths still arrive in great shape.
I twitch my nose and cheer the fix!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the primary change: suppressing [DONE] after an upstream in-band stream error.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}