Skip to content

fix: ends study if the API reaches a timeout in turn-based mode - #1187

Open
jacyanthis wants to merge 4 commits into
PAIR-code:mainfrom
jacyanthis:turn-based-api-timeout-fallback
Open

fix: ends study if the API reaches a timeout in turn-based mode#1187
jacyanthis wants to merge 4 commits into
PAIR-code:mainfrom
jacyanthis:turn-based-api-timeout-fallback

Conversation

@jacyanthis

Copy link
Copy Markdown
Contributor

Description

In turn-based mode, errors can make it impossible to move forward in a study when it is the AI agent's turn to speak, such as if the API fails to respond after several retries. This fix PR provides two ways to handle these:

  1. the AI agent can produce a hard-coded message, such as Error: Could not generate a message at this time. Please try again. and advance the turn or
  2. the human participant can be shown a pop-up ending the study with information, such as telling them that there has been a technical error so they should exit the study.

By default, the agent sends the error message and advances the turn if there are errors after failing to produce a valid message for 120 seconds of retries. This can be changed from an error message to a neutral response (e.g., Not sure yet.) with useNeutralTimeoutResponses.

By default, if this turn advance occurs a set number of times (default: 2; null for never), the pop-up is shown instead. In particular cases where the error in the API pipeline is not going to be resolved with retries (e.g., invalid API key), the pop-up is shown immediately.

This has no effect on non-turn-based functionality.

Manual testing as per template

Set up an experiment with (i) a turn-based group chat stage with an agent mediator and a human participant and (ii) disconnect the internet or fail to insert a valid API key.

  • 1 human participant needed
  • No agent participants needed
  • Expected behavior: The timeout error with retries can reliably show if you disconnect from the internet, and the system will retry as if you might just be having a quick drop in connection. If the API key is invalid, then when the participant enters the chat stage, the pop-up will immediately appear, instructing the participant to exit the study because an invalid API key can’t be fixed with retries.

Screenshot

image

@jacyanthis
jacyanthis force-pushed the turn-based-api-timeout-fallback branch 2 times, most recently from 771f8df to e1ee621 Compare July 24, 2026 06:15
# Conflicts:
#	docs/assets/api/schemas.json
#	scripts/deliberate_lab/types.py
#	utils/src/stages/chat_stage.ts
#	utils/src/stages/chat_stage.validation.ts
@jacyanthis
jacyanthis force-pushed the turn-based-api-timeout-fallback branch from e1ee621 to c3db03e Compare July 24, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant