Skip to content

Allow longer OpenAI Responses requests before timing out #1504

Description

@susano

In 0.32a3, OpenAI Responses SDK times out by default after 10 minutes and retries twice, leading to 30 minutes wait before time out error for long API calls (e.g. gpt-5.5-pro xhigh with max_tokens > 25,000).

Suggested fix is introducing --timeout and --max-retries model options for Response models. Example agent implementation that works for me is at 6519b1a

Details

I'm using llm 0.32a3 with an extra OpenAI model configured for gpt-5.5-pro:

- model_id: gpt-5.5-pro
  model_name: gpt-5.5-pro
  responses: true
  reasoning: true
  vision: true
  supports_tools: true
  supports_schema: true
  can_stream: false

I'm trying to run it with a template that gives it a large output budget:

model: gpt-5.5-pro
options:
  reasoning_effort: xhigh
  max_tokens: 25000

This currently works for me with max_tokens: 12500, but 25000 and above consistently time out after what seems to be exactly 30 minutes:

cat prompt | llm -t gpt-pro-max > out.pro
Error: Request timed out.

After adding model --timeout and --max-retries options (6519b1a) and changing the template to the following, I have had a successful 17 minutes command run.

model: gpt-5.5-pro
options:
  reasoning_effort: xhigh
  max_tokens: 50000
  max_retries: 0
  timeout: 7200 # two hours

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions