Skip to content

[DO NOT MERGE yet] Update wordle tutorial to native vllm serve - #1082

Draft
sergiopaniego wants to merge 1 commit into
mainfrom
update-wordle-vllm-serve
Draft

[DO NOT MERGE yet] Update wordle tutorial to native vllm serve#1082
sergiopaniego wants to merge 1 commit into
mainfrom
update-wordle-vllm-serve

Conversation

@sergiopaniego

Copy link
Copy Markdown
Member

What

Updates the vLLM server command in tutorial/examples/wordle.py from the deprecated trl vllm-serve to vLLM's native server, matching the migration in huggingface/trl#6765:

CUDA_VISIBLE_DEVICES=0 VLLM_SERVER_DEV_MODE=1 vllm serve Qwen/Qwen3-1.7B --host 0.0.0.0 --port 8000 \
    --weight-transfer-config '{"backend": "nccl"}' \
    --logprobs-mode processed_logprobs \
    --max-logprobs -1

Do not merge yet

The tutorial's uv header installs trl[vllm] from PyPI. The released TRL client (v1.10.0) still speaks the old TRL server protocol, so this command only works once:

  1. Use vLLM's own server instead of TRL's trl#6765 is merged, and
  2. a TRL release containing it is out.

Until then, the current trl vllm-serve command in the tutorial remains the correct one (and it keeps working after the release too, as a deprecated wrapper until TRL v2.0). Will mark ready for review once the TRL release ships.

AI-assisted PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant