Update TRL example paths after examples/ reorg

[sergiopaniego]

Follow-up to huggingface/trl#6820: TRL's examples/ was reorganized into self-contained per-example folders, so links to the old examples/scripts/... paths will 404 once it merges. This updates them to the new locations.

---

Note

Low Risk
Documentation-only URL and path updates with no runtime or security impact.

Overview
Updates documentation and tutorial links across OpenEnv so they point at TRL’s new per-example layout instead of the legacy examples/scripts/openenv/ and examples/notebooks/ paths (follow-up to TRL’s examples reorg).

Wordle GRPO: Colab badges and copy now reference examples/grpo_wordle/grpo_wordle.ipynb and examples/grpo_wordle/grpo_wordle.py in docs/source/tutorials/index.md, wordle-grpo.md, tutorial/04-training.md, and tutorial/examples/wordle.py (including the “copied from” URL and sample python invocations).

OpenCode async GRPO: opencode-agent-grpo.md links move to examples/async_grpo_opencode/async_grpo_opencode.py and opencode_hf_sandbox.py.

CARLA: examples/carla_env/README.md points fine-tuning readers to examples/grpo_carla/grpo_carla.py.

Rubrics tutorial: rubrics.md now describes TRL’s top-level examples/ tree and names example folders (grpo_wordle, grpo_catch, grpo_sudoku, …) instead of examples/scripts/openenv/.

^{Reviewed by Cursor Bugbot for commit d293789. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Alignment Review Report

Docs-only PR: updates TRL example links after the examples/ reorg in huggingface/trl#6820. 7 files, +11/−11, nothing under src/, no code logic. The new paths are the correct targets for the reorg — but that upstream PR has not merged yet, so every updated link currently 404s.

Automated Checks

• Lint: PASS (out of scope). .claude/hooks/lint.sh only lints src/ tests/ envs/; this PR touches docs/, tutorial/, and examples/carla_env/README.md — all Markdown, plus text inside a module-level string literal in tutorial/examples/wordle.py. No Python code changed, so there is no format/import-sort impact. (The hook also couldn't execute in this review env because uv isn't installed — an environment artifact, not a PR issue.)
• Debug code: CLEAN. .claude/hooks/check-debug.sh scans only src/, which this PR does not touch. The prints/TODOs it reports are all pre-existing in src/ and unrelated to this change.

Open RFCs Context

RFCs 000–003 and 005 are In Review, 010 is Draft, 004 (Rubrics) is shipped. Two touched docs sit in RFC areas, but only their external links change — no described design/behavior is altered, so no new RFC conflict is introduced:

• docs/source/tutorials/rubrics.md → RFC 004 (Rubrics)
• docs/source/tutorials/opencode-agent-grpo.md → RFC 005 (Agentic Harnesses). The standing RFC‑005‑vs‑shipped‑runtime reconciliation is unaffected by a link update.

Tier 1: Fixes Required

Broken links — the updated paths don't exist on trl@main yet (merge-ordering). Verified against the live GitHub API:

• TRL #6820 ("Reorganize examples into self-contained per-example folders", @qgallouedec) is OPEN / not merged (mergedAt: null, branch reorg-examples).
• New paths this PR adds → 404 on main: examples/grpo_wordle/, examples/async_grpo_opencode/, examples/grpo_carla/ (they exist only on the reorg-examples branch).
• Old paths this PR removes → still resolve on main: examples/scripts/openenv/{opencode,opencode_hf_sandbox,carla,wordle}.py and examples/notebooks/openenv_wordle_grpo.ipynb.
• Every changed reference targets trl/blob/main/… (or tree/main, refs/heads/main), so each 404s until #6820 lands.

Affected (will 404 until TRL #6820 merges):

• docs/source/tutorials/index.md:25 — Colab badge → examples/grpo_wordle/openenv_wordle_grpo.ipynb
• docs/source/tutorials/wordle-grpo.md:3 — Colab badge → examples/grpo_wordle/openenv_wordle_grpo.ipynb
• docs/source/tutorials/opencode-agent-grpo.md:59,61 — examples/async_grpo_opencode/{opencode,opencode_hf_sandbox}.py
• examples/carla_env/README.md:91 — examples/grpo_carla/carla.py
• tutorial/04-training.md:3 — Colab badge → examples/grpo_wordle/openenv_wordle_grpo.ipynb
• tutorial/examples/wordle.py — docstring "COPIED FROM" URL + 3 command refs → examples/grpo_wordle/wordle.py

Not affected: docs/source/tutorials/rubrics.md — its hyperlink moved from .../tree/main/examples/scripts/openenv to .../tree/main/examples (both resolve today); only the parenthetical folder names (grpo_wordle, grpo_catch, grpo_sudoku) are forward-looking prose.

Recommendation: land this PR at or after TRL #6820 (or merge them together). The paths are correct for the post-reorg layout; the only problem is timing. Per the two-tier model this is flagged rather than auto-fixed — merge ordering is a maintainer call, and I don't push fixes onto someone else's branch.

Tier 2: Alignment Discussion

Principle Conflicts

None identified. A docs-only link/path update touches no reset/step/state API, no MCP/WebSocket boundary, no in-env reward, no client/server import boundary, and no Pydantic wire type.

RFC Conflicts

None identified. The two RFC-adjacent files change only external example links; no design narrative is added or altered, so nothing new is canonized against an open RFC.

Summary

• 1 Tier‑1 correctness item — updated TRL links 404 until huggingface/trl#6820 merges; coordinate merge order.
• 0 principle conflicts.
• 0 RFC conflicts.

Suggested reviewers: @sergiopaniego (author, owns the reorg coordination) and @qgallouedec (TRL #6820 author); cc @burtenshaw (docs/tutorials owner).

  

_{Sent by Cursor Automation: Pre-review}

[cursor]

This Colab badge now points to examples/grpo_wordle/openenv_wordle_grpo.ipynb on trl@main, which 404s today — that path exists only on TRL #6820 (reorg-examples), which is still open. The current examples/notebooks/openenv_wordle_grpo.ipynb still resolves. Recommend landing this at/after TRL #6820 so the published tutorial link isn't broken in the interim. (Same applies to wordle-grpo.md:3 and tutorial/04-training.md:3.)

[cursor]

Both examples/async_grpo_opencode/opencode.py and …/opencode_hf_sandbox.py 404 on trl@main — they exist only on TRL #6820 (still open). The old examples/scripts/openenv/opencode*.py paths still resolve on main. Merge-order this after #6820 (or together) to avoid broken links.

[cursor]

examples/grpo_carla/carla.py 404s on trl@main (only present on TRL #6820, still open). The old examples/scripts/openenv/carla.py still resolves. Same merge-ordering note as the tutorials.

[qgallouedec]

Thanks! One catch: huggingface/trl#6820 also renames each example's main file after its folder, so 4 of the new links still 404 on the reorg branch:

• examples/grpo_wordle/openenv_wordle_grpo.ipynb -> examples/grpo_wordle/grpo_wordle.ipynb (docs/source/tutorials/index.md, wordle-grpo.md, tutorial/04-training.md)
• examples/grpo_wordle/wordle.py -> examples/grpo_wordle/grpo_wordle.py (tutorial/examples/wordle.py, 4 spots)
• examples/async_grpo_opencode/opencode.py -> examples/async_grpo_opencode/async_grpo_opencode.py (docs/source/tutorials/opencode-agent-grpo.md)
• examples/grpo_carla/carla.py -> examples/grpo_carla/grpo_carla.py (examples/carla_env/README.md)

The other two (opencode_hf_sandbox.py and the examples/ dir link) are correct, I validated against the reorg-examples branch.