feat: GEPA-style PR reflection feedback loop — increment 1 of #1584 - #1739
Open
Lexus2016 wants to merge 4 commits into
Open
feat: GEPA-style PR reflection feedback loop — increment 1 of #1584#1739Lexus2016 wants to merge 4 commits into
Lexus2016 wants to merge 4 commits into
Conversation
Add evolution_pr_reflection.py — a deterministic script that mines recently closed PRs (merged AND rejected), classifies outcomes, extracts repeat- rejection patterns, and writes a pr-reflection.txt sidecar. The analysis skill reads this sidecar (new step 6d) to calibrate selection based on actual merge/reject signals — a GEPA-style reflect-and-rewrite loop. - scripts/evolution_pr_reflection.py: PR mining + reflection + sidecar writer - tests/scripts/test_evolution_pr_reflection.py: 19 unit tests (all passing) - cron/evolution/pr-reflection.yaml: daily cron job (07:35, no_agent) - skills/evolution/evolution-analysis/SKILL.md: new step 6d reading sidecar This is the smallest coherent increment of #1584 — NOT the full multi- optimizer, just the feedback-collection + reflection loop. Co-Authored-By: Hermes Evolution <evolution@hermes.ai>
3 tasks
This was referenced Aug 6, 2026
This was referenced Aug 6, 2026
Contributor
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First coherent slice of #1584: GEPA-style reflective prompt evolution for the evolution pipeline.
What this delivers
A deterministic feedback loop that mines recently closed PRs (merged AND rejected), extracts actionable patterns from reject reasons, and writes a
pr-reflection.txtsidecar that the analysis stage reads to calibrate its next selection.Files
scripts/evolution_pr_reflection.py— PR mining + classification + reflection + sidecar writer (273 lines)tests/scripts/test_evolution_pr_reflection.py— 19 unit tests, all passingcron/evolution/pr-reflection.yaml— daily cron job (07:35, no_agent)skills/evolution/evolution-analysis/SKILL.md— new step 6d: readpr-reflection.txtsidecarHow it works
gh pr list --state closedqueries last 7 days of PRs[evolution-pr-reflection] closed=N merged=N rejected=N merge_rate=N% ...lineGEPA principle (arXiv:2507.19457)
"Read execution trace + metric feedback, reflect on failures, keep Pareto frontier."
This is the SMALLEST coherent increment — NOT the full multi-optimizer with DSPy.
Just the feedback-collection + reflection sidecar that closes the loop.
Checks
ruff check✓ (all checks passed)ruff format --check✓ (new files formatted)pytest tests/scripts/test_evolution_pr_reflection.py✓ (19/19 passed)evolution_skill_lint --skill-edit-budget✓ (within budget)Size note
517 lines total (273 script + 206 tests + 24 cron + 14 skill). Exceeds the 200-line
self-merge cap — this is a coherent slice that cannot be meaningfully split smaller
(script + tests are atomic). The merge gate will hold for human review.
Deferred (next increment)