proposal_evaluation_correctness: judge responses wrapped in markdown code fences fail score parsing (Claude judges)

[harche]

Summary

custom:proposal_evaluation_correctness feeds the judge's raw response
directly to json.loads
(src/lightspeed_evaluation/core/metrics/custom/custom.py,
_parse_proposal_eval_response). Judges that wrap JSON in markdown code
fences — Claude models do this by default — produce:

ERROR: Could not parse score from LLM response: ```json
{ "reasoning": "Diagnosis correctly identified the liveness probe misconfiguration ...

so the metric is recorded as ERROR even though the judge produced a
perfectly valid, parseable verdict. Observed with judge
vertex_ai/claude-opus-4-6; any fenced-JSON judge hits it.

Potential fix

Strip markdown code fences before json.loads:

cleaned = response.strip()
if cleaned.startswith("```"):
    cleaned = re.sub(r"^```[a-zA-Z]*\s*|\s*```$", "", cleaned)
data = json.loads(cleaned)

Verified: the identical captured trajectory that scored ERROR parses to a
0.63 score after the change. Patch (with the rest of our local eval-harness
changes) is attached to the testing-report gist below
(lightspeed-evaluation-changes.patch).

Context

Found while validating the operator end-to-end against a self-hosted vLLM
endpoint (Gemma 4 31B) with the lightspeed-evaluation proposal suite, and
A/B-comparing against Claude Opus 4.6 on Vertex. Full testing report with
setup details and all findings:
https://gist.github.com/harche/1cb5e80a1156c75cad53b44fe3976099

cc @mrunalp