Skip to content

Collect grader data from usage artifacts - #502

Closed
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/update-graders-data-collection
Closed

Collect grader data from usage artifacts#502
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/update-graders-data-collection

Conversation

Copilot AI commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

The operational-value fallback collector downloaded the large agent artifact, although grader results and eval data are available in the smaller usage artifact.

  • Collection

    • Download usage for fallback grader collection.
    • Continue discovering graders/grader_results.json recursively.
  • Contract

    • Require usage and prevent regression to agent.
    • Document the artifact source.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants