feat(skills): add nemo-analyst workflow - #1116
Draft
callingmedic911 wants to merge 2 commits into
Draft
Conversation
Contributor
|
Add the platform-bundled nemo-analyst skill so coding agents have a reliable route to `nemo agents analyst run` and the minimum correct operating knowledge to use it. The body stays lean and defers depth to docs/agents/insight-driven-optimization.mdx rather than restating it. It covers the Insight shape, the credential distinction from the Experimentalist (the Analyst reads only INFERENCE_API_KEY), the doctor pre-flight, the run, where Insights are stored, and a verification step. Persistence is documented as it behaves after #1054: Insights always go to the platform, and --insights-file-output mirrors what was stored. Routing tests follow the explicit/implicit/negative-control convention already used by nemo-explore. Verified that all six analyst-intent prompts resolve to nemo-analyst, that the skill introduces no new failures elsewhere in the catalog, and that it wins no ties by sort order. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Aditya Pandey <aditya@autospace.co> Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Drove the skill end to end in a container against a real LangGraph agent's telemetry. It carried the workflow, so this sharpens what it left the driver to work out rather than adding to it — net zero lines, paid for by cutting redundancy from the storage and verify sections. - `analyst doctor` takes no `--agent` flag, so it always reports a red profile line. Say that passing `--agent` makes that line noise instead of leaving the reader to reason past a red mark, or to write an `optimizer.yaml` it does not need. - `agent_name` is carried on agent-level spans, not on their model and tool children, so the old "matches the `agent_name` on the spans" pointed at a query that undercounts. - Every span came back `status=success` while the agent was badly broken, so state that the Analyst judges behavior rather than status or scores. - Telemetry that captures only the shape of a run is a third cause of an empty result, alongside scoping and volume. Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
callingmedic911
force-pushed
the
aditya/ase-745-analyst-skill-b080
branch
from
August 6, 2026 22:06
519f9f1 to
e463b0b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the platform-bundled
nemo-analystskill (ASE-745), so coding agents have a reliable route tonemo agents analyst runplus the minimum correct operating knowledge to use it.packages/nemo_platform_ext/src/nemo_platform_ext/skills/nemo-analyst/SKILL.md(160 lines)packages/nemo_platform_ext/src/nemo_platform_ext/skills/nemo-analyst/tests.json(12 routing tests)No code changes.
Deliberately lean
docs/agents/insight-driven-optimization.mdxalready covers the data model, the Analyst's tool set, and the full loop in 638 lines, and ASE-758 will add a per-agent README on top of that. Rather than become a fourth surface restating the same ground, the skill covers only what an agent needs to execute correctly and links out for depth. It links the GitHub blob URL rather than the docs site, because that page is currently gated out oflatest.ymland would 404.Content is scoped to: the Insight shape, prerequisites, the
doctorpre-flight, the run, where Insights are stored, a verification step, and the two ways an empty result happens.Two things worth a reviewer's eye
Persistence reflects #1054, not the older behaviour. Insights now always go to the platform, and
--insights-file-outputmirrors what was stored with platform IDs included. There is no longer an implicit profile-based redirect to.nemo-optimizer/insights.yaml. Note that the troubleshooting section ofinsight-driven-optimization.mdxis still stale on this point (it says a discovered profile routes Insights to the file "instead of the Insights API"); left alone here as out of scope, but it should be corrected before that page is published.The credential contract differs from the Experimentalist's and the skill says so explicitly, since anyone arriving from
nemo-experimentalistwill assume otherwise. The Analyst reads onlyINFERENCE_API_KEYand does not use theNEMO_EXPERIMENTALIST_MODELS_*tiers.Validation
scripts/skill-test.py, measured against a baseline withnemo-analystremoved from the catalog:The catalog has 54 pre-existing failures and a dozen skills with no
tests.json; the bar here was adding none. The tie check matters becausebest_matchsorts on score alone and Python's sort is stable, so an alphabetically early skill silently wins ties. An earlier draft of the description tiednemo-status,nemo-intake, andnemo-model-selectionon generic words and stole their tests; the description was tightened to distinctive vocabulary until that reached zero.All six analyst-intent prompts that previously scattered across
nemo-explore,nemo-spec,nemo-experiments-upload,agents-secure, andskills-optimizationnow resolve tonemo-analyst:scripts/skill-cli-lint.py --root .reports zero violations for this skill. Everynemocommand in the body was checked against live--help; this caught one hallucination during authoring (nemo insights insights listdoes not exist), so verification now reads the Insights API directly. The 8 violations the linter does report are pre-existing ininferenceandnemo-customizer.nemo skills listandnemo skills show nemo-analystboth resolve it.pre-commit run --files <both files>passes and leaves them unmodified.Notes
nemo-nemo-analyst. Every installer builds its path asf"nemo-{skill_name}"unconditionally, whileskills-spec.mdxrequires user-invocable skills to already be namednemo-*. That is a pre-existing installer defect affecting 12 skills, being reported separately; following the spec here is the correct call and working around it would not be.nemo-skill-selection's hardcoded catalog table is not updated, matching feat(skills): add Experimentalist workflow #1088. If we want the router to know about the loop's skills, that is one change coveringnemo-analystandnemo-experimentalisttogether.not-fornamesnemo-experimentalist, which lands in feat(skills): add Experimentalist workflow #1088. The reference is inert until then and the linter does not require the sibling to exist.tests.jsonuses explicit/implicit/negative-control, matchingnemo-exploreand every other fixture in the repo. The spec doc's fourthcontextualmode is skipped because the doc andskills-spec.mdxdisagree withscripts/skill-test.pyon whether it should fire.Closes ASE-745.