[cli-consistency] CLI Consistency Issues - 2026-08-13

[github-actions]

Summary

Inspection of the gh-aw CLI help output (/tmp/gh-aw/agent/all-help.txt, all 429 per-command help files) against docs/src/content/docs/setup/cli.md found 1 documentation/behavior mismatch (high severity). No typos, flag-naming inconsistencies, or other doc mismatches were found across the ~35 top-level and nested commands sampled (add, add-wizard, new, secrets, doctor, fix, compile, validate, lint, trial, run, disable, enable, audit, checks, experiments, forecast, health, list, logs, outcomes, status, completion, hash-frontmatter, mcp-server, pr/pr transfer, project/project new, remove, update, upgrade, deploy, env, domains, init).

Severity Breakdown

• High: 1
• Medium: 0
• Low: 0

Finding 1 (High): init docs overstate default behavior for MCP/custom-agent creation

Affected command: gh aw init

Doc text (docs/src/content/docs/setup/cli.md, #### init section):

Initialize repository for agentic workflows. Configures .gitattributes, creates the dispatcher skill file (.github/skills/agentic-workflows/SKILL.md), creates the Agentic Workflows custom agent (.github/agents/agentic-workflows.md), and performs non-interactive setup. Enables MCP server integration by default (use --no-mcp to skip).

This reads as if the custom agent (.github/agents/agentic-workflows.md) and MCP server integration are created unconditionally by default.

Actual CLI help (gh aw init --help, from /tmp/gh-aw/agent/help-output/init.txt):

With --engine copilot:
- Creates the custom agent at .github/agents/agentic-workflows.md

With --engine copilot (without --no-mcp):
- Creates .github/workflows/copilot-setup-steps.yml with gh-aw installation steps
- Creates .github/mcp.json with gh-aw MCP server configuration

This is corroborated by pkg/cli/init_command.go (lines ~33-46), where the custom-agent creation and MCP configuration are explicitly gated behind --engine copilot, with --no-mcp/--no-agent only relevant in that branch.

Expected vs actual:

• Expected (per docs): custom agent + MCP integration created by default for init, regardless of engine.
• Actual (per CLI help + source): custom agent + MCP integration are only created when --engine copilot is passed (or is the resolved default engine); with other engines (claude, codex, gemini, pi) these Copilot-specific artifacts are skipped entirely.

Suggested fix: Update the init section of docs/src/content/docs/setup/cli.md to clarify that the custom agent and MCP server integration files are only generated With --engine copilot (matching the CLI help text), and that --no-mcp/--no-agent only apply in that case. For example:

Initialize repository for agentic workflows. Configures .gitattributes, creates the dispatcher skill file (.github/skills/agentic-workflows/SKILL.md), and performs non-interactive setup. When using the Copilot engine (--engine copilot, the default engine), also creates the Agentic Workflows custom agent (.github/agents/agentic-workflows.md) and enables MCP server integration by default (use --no-mcp/--no-agent to skip either). Non-Copilot engines skip these Copilot-specific artifacts.

Priority: High — this is user-facing documentation that could lead users on non-Copilot engines to expect files that will not be created.

Inspection Metadata

• Source of truth: /tmp/gh-aw/agent/all-help.txt (22,567 lines) and /tmp/gh-aw/agent/help-output/*.txt (429 files), pre-collected by pre-agent-steps
• Comparison target: docs/src/content/docs/setup/cli.md (1,028 lines)
• Method: Read full CLI help tree, cross-referenced flag lists, examples, and descriptions against the docs page section-by-section (Setup, Building, Testing, Monitoring, Management, Advanced, Utility Commands). Verified suspicious claims (e.g., project new doc URL, audit --repo requirement, logs --cache-before/--after alias) directly against pkg/cli/*.go source where the help text alone was ambiguous.
• Date: 2026-08-13
• Run: github/gh-aw run 31705827863

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• githubnext.github.io

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "githubnext.github.io"

See Network Configuration for more information.

Generated by ✅ CLI Consistency Checker · auto · 85.6 AIC · ⌖ 2.29 AIC · ⊞ 6.7K · ◷

• expires on Aug 15, 2026, 5:48 AM UTC-08:00

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · gpt54 · 4.46 AIC · ⌖ 6.49 AIC · ⊞ 12.6K · ◷