Skip to content

fix(cli): make --llms-full complete and accurate - #474

Open
mldangelo-oai wants to merge 21 commits into
mainfrom
mdangelo/codex/fix-llms-full-manifest
Open

fix(cli): make --llms-full complete and accurate#474
mldangelo-oai wants to merge 21 commits into
mainfrom
mdangelo/codex/fix-llms-full-manifest

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make --llms-full a complete reference for the installed CLI. The reference uses the live command schemas and documents the options, constraints, and output modes that the CLI supports.

Changes

  • Render canonical kebab-case flags, required arguments and options, allowed values, defaults, constraints, and deprecated aliases.
  • Reuse the packaged README for the operating guide and include current runtime metadata, without duplicating the guide in scoped command output.
  • Share command-result restrictions with CLI enforcement while preserving discovery, structured manifests, and shell completion.
  • Preserve the schema-based handling of optional issue arguments and project selectors, with generic contract tests for requiredness and deprecation.
  • Merge current main and keep both the CLI manifest and shared Codex prompt module in the package contract.

Testing

Current head 025bf950:

  • Focused manifest, CLI, and workbench-command tests: 172 passed, zero failures.
  • Default and randomized seed-12345 full suites: 1,356 passed, 14 skipped, zero failures each.
  • Types, formatting, build, and diff checks: passed.
  • Packed and checked the actual npm artifact: 230 entries, installed CLI/SDK, 106 bundled plugin files, and a nested worker without a global Codex installation. A minimal strict NodeNext TypeScript consumer also passed.
  • Three fresh native reviews and independent verification: clean.

Earlier full-suite, package, and review results remain historical evidence, not results for this merge.

Risk and rollout

This keeps Incur's live parser schemas and the existing command-result policy. There is no dependency, persisted-state, or bundled-plugin-version change. Current-head cross-platform CI, required human review, and disposition of older source-fixed review threads remain merge conditions. This update does not merge or release anything.

Public disclosure review

Newly authored material and the current public PR surface were reviewed for disclosure. Existing automated review comments contain access-restricted report references; maintainer cleanup remains outstanding.

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the bug Something isn't working label Aug 16, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5789da8956

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 5789da8956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai
mldangelo-oai marked this pull request as ready for review August 16, 2026 03:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5789da8956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/tests-ts/cli-manifest.test.ts Outdated

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: d2e63a0a1d

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2e63a0a1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +45 to +46
INCUR_VALUE_OPTIONS.has(argument) &&
argv[index + 1] !== undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Strip equals-form global options before deriving scope

When an accepted equals-form global option precedes the command path, such as codex-security --filter-output=scan scans show --llms-full, this exact lookup does not consume it. renderFullMarkdownManifest consequently treats the option as the first command argument and leaves the scope empty, producing a root heading and root operating guide even though Incur selected only scans show; handle equals forms for --filter-output, --token-limit, and --token-offset before deriving the scope.

Useful? React with 👍 / 👎.

Comment on lines +173 to +177
if (options && property.type === "array") {
details.push("Repeat this flag for multiple values.");
if (Array.isArray(property.default)) {
details.push(`Default: ${codeValue(property.default)}.`);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document constraints on repeatable option values

Every repeatable option such as --path, --knowledge-base, and --codex is an array whose string items have a minimum length, but Zod places that constraint under property.items and this branch only describes repetition and the outer default. Because Incur does not render these constraints itself, the full manifest omits that empty repeated values are rejected; include applicable item-level constraints when documenting arrays.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 59ae515c1f81f982a0edeaaa0b07574653e899db.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 59ae515c1f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59ae515c1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

{ commands: [{ name: command }] },
[command],
);
expect(scoped).toMatch(/--filter-output.*only to command results/u);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Test filtering semantics without pinning renderer prose

The latest commit reintroduces an assertion on exact generated English by requiring the words “only to command results,” so a harmless copy edit will fail this test even when filtering behavior and documented restrictions remain correct. Assert a stable semantic property or source-of-truth value instead; this is fresh evidence after the earlier wording assertions were removed.

AGENTS.md reference: sdk/typescript/AGENTS.md:L29-L32

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, beedc8eee5352c29787ba0882c2564a75ad1277e.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: beedc8eee5

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: beedc8eee5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

zcrab-oai
zcrab-oai previously approved these changes Aug 16, 2026

@zcrab-oai zcrab-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the generated CLI reference against actual command schemas; verified scoped equals-form options and repeatable-value constraints.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 5f8ccba09350c854495abfda2659d04fcba1c7a4.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5f8ccba093

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 5f8ccba093

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 025bf950a16979dba72d880034019a29b95b3ad7, including the current-main package-contract merge. The normal and seed-12345 full suites each passed 1,356 tests with 14 skipped. Types, formatting, the installed-package smoke test, and a strict NodeNext consumer passed. Three fresh native review passes and an independent verifier are clean. Older source-addressed review threads remain open for maintainer disposition.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 025bf950a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator Author

@codex security review

Please review the current head 025bf950a16979dba72d880034019a29b95b3ad7.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 025bf950a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +315 to +318
...commands.map((command) => {
const restrictions = commandResultRestrictions(command.name ?? "");
return [
Skill.generate(cli.name, [command]).replace(/^#/gmu, "###"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include patch option dependencies in the reference

The generated patch --llms-full reference lists these options independently, but the live handler in cli.ts rejects combining --linear-issue with --linear-project, requires --linear-project for --linear-filter, and requires either selector for --linear-api-key. Because patch has no hint containing those rules and scoped output omits the README operating guide, an agent following this reference can construct commands that are guaranteed to fail; derive these restrictions from the runtime rules or add shared command guidance.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the exact head 025bf950 against main 37bf87a6. The patch option checks on main and the missing command hint are unchanged by this PR. The new reference uses the live command schemas and hints; scoped output deliberately points to the root operating guide.

A shared runtime/reference hint could improve discoverability, but the independent source review found no introduced regression or violation of the schema-derived reference contract. I have left this as a follow-up suggestion rather than changing the command-validation model during the CI/conflict repair. Leaving the thread open for maintainer disposition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants