Ordered tool-permission policy (glob rules via commons.MatchItems)

[moshloop]

Ordered tool-permission policy (glob rules via commons.MatchItems)

Replace the four uncoordinated chat tool-permission mechanisms (toolGroupDefaultPermission, clicky verb-default annotation stamping,
xeroChatToolPermission, applyChatToolMetadata) with one ordered rule list evaluated last-match-wins ("later overrides earlier"), where each rule
matches on name / group / parent / verb / method / scope / hints using commons/collections.MatchItems glob patterns (! negation, comma-separated,
case-insensitive * prefix/suffix/contains).

Layer order (weakest → strongest): group baselines → verb defaults → hint/method defaults → app rules (xero-cli) → surface rules (.prompt frontmatter
toolPolicy:) → user rules (UI; a group toggle emits a group rule, a per-tool toggle emits a name rule appended after).

Motivated by the adversarial review of the Ask AI surfaces: group baselines like provider.xero.read: off never reach the execution path (clicky stamps
list/get → on into the same DefaultPermission slot), the OpenAPI catalog and the chat executor can disagree, and preference matching is exact-string
only in captain, clicky-ui, and the prompts.

Repos touched (dependency order): captain → clicky → xero-cli (Go) → clicky-ui → xero-cli webapp. The full step-by-step design is in the attached plan.

Acceptance Criteria

• captain pkg/api defines ToolMatch/PermissionRule/PermissionPolicy with last-match-wins Resolve, Validate (rejects empty match and bad
  modes), and FromPreferences (sorted keys, group rules before name rules); Spec.ToolPolicy round-trips through specMarshal/marshalValue
• ResolveDefinitions applies FromPreferences(prefs) ++ policy and all 6 call sites pass it (aichat service, approval resume, codex, claudeagent,
  genkit, callertools) — .prompt frontmatter toolPolicy: reaches non-chat agent runs
• clicky no longer stamps verb defaults into clicky/tool-default-permission; the annotation carries only explicit registrations, and list/get tools
  still resolve On via inferred ReadOnlyHint
• xero-cli builds one layered PermissionPolicy (group baselines, verb defaults incl. read/show/lookup/search/overview, hint/method defaults, app
  rules) applied identically by the chat Permission callback and applyChatToolMetadata — execution and OpenAPI catalog agree for every exposed tool
• provider.xero.read → off actually hides live Xero list/get tools from chat (the review's F1 defect), verified by a Go test
• all 10 surface prompts declare ordered toolPolicy: lists; the toolPreferences: map is gone; the empty groups (accounting.transactions.write,
  provider.takealot.write, comments.read) and no-op default restatements are removed; admin.*/provider.* restrictions added per surface
• clicky-ui resolves [...surfaceRules, ...userRules] with a TS MatchItems port, sends toolPolicy instead of the flattened per-name map, group
  toggle emits a group rule, per-tool toggle appends a name rule that beats it; stored prefs migrated (TOOL_TIER_VERSION bump in webapp)
• lint, build, and focused tests green in captain, clicky, xero-cli, clicky-ui packages/ui, and xero-cli webapp

Plan

Ordered tool-permission policy (glob rules via commons.MatchItems)

Context

The adversarial review of the Ask AI chat surfaces found the tool-permission model is split across four
uncoordinated mechanisms — toolGroupDefaultPermission, clicky's verb-default annotation stamping,
xeroChatToolPermission, and applyChatToolMetadata — with the result that group baselines like
provider.xero.read: off never reach the execution path (clicky stamps list/get → on into the same
DefaultPermission slot), the OpenAPI catalog and the chat executor can disagree, and preference keys are
exact-string only (captain EffectivePreference, clicky-ui effectiveToolPreferences).

This change replaces all of it with one ordered rule list, evaluated last-match-wins ("later overrides
earlier", per user decision), where each rule matches on name / group / parent / verb / method / scope /
hints using commons/collections.MatchItems glob patterns (! negation, comma-separated, case-insensitive
* prefix/suffix/contains). Layer order (weakest → strongest):

1. group baselines → 2. verb defaults → 3. hint/method defaults → 4. app rules (xero-cli) →
2. surface rules (.prompt frontmatter) → 6. user rules (UI; a group toggle emits a group rule that
   applies to everything in the group, a per-tool toggle emits a name rule appended after).

Layers 1–4 resolve server-side in xero-cli and are stamped as each definition's DefaultPermission —
identically for the chat executor and the OpenAPI catalog, killing the divergence. Layers 5–6 travel from
the webapp as an ordered toolPolicy list on the chat request and are applied by captain's
ResolveDefinitions on top. Prompt frontmatter migrates from the toolPreferences: map to an ordered
toolPolicy: list with the review cleanup (drop the three empty groups, drop no-op restatements, add
the missing admin.*/provider.* restrictions).

Core types (captain, new pkg/api/toolpolicy.go)

type ToolMatch struct { // every string field is a MatchItems pattern list; empty = unconstrained
    Name, Group, Parent, Verb, Method, Scope string
    ReadOnly, Destructive, Idempotent *bool  // exact match vs hints; nil = any
}
type PermissionRule struct { Match ToolMatch; Mode ToolMode }
type PermissionPolicy []PermissionRule

• AND semantics across non-empty selectors; selector strings evaluated with
  collections.MatchItems (commons/collections/slice.go:170; captain already imports commons —
  pkg/claude/tooluse.go:616). Verb/Method/Scope match ToolInfo.Annotations["clicky/verb|method|scope"].
• Resolve(info): iterate the whole list; last matching rule wins; rules with Mode: auto are skipped
  (no opinion). No match → fall through to the definition's DefaultPermission, else ask.
• Validate(): reject a fully-empty Match (catch-all must be explicit: {name: "*"}), reject modes
  that fail NormalizeToolMode. Explicit camelCase json/yaml tags on every field (frontmatter decode
  uses yaml KnownFields(true) — untagged fields fail loud, which is what we want).
• FromPreferences(ToolPreferences): legacy map → rules, deterministic via existing sortedKeys
  (api/tooldef.go): all group-keyed rules first, then name-keyed rules (preserves name-beats-group).
• Alias the types in pkg/ai/tools/policy.go (same pattern as ToolPreferences).

Implementation steps (dependency order — captain → clicky → xero-cli Go → clicky-ui → webapp)

Step 1 — captain

Files: pkg/api/toolpolicy.go (new), pkg/api/spec.go, pkg/aichat/wire.go, pkg/aichat/messages.go,
pkg/ai/tools/tools.go (+ new pkg/ai/tools/policy.go).

1. Add the types above.
2. Spec.ToolPolicy PermissionPolicy — must also be added to specMarshal + marshalValue()
   (spec.go:47-61, 120-143) or it is silently dropped from the persisted execution spec, breaking the
   approval-resume path (approval_execution.go:26 replays the persisted spec). Spec.Validate() calls
   ToolPolicy.Validate(). Spec.Merge keeps replace-wholesale slice semantics (document it — server
   layers are already baked into definitions, so no concat is needed at merge time).
3. ChatRequest.ToolPolicy in wire.go:25; requestSpec (messages.go:79) passes it through.
4. ResolveDefinitions → ResolveDefinitions(definitions, ResolveOptions{Preferences, Policy}).
   Effective policy = FromPreferences(Preferences) ++ Policy (later wins). Keep the existing order:
   validate names/handlers/duplicates → apply policy → drop off → auto → hint fallback. Update all
   6 call sites: aichat/service.go:196, aichat/approval_execution.go:26,
   ai/provider/codex_appserver.go:392, ai/provider/claudeagent/caller_tools.go:30,
   ai/provider/genkit/tools.go:53, ai/callertools/runtime.go:41,88 — so non-chat agent runs honor
   toolPolicy: in .prompt frontmatter too. Keep EffectivePreference-based helpers
   (WithRuntime/ShouldRequireApproval, tools.go:153-196) consistent by routing them through the same
   resolution.

Step 2 — clicky

1. Remove the verb-default stamping at entity/annotations.go:200-202 (delete
   verbDefaultToolPermission): the clicky/tool-default-permission annotation now carries explicit
   registrations only; verb defaults become policy layer 2. Explicit withToolPermission /
   WithToolPermission paths unchanged.
2. Confirmed safe fallbacks: aichat/tools_clicky.go defaultToolPermission still yields On for
   list/get via EffectiveToolHints' inferred ReadOnlyHint; entity/entity.go:1077 promoted-root copy
   becomes a no-op; rpc/converter.go:187-190 still tags promoted roots verb=list.
3. Accepted behavior change to verify against other consumers: MCP _meta["defaultPermission"]
   (mcp/registry.go:144-166) disappears for verb-defaulted tools — check no sibling app (oipa-cli, api3)
   reads it before relying on inference.

Step 3 — xero-cli server

Files: pkg/commands/chat_tool_groups.go, chat.go, chat_tool_openapi.go, chat_tools.go,
pkg/chatprompts/* + prompts/*.prompt.

1. Replace toolGroupDefaultPermission + xeroChatToolPermission with one ordered
   xeroChatToolPolicy api.PermissionPolicy (layers 1–4 concatenated):

   • layer 1: the current group table as {group: G} → mode rules (incl. provider.xero.read/write → off,
     provider.takealot.write → off);
   • layer 2: {verb: "list,get"} → on, {verb: "create,update,delete"} → ask, plus xero's wider
     read-verb set {verb: "read,show,lookup,search,overview"} → on (today handled by
     isReadOnlyChatVerb; without this rule those tools regress to ask);
   • layer 3: {readOnly: true} → on, {method: "GET,HEAD,OPTIONS"} → on,
     {method: "POST,PUT,PATCH,DELETE"} → ask;
   • layer 4: app rules (this is where future tightening like {name: "sync"} → off lands).

2. One shared resolver applies that policy to a ToolInfo and is used by BOTH paths:
   the CobraToolProviderOptions.Permission callback (chat.go:29) and applyChatToolMetadata
   (chat_tool_openapi.go) — each stamps the resolved mode as the definition's/operation's
   defaultPermission. withToolGroup (chat_tool_groups.go:68) stops stamping
   DefaultPermission (keeps Group/Parent only), or layer 1 stays baked in as pseudo-explicit.
   Keep xeroChatToolEnabled's disabled-group filter as a filter (an off rule could be overridden by
   a later user rule; filtered tools must not be). Audit the 7 static tools in chat_tools.go — their
   explicit DefaultPermission remains the fall-through when no rule matches; ensure their
   clicky/verb|method annotations are complete enough for the layer rules that should reach them.

3. chatprompts: PagePrompt.ToolPolicy (from the rendered spec's ToolPolicy); delete
   PagePrompt.ToolPreferences. Rewrite the 10 surface prompt frontmatters from the toolPreferences:
   map to short ordered toolPolicy: lists with cleanup. Pattern (editor surfaces):

   toolPolicy:
     - match: { group: "admin.*" }
       mode: "off"
     - match: { group: "provider.*" }
       mode: "off"
     - match: { group: "accounting.metadata.write" }
       mode: "off"

   Per-surface deltas: render views (template-render, document-render) add
   {group: "templates.write"} → off and keep {group: "comments.write"} → ask;
   transaction-detail re-grants {group: "accounting.metadata.write"} → ask and
   {group: "accounting.transactions.write"} is dropped (empty group); the three empty groups
   (accounting.transactions.write, provider.takealot.write, comments.read) disappear everywhere;
   no-op restatements of on/ask defaults are dropped. markdown-cell and global stay policy-free.

4. Tests: extend chat_tool_groups_test.go so every rule's group selector matches ≥1 real group and —
   the new invariant — the execution path and the OpenAPI catalog resolve identical modes for every
   exposed tool.

Step 4 — clicky-ui

Files under packages/ui/src/data/ai/: new policy.ts, ChatWindow.tool-catalog.ts, ChatWindow.tsx,
ChatWindowRequestBody.ts, ChatWindow.preferences.ts, chat-window-context.ts, ChatWindowManager.tsx,
ToolPreferences.model.ts, ToolPreferencesList.tsx; plus data/chat/clickyOperationsToTools.ts.

1. TS port of the MatchItems subset (comma split, ! negation-wins, * affix/contains, case-insensitive,
   empty-list-true, exclusion-only-true; skip url.QueryUnescape — document the divergence) + a
   resolvePolicy(rules, toolMeta) last-match-wins resolver. Add verb/scope to ToolMeta
   (data/chat/types.ts) sourced from x-clicky in clickyOperationsToTools.ts and normalizeToolMeta.
2. effectiveToolPreferences/surfaceDefault → policy resolution over
   [...surfaceRules, ...userRules] with the catalog defaultPermission as fall-through.
3. toolDefaults: Record<string, ToolMode> → toolPolicy: PermissionRule[] on the panel
   (chat-window-context.ts:30, ChatWindowManager.tsx:85,149; loadPanels resets it as today).
4. User prefs become an ordered rule list: ToolPreferencesList group toggle emits/updates a
   {group} rule, a row toggle appends a {name} rule (later wins — a per-tool choice beats the user's
   own group toggle). Badge/"Custom" computation runs the resolver per tool instead of reading a flat map.
   Persisted shape changes → migrate/clear stored toolPrefs in ChatWindow.preferences.ts.
5. ChatWindowRequestBody.ts sends toolPolicy: [...surfaceRules, ...userRules] instead of the
   flattened per-name toolPreferences map.
6. Build in place per repo convention (pnpm run build in packages/ui) — no branch.

Step 5 — xero-cli webapp

Files: src/chat/surface-prompts.ts, src/ChatWidget.tsx, src/chat/tool-tier-preferences.ts,
src/chat/tool-tiers.ts, src/chat/tool-catalog.ts.

1. SurfacePrompt.toolPolicy replaces toolPreferences; ChatWidget.tsx:161 passes it as the panel's
   toolPolicy. Surface pages (TemplateEditorPage, TransactionDetailPage, …) are untouched — they
   pass the whole prompt object through.
2. Bump TOOL_TIER_VERSION in tool-tier-preferences.ts (existing reset mechanism) for the stored-prefs
   shape change.
3. presentXeroTools (tool-tiers.ts:32-52) keeps its throw-on-missing-defaultPermission guard — the
   Step 3 shared resolver guarantees every catalog entry carries one.

Edge cases to encode in tests

• MatchItems quirks: zero patterns → true (why empty ToolMatch is rejected); ! wins regardless of
  position; exclusion-only list matches everything not excluded; url.QueryUnescape silently drops
  malformed tokens (a selector reduced to zero patterns matches nothing); interior * does NOT glob
  (ap*e ≠ apple) — only prefix/suffix/contains.
• auto rules are skipped, not recorded (an auto rule must not mask an earlier on/off).
• FromPreferences determinism (sorted keys, group rules before name rules).
• Approval-resume replays the persisted spec's policy (guards the specMarshal addition).
• api.IsEmpty / spec_merge_differential_test.go fixtures gain the new field.

Verification

• captain: make lint && make build, ginkgo over pkg/ai/tools, pkg/api, pkg/aichat,
  pkg/ai/provider/..., pkg/ai/callertools (new policy_ginkgo_test.go covers the matcher table).
• clicky: go build ./... && go test ./entity/... ./mcp/... ./rpc/... ./aichat/... — rework
  entity/toolpermission_test.go, rpc/converter_toolgroup_test.go, aichat/adapter_ginkgo_test.go.
• xero-cli: make lint && make build; go test ./pkg/commands/... ./pkg/chatprompts/... — reworked
  chat_tool_groups_test.go, chat_tool_metadata_test.go, chat_test.go, chatprompts_test.go; the new
  execution-vs-catalog agreement test is the acceptance gate for the review's F1 defect.
• clicky-ui: pnpm lint && pnpm test && pnpm build in packages/ui (vitest: tool-catalog, preferences,
  ChatWindow body assertions, new matcher tests).
• webapp: pnpm test && pnpm build; live smoke via agent-browser: open template-editor Ask AI → catalog
  badges match the surface policy (provider/admin groups off), toggle a group off and confirm the rule
  rides the request, per-tool re-enable beats the group toggle, approve a write tool, resume an approval,
  and confirm xero accounts list (live Xero read) no longer auto-runs anywhere.

Out of scope (tracked separately from the review)

Regrouping rates out of templates.*, splitting client-action tools into an editor.actions group,
disabling sync in chat, and deleting the redundant overlayChatToolMetadata entries — the policy
mechanism makes these one-line rule/group edits later.

Verification

---

timeout: 45m

Structural checks

Name	Command	Exit Code	CEL Validation
PermissionPolicy types exist in captain pkg/api	test -f /Users/moshe/go/src/github.com/flanksource/captain/pkg/api/toolpolicy.go	0	exitCode == 0
ResolveDefinitions wired to ToolPolicy	rg -q "ToolPolicy" /Users/moshe/go/src/github.com/flanksource/captain/pkg/ai/tools/	0	exitCode == 0
clicky verb-default stamping removed	rg -c "verbDefaultToolPermission" /Users/moshe/go/src/github.com/flanksource/clicky/entity/annotations.go	1	exitCode == 1
prompts declare ordered toolPolicy	rg -l "toolPolicy:" /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/chatprompts/prompts/	0	stdout.contains(".prompt")
toolPreferences map gone from prompts	rg -l "toolPreferences:" /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/chatprompts/prompts/	1	exitCode == 1
empty group accounting.transactions.write gone	rg -l "accounting.transactions.write" /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/chatprompts/prompts/	1	exitCode == 1
empty group provider.takealot.write gone	rg -l "provider.takealot.write" /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/chatprompts/prompts/	1	exitCode == 1
empty group comments.read gone	rg -l "comments.read" /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/chatprompts/prompts/	1	exitCode == 1
static default-permission table deleted	rg -c "toolGroupDefaultPermission " /Users/moshe/go/src/github.com/flanksource/xero-cli/pkg/commands/chat_tool_groups.go	1	exitCode == 1
clicky-ui request body sends toolPolicy	rg -q "toolPolicy" /Users/moshe/go/src/github.com/flanksource/clicky-ui/packages/ui/src/data/ai/ChatWindowRequestBody.ts	0	exitCode == 0
webapp surface prompts expose toolPolicy	rg -q "toolPolicy" /Users/moshe/go/src/github.com/flanksource/xero-cli/webapp/src/chat/surface-prompts.ts	0	exitCode == 0

Test suites

command: captain policy, spec, aichat, and provider tests pass

timeout: 900

cd /Users/moshe/go/src/github.com/flanksource/captain
go test ./pkg/api/... ./pkg/ai/tools/... ./pkg/aichat/... ./pkg/ai/callertools/... 2>&1 | tail -20

• cel: exitCode == 0
• not: contains: FAIL

command: clicky entity, rpc, mcp, and aichat tests pass

timeout: 600

cd /Users/moshe/go/src/github.com/flanksource/clicky
go test ./entity/... ./rpc/... ./mcp/... 2>&1 | tail -20
cd aichat && go test ./... 2>&1 | tail -10

• cel: exitCode == 0
• not: contains: FAIL

command: xero-cli commands and chatprompts tests pass

timeout: 900

cd /Users/moshe/go/src/github.com/flanksource/xero-cli
go test ./pkg/commands/... ./pkg/chatprompts/... 2>&1 | tail -20

• cel: exitCode == 0
• not: contains: FAIL

command: execution path and OpenAPI catalog agree, and provider.xero.read is off

A dedicated Go test must assert that (a) every chat-exposed tool resolves to the same defaultPermission on the cobra execution path and the OpenAPI catalog
path, and (b) xero-parented live list/get tools (group provider.xero.read) resolve to off — the adversarial review's F1 defect.

timeout: 600

cd /Users/moshe/go/src/github.com/flanksource/xero-cli
rg -l "provider.xero.read" pkg/commands/ --glob '*_test.go'
go test ./pkg/commands/ -run 'Agree|Policy|Catalog' -count=1 2>&1 | tail -10

• cel: exitCode == 0
• not: contains: FAIL
• not: contains: no test files

command: clicky-ui ai/chat vitest passes

timeout: 600

cd /Users/moshe/go/src/github.com/flanksource/clicky-ui/packages/ui
pnpm vitest run src/data/ai src/data/chat 2>&1 | tail -15

• cel: exitCode == 0

command: webapp chat vitest passes

timeout: 600

cd /Users/moshe/go/src/github.com/flanksource/xero-cli/webapp
pnpm vitest run src/chat 2>&1 | tail -15

• cel: exitCode == 0

Builds

command: xero-cli and clicky-ui build

timeout: 900

cd /Users/moshe/go/src/github.com/flanksource/xero-cli
make build 2>&1 | tail -5
cd /Users/moshe/go/src/github.com/flanksource/clicky-ui/packages/ui
pnpm run build 2>&1 | tail -5

• cel: exitCode == 0

[coderabbitai]

🔗 Related PRs

#13 - Feat/captain subagent history [merged]
#37 - Improve session and agent workflows [merged]
#38 - Improve transcript ingestion and setup [open]
#45 - Add sandbox adapter framework for CLI confinement [merged]
#46 - Implement git-agent protocol for remote task execution [open]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[adityathebe]

Where this sits in the overall plan: a cross-repo program of its own (captain → clicky → xero-cli → clicky-ui → webapp), but its captain step lands inside the same wire/spec seams as the chat-contract cluster (#54/#53), so the captain slice should be scheduled with that work. Three coordination points from reviewing all open issues + PRs together against main @ 5db1179:

1. Name collision to resolve in Step 1: api.ToolPolicy already exists as a per-tool allow/deny/auto mode type (pkg/api/permissions.go:96 and the Tools.Policies() map). The plan's Spec.ToolPolicy PermissionPolicy field would shadow/collide. Either rename the new field (Spec.PermissionPolicy?), or migrate/rename the legacy type first — silent coexistence of two "ToolPolicy" meanings in pkg/api will confuse every later step.
2. PR Resolve hierarchical runtime profiles #62 (SpecLayer/ResolvedSpec) hand-rolls the same layering this plan needs. It introduces ordered scopes global → context → surface → user with provenance (Trace) and constraint intersection, and routes .prompt frontmatter through PromptSpecLayer + ResolveSpecLayers. The plan's layer order (group/verb/hint/app baked server-side; surface rules from .prompt frontmatter; user rules from the UI) maps cleanly onto it: if PermissionRule lists live in SpecLayer and concatenate in scope order during ResolveSpecLayers, layer ordering, provenance ("which layer set this mode" — useful for the UI badge), and frontmatter plumbing come for free, and the Spec.Merge replace-wholesale question mostly disappears. Worth deciding before writing pkg/api/toolpolicy.go, since Resolve hierarchical runtime profiles #62 is open now.
3. One wire revision, not two: ChatRequest currently carries ToolPreferences (pkg/aichat/wire.go:25); this plan replaces it with toolPolicy while Make Captain chat threads and provider sessions authoritative #54 removes client-supplied identity authority from the same struct. Batch the breaking change.

The captain-side types + ResolveDefinitions threading (all 6 call sites, specMarshal persistence for approval-resume) are otherwise self-contained and can proceed in parallel with everything except the two items above. The xero-cli/clicky/clicky-ui steps are outside this repo's cluster and unaffected by the rest of the captain issue set.

---

Generated by Claude Code