Make model listing and availability caches auth-consistent

[moshloop]

Problem

The active model listing path combines three caches: a 60-second in-memory adapter/auth snapshot, a 24-hour persisted API model catalog, and runtime-specific local catalogs. The API path is not consistently bound to one credential.

ProbeAdapters accepts caller-supplied APICredentials and reports auth from that snapshot, but fetchAPIModels uses the snapshot only to decide that a key exists. ResolveModels then re-reads the process vault/environment for fingerprinting and re-reads it again while fetching. A caller can therefore receive auth status for token/account B with model availability fetched under token/account A. Token rotation during one probe creates the same time-of-check/time-of-use mismatch.

The persisted catalog is also a single models.json slot while authenticated backends resolve concurrently. Every backend writes the same models.json.tmp path, so multiple configured API tokens race and only one backend remains cached. The outer adapter cache is timestamp-only, leaving changed or removed tokens and local login methods stale for up to 60 seconds.

Scope

• Carry one immutable resolved credential snapshot through auth reporting, cache fingerprinting, and the provider model-list request. Never re-resolve a global credential during the same operation.
• Ensure ProbeAdapters uses the exact credential supplied in AuthProbe, including custom callers and tests.
• Partition persisted API model availability by backend, provider endpoint, resolution options/schema version, and a non-secret namespace for the exact effective token or account.
• Treat vault and environment source changes with the same effective token consistently, while always separating different tokens and API versus CLI/OAuth auth.
• Keep CLI/local-runtime catalogs separate from API-key-gated model lists. Where a local catalog is account-specific, include a stable account namespace or do not persist it.
• Replace the single cache slot and shared temporary filename with concurrency-safe per-key entries or one locked multi-entry document using unique temporary files.
• Invalidate availability immediately when the effective token, login account, endpoint, or credential presence changes; do not rely solely on the 60-second TTL.
• Do not derive or persist fingerprints for unrelated provider tokens, or for catalog-only queries with token use disabled.
• Store model rows and any credential verifier under 0700/0600 permissions. Use a machine-local keyed HMAC rather than a fixed-salt, low-cost token verifier, and never persist or log raw credentials.
• Preserve an explicit refresh path without allowing refreshes for one backend to evict another backend cache entry.

Security cases

• A caller-supplied token must be the exact token used for the live model request and its cache namespace.
• Token A model availability must never be returned as token B availability.
• Removing a token or changing a local login must immediately change readiness and availability.
• Multiple simultaneously configured provider tokens must retain independent 24-hour cache entries without write races.
• Private, restricted, or fine-tuned model IDs must not be exposed to other local users.

Non-goals

Generated provider output caching is tracked separately.

Verification

---

cwd: .
timeout: 15m
codeBlocks: [test, lint]
ai: {}
verify:
scope: diff
threshold: 100

Focused tests

paths: [./pkg/ai]
framework: [go, ginkgo]
test-timeout: 5m
show-passed: true

Changed-code lint

changed: true
fix: false
timeout: 5m

Acceptance Criteria

• One resolved credential snapshot drives adapter auth status, cache identity, and the live model-list request.
• ProbeAdapters uses caller-supplied APICredentials rather than silently substituting process-global vault or environment credentials.
• Different API tokens, provider endpoints, and API versus CLI/OAuth auth have isolated availability entries.
• Vault and environment sources with the same effective token behave consistently without persisting the raw token.
• Token rotation, token removal, and local account changes invalidate readiness and model availability without waiting for the outer TTL.
• Concurrent resolution for at least two authenticated API backends retains correct independent cache entries and has no shared temporary-file race.
• A warm cache avoids new provider model-list requests for every cached backend, not only the last writer.
• Catalog-only resolution with token use disabled persists no credential-derived verifier.
• Cache files and directories use 0600 and 0700 permissions respectively, and credential fingerprints use a machine-local keyed HMAC.
• Raw tokens and reusable credential material are absent from persisted data and logs.
• Focused credential-substitution, rotation, multi-backend concurrency, invalidation, and permissions tests fail against the old implementation and pass with the fix.
• make lint and make build pass.

[coderabbitai]

🔗 Related PRs

#13 - Feat/captain subagent history [merged]
#19 - fix(security): resolve open CodeQL code-scanning alerts [merged]
#38 - Improve transcript ingestion and setup [open]
#45 - Add sandbox adapter framework for CLI confinement [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[adityathebe]

Where this sits in the overall plan: independent parallel track — the only open issue confined to pkg/ai with no coupling to the chat-contract or storage workstreams. It can start anytime without waiting on anything. Reviewed against main @ 5db1179; every claim verified:

• ProbeAdapters honours caller-supplied APICredentials for auth/availability, but fetchAPIModels uses the probe only as a gate (adapter_models.go:50 calls resolveModelRows with ResolveOptions that has no credential field), and ResolveModels re-reads the process vault/env twice (catalog_resolve.go:167 in unionLive, :228 in resolveFingerprint) — the caller's snapshot never reaches the fetch or the cache identity.
• Single cache slot + shared temp file confirmed: ~/.config/captain/models.json with tmp := path + ".tmp" (model_cache.go:29-39,75), while fetchAPIModels fans out one goroutine per backend — so concurrent authenticated backends race one file and only one survives, exactly as described.
• Permissions are 0755/0644 (world-readable) and the fingerprint is fixed-salt (resolveFingerprintSalt constant, catalog_resolve.go:222) PBKDF2 truncated to 8 bytes — comparable across every Captain install, enabling offline token confirmation. The keyed-HMAC + 0700/0600 requirements in the scope are the right fix. Same shared-.tmp pattern exists in pkg/ai/pricing/openrouter.go:223 — worth sweeping in the same pass.

Two light coordination notes from the joint review:

1. PR Resolve hierarchical runtime profiles #62 annotates model/runtime availability with profile-constraint provenance on top of this same listing path — no code conflict expected (it consumes, this issue fixes the producer), but land whichever goes second with awareness of the other's tests.
2. The credential-snapshot type this issue introduces ("one immutable resolved credential snapshot through auth, fingerprint, fetch") is the natural carrier for Make Captain chat threads and provider sessions authoritative #54's "resolved backend" reporting later — worth shaping it as a value type the aichat layer can echo, not an internal of ResolveModels.

---

Generated by Claude Code