[EAI-7305]: Remove cluster-auth from the stack

[johnl-amd]

• Drop the cluster-auth and cluster-auth-config apps from root/values.yaml and from enabledApps in all three sizings, plus the medium resources override
• Delete the cluster-auth (0.5.0, 0.5.9) and cluster-auth-config chart sources
• Delete the cluster-auth ext_authz SecurityPolicy and its ReferenceGrant
• Stop writing the OpenBao root token to secrets/cluster-auth-openbao-token, and drop the now-inert INIT_MODE export that was its only trigger
• Drop the security-policy-extauth patch block from the OpenShift installer, which would abort the install under set -euo pipefail once the template is gone
• Regenerate the SBOM

Nothing depends on cluster-auth any more. The AIM serving routes are covered by the route-scoped workloads-extauth policy and ai-gateway by its gateway-scoped default-deny, and neither the airm nor the aiwb chart has a live reference. On app-dev its own counters show 230k checks over 15 days, all fast_path with authenticator=none and zero denies.

The generated Applications carry no resources-finalizer.argocd.argoproj.io, so pruning them leaves the running Deployment, Service and namespace in place rather than deleting them. Rollback is a revert; the workloads are never torn down. Cleanup of the orphaned cluster-auth namespace and the stale secrets/cluster-auth-openbao-token KV entry is manual.

Out of scope: secrets/cluster-auth-admin-token stays until core's aiwb chart drops the ExternalSecret that points at it, and the standalone cluster-auth shim stays in the install scripts.

https://jira.amd.com/browse/EAI-7305

How to test

Confirm no route on the target cluster relies on cluster-auth before merging:

kubectl get httproute -A -o json | jq -r '.items[] | select(.metadata.annotations // {} | keys[] | test("cluster-auth")) | "\(.metadata.namespace)/\(.metadata.name)"'

Empty output means removal is a no-op there. Locally: helm lint and helm template pass on all four values files, envoy-gateway-config renders with aiGateway on and off, and the four sbom/validate-*.sh scripts pass.

[johnl-amd]

Deployed this branch to app-dev and ran the auth matrix against both endpoints.

Heads up for anyone deploying this: you have to remove two lines from your cluster-values values.yaml first.

enabledApps:
  - cluster-auth          # remove
  - cluster-auth-config   # remove

Helm replaces lists rather than merging them, so an overlay that pins its own enabledApps keeps requesting apps that no longer exist in root/values.yaml, and cluster-apps.yaml uses required. That fails the render of the entire root chart, not just cluster-auth, so ArgoCD stops syncing everything until the overlay is edited. Do the overlay first, then the cluster-forge bump.

Cluster state after the sync

Root app Synced and Healthy, 52 Applications. Both the cluster-auth and cluster-auth-config Applications are gone. The generated Applications carry no resources-finalizer, so the cluster-auth Deployment and Service are still running but orphaned rather than deleted, which is the intended benign outcome. cluster-auth-extauth-policy is gone and only ai-gateway-default-deny plus workloads-extauth remain. Both gateways report Programmed and Accepted, and the envoy pods have zero restarts, so the https proxy picked up the policy removal over xDS without a restart.

Positive, using a key bound to its own model

case	result
ai-gateway, key with routing headers	200, real completion
ai-gateway, key with body only	200, real completion
workloads, key	200, real completion

Negative, everything refused

case	result
no key, both endpoints	401
bogus key, both endpoints	401
valid key bound to a different model, both endpoints	401
unauthorized model, full routing headers	401
unauthorized model, x-ai-eg-model only	401
unauthorized model, body only	401

The body-only rows matter because that is the path a standard OpenAI client takes, and it is still enforced by the ext_authz handler. The workloads rows come back with "not authorized for this workload", which confirms workloads-extauth is doing that work rather than anything left over from cluster-auth.

One test looked alarming at first and turned out to be my mistake. Sending x-ai-eg-backend for a different workload while the body named the authorized model returned 200, but the response .model came back as the authorized model, so the header never redirected the request. Asking for the other workload's actual model is refused on all three routing paths.

[brownzebra]

Awesome! Thank you.