Skip to content

nvsnap: agent API bearer token travels over plaintext HTTP #759

Description

@balajinvda

CodeRabbit raised this across four call sites on #555 (internal/agent/auth.go, internal/agent/agent.go, cmd/nvsnap-mount-prep/main.go, internal/webhook/mount_prep_init.go). Filing it as one item because it is one decision, not four fixes.

What

The agent API bearer token added in #555 is sent as an Authorization: Bearer header over http://. Every hop is plaintext:

  • server -> agent (http://<nodeIP>:8081)
  • agent -> peer agent (cascade fetch)
  • mount-prep init container -> agent (node-local)

Anything that can observe those packets can capture the token and replay it against a privileged API that restores and deletes checkpoints and serves arbitrary files from inside a checkpoint.

Why it was not fixed in #555

Closing this means TLS on the agent listener: cert issuance and rotation for a hostNetwork DaemonSet, https:// in every advertised peer URL, and a trust story for the mount-prep init container. That is a design change, not a review fix, and it interacts with a decision already taken for #555: the cluster network is treated as trusted, the token is defense-in-depth against accidental exposure of a hostNetwork port rather than against an on-path attacker, and auth ships off by default with permissive as the rollout state.

The design note in auth.go also records a throughput reason to prefer a header over mTLS: the same router serves multi-GB peer fan-out, where per-byte encryption does not amortize the way a handshake does. Any TLS proposal here should say what that costs on the cascade path.

What #555 did fix

The related redirect leak, which was cheap and unambiguous: net/http strips Authorization on a cross-origin redirect, but a header-adding RoundTripper re-adds it on the redirected request. Both the server client wrapper and the agent peer client now refuse redirects (cbc318c13), so a peer cannot bounce a caller at an arbitrary host and collect the token.

Decision needed

Whether to do TLS/mTLS on the agent API at all, given the trusted-network position, or to close this as accepted risk with the reasoning written down.

Metadata

Metadata

Assignees

Labels

nvsnapnvsnap GPU checkpoint/restore

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions