Radius.Compute/containerImages type: add rootless BuildKit sidecar to dynamic-rp chart#11882
Draft
willdavsmith wants to merge 12 commits into
Draft
Radius.Compute/containerImages type: add rootless BuildKit sidecar to dynamic-rp chart#11882willdavsmith wants to merge 12 commits into
willdavsmith wants to merge 12 commits into
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the deploy/Chart dynamic-rp deployment to optionally run a rootless BuildKit sidecar (default on) and adds supporting documentation/tests, enabling in-cluster image build/push scenarios without relying on a host Docker socket.
Changes:
- Add
dynamicrp.buildkit.*values and wire abuildkitdsidecar +buildctl-initinit container into the dynamic-rp Deployment. - Fix Terraform pre-mount pathing in the chart and add a drift-guard helm-unittest to keep chart/runtime paths aligned.
- Add operator-facing NOTES warnings plus new design/contributor docs for the subsystem.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/recipes/terraform/install.go | Logs an INFO message when no pre-mounted Terraform binary is present (to aid diagnosis). |
| eng/design-notes/recipes/2026-04-container-images-resource-type.md | Adds a design note for the containerImages resource type and how BuildKit is used. |
| docs/contributing/contributing-code/contributing-code-writing/buildkit-recipes.md | Documents the chart’s BuildKit sidecar and the recipe authoring pattern it enables. |
| deploy/Chart/values.yaml | Introduces the dynamicrp.buildkit values surface (enabled/psaMode/image/credentials/resources). |
| deploy/Chart/tests/helpers_test.yaml | Adds helm-unittest coverage for BuildKit enable/disable and Terraform-path drift guard. |
| deploy/Chart/templates/NOTES.txt | Adds install-time warnings for incompatible PSA mode / missing registry credentials. |
| deploy/Chart/templates/dynamic-rp/rbac.yaml | Grants dynamic-rp RBAC permissions for batch Jobs. |
| deploy/Chart/templates/dynamic-rp/deployment.yaml | Implements the terraform pre-mount fix and adds BuildKit containers/env/volumes. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11882 +/- ##
==========================================
+ Coverage 51.90% 51.94% +0.04%
==========================================
Files 732 732
Lines 46272 46305 +33
==========================================
+ Hits 24016 24052 +36
+ Misses 19957 19953 -4
- Partials 2299 2300 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
willdavsmith
force-pushed
the
feat/containerimages-buildkit-sidecar
branch
from
5df989d to
9d6ddfa
Compare
willdavsmith
changed the title
willdavsmith
force-pushed
the
feat/containerimages-buildkit-sidecar
branch
from
48b5a64 to
1f983a8
Compare
Adds an optional, rootless BuildKit sidecar to the dynamic-rp Pod that in-cluster Terraform recipes can drive via the buildctl CLI to build and push container images. Motivating consumer is the new Radius.Compute/containerImages resource type in resource-types-contrib. Chart additions (deploy/Chart): - buildkitd sidecar container (rootless, no privileged Pod, no host Docker socket) with default-on enabled flag, configurable image, PSA mode (restricted/baseline), credentialsSecret, and resource limits/requests - buildctl-init init container that mounts the buildctl CLI into the dynamic-rp container's PATH - registry-credentials volume mounting an operator-supplied Secret at ~/.docker/config.json - RBAC: batch/jobs access for recipe-spawned Jobs - NOTES.txt warnings for misconfigured PSA mode and missing creds - helm-unittest cases covering buildkit sidecar shape (default-on / disabled) and a drift-guard against the install.go path contract Other changes: - pkg/recipes/terraform/install.go: log INFO when no pre-mounted Terraform binary is present, naming the expected paths - Fixes a path-mismatch bug in the chart's Terraform pre-mount init script (was writing to a directory the runtime never reads from) - New contributor doc covering the buildkit subsystem and the local-exec recipe pattern - Design doc copied to eng/design-notes/recipes/ Coordinates with resource-types-contrib PR for the resource type and recipe. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
The containerImages recipe now reads registry credentials from a per-resource Radius.Security/secrets resource via the kubernetes_secret_v1 data source, matching the mysql pattern. The chart no longer needs to mount a Docker config.json — drop dynamicrp.buildkit.credentialsSecret value, volume, and volumeMount. Update fsGroup comment to reflect the buildctl binary mount (TCP, no socket sharing). Rewrite NOTES.txt to point platform engineers at the recipe-registration flow. Add a helm unittest covering the buildctl-init init container when terraform is disabled. Spelling list additions for new tech terms. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
The previous default of `restricted` requires Kubernetes 1.30+ with the UserNamespacesSupport feature gate, which is not available out of the box on kind, k3d, Docker Desktop, or older managed clusters. This forced almost every operator trying out the BuildKit sidecar to immediately discover the failure mode and reinstall with --set dynamicrp.buildkit.psaMode=baseline. Flip the default so `rad install kubernetes` is a one-liner on every supported Kubernetes version. Operators who enforce PSA restricted cluster-wide and run a recent enough kernel can opt into the stricter sidecar profile with --set dynamicrp.buildkit.psaMode=restricted; the existing NOTES.txt preflight surfaces a clear remediation if it's selected on an incompatible cluster. Also update the NOTES.txt registry-credentials hint to reflect the PE-owned dockerconfigjson Secret model rather than the developer-owned Radius.Security/secrets language. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
The buildkit recipes contributing guide and the container-images design note now live on the upstream PR branch; remove the demo submodule copies. Drop unused cspell entries (Buildah, Kaniko, binfmt, buildctl, buildkitd) since they no longer appear in any file shipped by the submodule. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
…buildkit-sidecar Signed-off-by: willdavsmith <willdavsmith@gmail.com> # Conflicts: # deploy/Chart/templates/dynamic-rp/deployment.yaml
…buildkit-sidecar Signed-off-by: willdavsmith <willdavsmith@gmail.com>
…kit branch) Signed-off-by: willdavsmith <willdavsmith@gmail.com>
- deployment.yaml: the .Values.dynamicrp.resources block was checking dynamicrp but rendering rp.resources, so any dynamicrp-scoped resource overrides silently fell through to applications-rp's values. Render dynamicrp.resources to match the if-check. - Tighten verbose multi-line comments around hostUsers, fsGroup, buildctl-init, env vars, the buildkitd sidecar header, and the PSA seccomp/AppArmor + newuidmap rationale. Preserve load-bearing comments (/terraform hardcode, pinned Terraform version, GLOBAL_DIR path duplication). - values.yaml: trim verbose buildkit field docs. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
willdavsmith
force-pushed
the
feat/containerimages-buildkit-sidecar
branch
from
e200448 to
5903352
Compare
The containerImages recipe uses the in-pod BuildKit sidecar (buildctl over the sidecar's socket); it does not create Kubernetes Jobs, so this grant is unnecessary. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
Drop changes unrelated to the Radius.Compute/containerImages feature: - NOTES.txt operator guidance (belongs in resource-type README) - dynamicrp.resources rename in deployment.yaml (pre-existing chart bug, tracked separately) - terraform binary-path drift-guard test (about install.go, not containerImages) Signed-off-by: willdavsmith <willdavsmith@gmail.com>
Signed-off-by: willdavsmith <willdavsmith@gmail.com>
HOME=/home/rpuser was set so buildctl could find a chart-mounted ~/.docker/config.json. That credentials-volume approach was dropped in favor of a per-recipe Docker config written under the module path with DOCKER_CONFIG set in the recipe's local-exec environment, so dynamic-rp itself never reads $HOME/.docker/config.json. Remove the env var to avoid implying credentials wiring that doesn't exist. Signed-off-by: willdavsmith <willdavsmith@gmail.com>
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
Comment on lines
+41
to
+47
| {{- if .Values.dynamicrp.buildkit.enabled }} | ||
| # fsGroup 65532 lets the dynamic-rp container (UID 65532) read | ||
| # the buildctl binary copied into the shared emptyDir by the | ||
| # buildctl-init init container. | ||
| securityContext: | ||
| fsGroup: 65532 | ||
| {{- end }} |
Comment on lines
+200
to
+202
| # buildctl lives in the mounted emptyDir. | ||
| - name: PATH | ||
| value: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/buildctl" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a rootless BuildKit sidecar to the
dynamic-rpPod so in-cluster Terraform recipes can build and push container images without a host Docker socket, a privileged Pod, or per-node host preparation.The motivating consumer is the new
Radius.Compute/containerImagesresource type — see companion PR radius-project/resource-types-contrib#151.What's in this PR
Scope is deliberately limited to the chart changes the BuildKit-backed recipe needs to function:
buildkitdsidecar container (rootless, listens on Pod loopback TCP127.0.0.1:1234) added to the dynamic-rp Pod.buildctl-initinit container that copies thebuildctlCLI into a sharedemptyDir, mounted into the dynamic-rp container'sPATH.dynamicrp.buildkit.*values surface:enabled(defaulttrue),psaMode(baselinedefault,restrictedopt-in on K8s ≥ 1.30 withUserNamespacesSupport),image,resources.helm-unittestcases for the buildkit sidecar shape (default-on / disabled).Three files, +191 / -1. No Go code changes, no RBAC changes, no documentation changes.
Wave structure
This PR is Wave 1: the chart runtime piece the recipe needs to function end-to-end with git-context builds. It is independently reviewable and mergeable.
Wave 2 — follow-up PRs in this repo (none depend on Wave 1 being unmerged):
docs/contributing/.../buildkit-recipes.md— contributor doc explaining the sidecar +local-exec-via-buildctlrecipe pattern.NOTES.txtpreflight surfacingKubernetes ≥ 1.30 + UserNamespacesSupport requiredwhenpsaMode=restrictedis selected on an incompatible cluster.Wave 3 — local context upload (depends on Wave 1):
emptyDirfor the recipe to consume.radCLI local-path detection: whenbuild.sourceis a local path, tar with.dockerignorehonored and POST to dynamic-rp before recipe execution.radius-project/resource-types-contribto accept the staged context path.Until Wave 3 lands,
build.sourceis restricted togit::https://...URLs and absolute filesystem paths already available to the recipe runtime.Notable details
enabled: true. The buildkit sidecar runs by default on a fresh install. Operators who don't want it can--set dynamicrp.buildkit.enabled=false.psaMode=baseline(default) works on every supported Kubernetes version.psaMode=restrictedrequires Kubernetes ≥ 1.30 withUserNamespacesSupport(useshostUsers: false).Radius.Security/secretsresource and read by the recipe viadata "kubernetes_secret". Nothing is mounted at chart level.Testing
helm-unittestpasses (including new buildkit cases).Coordination
Companion PR: radius-project/resource-types-contrib#151 (resource type + recipe). They should land together; this one is reviewable independently.
Design: #11734.