allow passing arbitrary secrets to workflows

[jameslamb]

This PR enables populating arbitrary environment variables for script: from GitHub secrets, for the following workflows:

• conda-cpp-tests
• conda-python-tests
• wheel-tests

Proposing this because something like it is needed for NVIDIA/cuopt#748. In that PR, cuopt workflows want to download datasets from a private S3 bucket, and need a way to pass in credentials and other sensitive configuration.

Notes for Reviewers

Why this design?

GitHub Actions limitations:

• you cannot set env: at the callsite of a reusable workflow
• you cannot access the secrets.* context at the callsite of a reusable workflow
• the GitHub CLI and API do not offer a way to programmatically retrieve the value of a secret

Other design considerations:

• want to be able to pass secrets from a repo in one org (NVIDIA) to workflows sourced from another (rapidsai)
• use case will change from repo-to-repo (this one is somewhat cuopt-specific)

GitHub Actions offers a preferred way to do this. The workflow_call: workflow trigger accepts a mapping called secrets:, which is similar to inputs: but where GitHub knows to treat the values as sensitive (e.g. to obscure them from the UI and logs). See https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows#passing-inputs-and-secrets-to-a-reusable-workflow

This design of having separate inputs for the key and value also inspired by the envFrom concept in Kubernetes:

spec:
  containers:
  - name: envars-test-container
    env:
    - name: SECRET_USERNAME
      valueFrom:
        secretKeyRef:
          name: backend-user
          key: backend-username

(kubernetes docs link)

Wait don't we already handle secrets in this repo? And differently?

Yes, there are a couple places where workflows accept a string in inputs: with the name of a secret, and then later look it up. For example, in build-in-devcontainer:

shared-workflows/.github/workflows/build-in-devcontainer.yaml

Lines 66 to 70 in 58dc4f0

	rapids-aux-secret-1:
	description: |
	The NAME (not value) of a GitHub secret in the calling repo.
	This allows callers of the workflow to make a single secret available in the devcontainer's
	environment, via environment variable `RAPIDS_AUX_SECRET_1`.

shared-workflows/.github/workflows/build-in-devcontainer.yaml

Line 168 in 58dc4f0

RAPIDS_AUX_SECRET_1=${{ inputs.rapids-aux-secret-1 != '' && secrets[inputs.rapids-aux-secret-1] || '' }}

I think the pattern introduced here is preferable, for a couple reasons:

1. allows you to set arbitrary environment variables to a secret value (instead of needing an intermediate one like RAPIDS_AUX_SECRET_1 to be set and then processing that in scripts)
2. makes the usage of the secret a bit easier to understand

How I tested this

On cuopt PRs pointed at this branch:

• https://github.com/NVIDIA/cuopt/pull/755/files#r2677219295
• Move datasets to s3 to reduce dependency on third-party download sites NVIDIA/cuopt#748 (comment)

[msarahan]

I think you need a default value here. What happens if the value is set but the key is not? It's a silly situation, but there's no other way to ensure sanity here.

perhaps just SCRIPT_ENV_SECRET_X_KEY, where X is 1, 2 or 3?

[jameslamb]

I agree but unfortunately the secrets: inputs don't allow you to set a default. They only take description: and required:.

From actionlint:

.github/workflows/checks.yaml:55:9: unexpected key "default" for "secrets" section. expected one of "description", "required" [syntax-check]

I could add more handling in the shell script that processes these.

Also side note... would that script be a good use case for an action? I think yes.

[msarahan]

Also side note... would that script be a good use case for an action? I think yes.

I think you mean a shared-action? Maybe. At some point, I think it's hard to maintain what github considers to be secret. Will pipelining this through a shared-action, then returning its validated values still correctly mask stuff? I dunno. It's worth a try, but I wouldn't go too deep on any rabbit holes.

[jameslamb]

Good point, good point. Ok I'll just leave this as an inline script for now, even with the extra handling to defend against "empty key, non-empty value", it won't be that complex.

[jameslamb]

Ok think I've handled this in the latest commits. Setting the value but not the key should now result in a loud and clear error.

I've also moved most of the processing into a bash function, so that if we wanted to allow more than 3 secrets all the logic wouldn't need to be repeated.

[jameslamb]

Might as well update to the latest zizmor while we're doing this, to give it a chance to catch any security issues with this (it did not report any new issues).

[msarahan]

I think 3 secrets is very likely enough to cover all reasonable situations. If we need more, we can add one or two, but I wonder if we'll hit a limit at some point, like there is for inputs.

This is just musing, not requesting any change. I don't know what the upper limit is. Github's copilot says this:

[Copilot](https://docs.github.com/en/copilot/responsible-use-of-github-copilot-features/responsible-use-of-github-copilot-chat-in-githubcom) uses AI. Check for mistakes.
Inputs

For reusable workflows (on.workflow_call.inputs) each input must declare a type of boolean, number, or string.
If a caller passes an input that the called workflow does not declare, the run fails with an error.
For workflow_dispatch (manual trigger) inputs: maximum 25 top-level input properties and a maximum payload of 65,535 characters. (Default values when omitted: boolean → false, number → 0, string → "".)
Secrets

In a reusable workflow (on.workflow_call.secrets) declare the named secrets the called workflow expects. If the caller passes a secret that the called workflow did not declare, the run errors.
To pass secrets from the caller: use jobs.<job_id>.secrets to pass named secrets, or secrets: inherit to pass all of the caller's secrets to the directly called workflow.
Secrets are only passed to directly called workflows. In a chain A → B → C, C receives a secret from A only if A passed it to B and B then passed it to C.
Nested reusable workflows are limited to 10 levels (caller + up to 9 nested). Permissions and secrets can only be maintained or reduced through the chain — they cannot be elevated.
Best practices and gotchas

Avoid using structured data as a single secret (JSON/YAML blobs) because redaction may fail.
Audit and pin third-party actions; follow least-privilege for tokens used in workflows.
Make sure the called workflow declares any inputs and secrets you intend to pass to avoid errors.
Quick checklist to avoid errors

In the called workflow (on.workflow_call) declare every input with type and declare expected secrets.
In the caller job use with: for inputs and secrets: (or secrets: inherit) for secrets.
For nested workflows, re-pass secrets explicitly from each caller to the next.

Maybe that means it shares the 25-item limit on inputs? Maybe they each get 25? Who knows? I think we're safe for the foreseeable future.

[msarahan]

good check!

[jameslamb]

Thanks for considering the input limit, not something I'd thought of.

It does look like 25 (https://github.com/orgs/community/discussions/8774, https://github.blog/changelog/2025-12-04-actions-workflow-dispatch-workflows-now-support-25-inputs/), unsure if that includes both inputs: and secrets:.

Either way yeah, I think this should be fine for the foreseeable future and if we needed to support more than 25 in the future we could adjust to that then.