Sandbox isolation leaks host state and weakens no-internet guarantees

[xdotli]

Summary

The sandbox/security audit found multiple places where untrusted task definitions or benchmark configs can leak host state or bypass documented isolation guarantees.

This should be treated as a production-release blocker for running third-party benchmark repos locally.

Findings

1. allow_internet = false is not enforced for LLM agent runs

In Rollout.__init__, BenchFlow derives _disallow_web_tools, but _create_sandbox_environment(..., preserve_agent_network=True) then flips env_config.allow_internet back to true for LLM agent runs.

The code comment says BenchFlow enforces no-web at the agent layer instead. That only disables supported web tools. Coding agents can still use shell/network paths like curl, Python requests, package managers, etc.

Existing proof test:

uv run python -m pytest tests/test_internet_policy.py::test_create_environment_preserves_agent_network_for_llm_runs -q

Expected: allow_internet=false should mean no outbound task network, or docs/config should be renamed to make the weaker guarantee explicit.

2. Docker Compose receives the full host environment

DockerSandboxEnvVars.to_env_dict(include_os_env=True) starts from dict(os.environ), and _run_docker_compose_command() passes that to Docker Compose while also loading task-supplied environment/docker-compose.yaml.

A malicious compose file can interpolate host secrets or paths such as ${OPENAI_API_KEY}, ${GITHUB_TOKEN}, ${HOME}, etc.

Affected code:

• src/benchflow/sandbox/docker.py around DockerSandboxEnvVars.to_env_dict()
• src/benchflow/sandbox/docker.py around _run_docker_compose_command()

Expected: Compose should receive a minimal allowlisted environment containing only BenchFlow's required compose variables plus explicitly configured task env.

3. Dockerfile dependency staging allows .. traversal outside context_root

_stage_copy_source() does:

abs_src = context_root / src_path

but does not resolve and enforce containment. With SDK/runtime context_root, a Dockerfile can use COPY ../outside-secret.txt /loot, and BenchFlow stages it into environment/_deps.

Affected code: src/benchflow/sandbox/setup.py around _stage_copy_source().

Expected: resolve both paths and reject any source outside context_root.

4. Verifier env secrets are base64-encoded into Docker exec argv

The raw value is not printed directly, but the base64 blob in the command line decodes to the exported env file. This weakens the intended protection against ps/argv leakage for [verifier].env API keys.

Affected code:

• src/benchflow/sandbox/docker.py around _wrap_command_with_env_file() / exec wrapping
• src/benchflow/task/verifier.py verifier env path

Expected: secret-bearing verifier env should be passed via temp files, Docker env-file, stdin, or another mechanism that does not put reversible values in process argv.

Impact

High. These issues can invalidate offline benchmarks, leak host secrets into untrusted Docker Compose evaluation contexts, and stage files outside an intended build context.

Suggested fix

• Split “agent model/network access” from “task internet access” so LLM API access does not imply arbitrary shell network access inside the task container.
• Use a minimal allowlist for Docker Compose env instead of dict(os.environ).
• Enforce context_root containment with resolved paths for all staged Dockerfile sources.
• Avoid putting verifier secret material, including reversible base64, in command argv.
• Add regression tests that use fake secret env vars and malicious task fixtures, without exposing real secrets.

[xdotli]

Fixed in #430 (fix: redact COOKIE/AUTHORIZATION env vars; sandbox isolation) — merged to v0.5-integration. Closing as resolved.

[bingran-you]

Reproduced on v0.5-integration (HEAD 5934433).

PRs #430/#471 fix part of the issue, especially Dockerfile .. staging and Daytona DinD raw -e KEY=VALUE argv. But the issue as filed is not fully fixed. Safe probes still show allow_internet=false being flipped to True for LLM runs, Docker Compose receiving host env (host_secret_env_forwarded=True), and Docker exec env still living in a reversible base64 argv payload. Marking reproduced.