Skip to content

fix(fetch): enforce URL policy on agent probes - #179

Open
chaliy wants to merge 1 commit into
mainfrom
2026-08-21-fix-agent-probes-bypassing-url-prefix-policy
Open

fix(fetch): enforce URL policy on agent probes#179
chaliy wants to merge 1 commit into
mainfrom
2026-08-21-fix-agent-probes-bypassing-url-prefix-policy

Conversation

@chaliy

@chaliy chaliy commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • A path-scoped allow/block URL policy was bypassed by generated same-origin agent discovery probes, allowing HEAD probes to be sent to disallowed prefixes during markdown fetches.
  • The intent is to ensure every outbound request triggered during a fetch honors the caller's URL prefix policy to prevent unintended egress to blocked or out-of-scope paths.

Description

  • Enforce prefix policy validation before each outbound request by calling super::validate_url_policy(&current_url, options)?; at the start of the per-hop loop inside send_request_following_redirects so generated probes and redirect hops are checked.
  • Add a regression unit test test_agent_resource_probes_honor_url_prefix_policy that verifies a /docs-scoped allowlist permits the seed GET but prevents root-level discovery probes from being dispatched.
  • Limit change surface to the transport/redirect path so existing DNS/IP/port checks and probe behavior remain intact while applying the prefix policy to all outgoing URLs.

Testing

  • Ran cargo fmt --all -- --check and it passed.
  • Ran the new focused unit test via cargo test -p fetchkit test_agent_resource_probes_honor_url_prefix_policy and it passed.
  • Ran cargo test -p fetchkit for the crate test suite and tests completed successfully with the new regression included.
  • cargo clippy was attempted in this environment but did not finish within the execution window; please run cargo clippy --workspace --all-targets -- -D warnings locally or in CI as part of pre-merge checks.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant