Skip to content

fix(01-harness): correct the gateway docs, harden the pollers, stop leaking the role - #1870

Merged
evandrofranco merged 1 commit into
awslabs:mainfrom
rmncardoso:fix/gateway-integration-docs-and-role-cleanup
Aug 3, 2026
Merged

fix(01-harness): correct the gateway docs, harden the pollers, stop leaking the role#1870
evandrofranco merged 1 commit into
awslabs:mainfrom
rmncardoso:fix/gateway-integration-docs-and-role-cleanup

Conversation

@rmncardoso

@rmncardoso rmncardoso commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Concise description of the PR

The 02-gateway-integration sample works end to end, but the README describes a different sample than the one that ships, the gateway pollers can't report the failures the service actually returns, and every run leaves the IAM execution role behind.

This builds on #1863 (this branch's direct parent), which fixed the harness poller in this file. The gateway pollers a few lines above it have the same two defects — a timeout shorter than the documented wait, and status checks against values the API never returns — and neither was in scope there.

Documentation

  • A "Create routing rule" step the sample never performs. A gateway routes to its targets as soon as they report READY. Rules are real (CreateGatewayRule exists) but are a separate, optional feature for shaping traffic across multiple targets. Replaced with a note saying so.
  • "IAM auth (NONE type)" contradicts itself. NONE means no inbound authorizer; AWS_IAM is a distinct value this sample doesn't use. It also collapsed two independent controls into one — the distinction 07-oauth gets right, so the two samples disagreed. Replaced with a "Two independent authorization sides" section and an inbound/outbound table.
  • A harness "wired to the gateway's ARN" — no such binding exists. CreateHarness takes no gateway parameter; the ARN travels per-invoke in tools. Stated the decoupling explicitly.
  • Step numbers didn't match the output — README and docstring numbered 1–7, the script prints Step 0Step 4 plus cleanup.
  • The flow list promised cleanup would delete the IAM role, which appears nowhere in the code, and the Clean Up snippet omits it too.
  • Three new troubleshooting entries: HTTP 429 from Exa's shared free tier (with the outbound API_KEY config), the UPDATE_UNSUCCESSFUL/SYNCHRONIZE_UNSUCCESSFUL statuses, and a target stuck in CREATE_PENDING_AUTH.

Code

  • GATEWAY_POLL_TIMEOUT was 120s while the README promised "2-3 minutes." Raised to 300s, matching 07-oauth's budget for these same operations. Both resources actually reach READY in ~5s, so the old ceiling only bit on the slow runs it was least able to survive.
  • The pollers checked for statuses the API never returns. A gateway reports UPDATE_UNSUCCESSFUL, not FAILED, when an update fails; a target adds SYNCHRONIZE_UNSUCCESSFUL. Neither was terminal, so a failed resource was polled until timeout and statusReasons never surfaced. The target poller also tested DELETE_FAILED, which isn't in the target enum at all — a dead branch. The three *_PENDING_AUTH states now fail fast instead of spinning on a state that can't clear by itself.
  • The stream handler named only internalServerException. validationException and runtimeClientError are modelled members of the same event-stream union, so the handler now names them too. Being precise about what this does and does not do, because my first draft of this description overstated it: with standard AWS exception framing (:message-type: exception) botocore sets a 400 and raises EventStreamError before the loop ever sees a dict, so these two branches are defensive completeness rather than a fix for a failure I reproduced. The live path for a rejected invoke is the EventStreamError catch already in stream_response. I was unable to trigger either event against a real harness.
  • REGION read AWS_DEFAULT_REGION alone, while utils/client.py honours AWS_REGION too and treats "" as unset. A shell exporting AWS_DEFAULT_REGION="" built a working harness_control and a gw_control that died on Invalid endpoint: https://bedrock-agentcore-control..amazonaws.com. Now imports the shared REGION.
  • _cleanup deletes the execution role, guarded by created_role so a role passed via --role-arn is never touched — the name is shared by all 8 samples in this folder. Mirrors the guard merged in fix(01-harness): correct execution limits, share the poller, and stop leaking the role in 4 advanced samples #1864.

Also annotated the create_gateway / create_gateway_target call sites with which authorization side each parameter controls, dropped the unused os import, and restored alphabetical import order.

User experience

Before — a successful run on unmodified main:

Cleaning up...
  Deleted harness: GatewayHarness_81cc1fb4-Ow49bI1ek1
  Deleted target: FMBJ1RTMRS
  Deleted gateway: gatewaydemo-f5c5ed82-ilqcro0d78

$ aws iam get-role --role-name HarnessExecutionRole
HarnessExecutionRole    2026-08-01T00:01:09+00:00     <- left behind

After: the same three deletes, followed by Deleted inline policy: HarnessExecutionPolicy and Deleted role: HarnessExecutionRole.

A reader following the README no longer looks for a routing-rule step that doesn't exist, and a failed gateway or target stops the run with the service's own reason instead of timing out silently.

Testing

Live end-to-end runs in us-west-2 — one on unmodified upstream/main to capture the shipped behaviour above, one on the exact bytes committed here (md5-verified before and after). Every run: gateway and target READY, the agent called exa-search___web_search_exa through the gateway and returned real results, exit 0. Post-run sweeps confirmed a clean account, including the auto-provisioned managed memory, which drains on its own.

Unit-checked against stubbed clients, since most new branches can't be reached from a healthy account: both pollers (happy path, every terminal and pending-auth status, transient passthrough, timeout); the stream handler (tool-use, raw mode, tools payload shape, and each error event injected as a plain dict); _cleanup (created_role true/false/default, and failure before any resource exists). The empty-region regression was reproduced before the fix and confirmed resolved after.

Worth a reviewer's attention: the two new stream-error branches are exercised only by injected dicts. Against the real service these exceptions arrive as :message-type: exception, which botocore turns into a raised EventStreamError, so the branches are unreachable on that path. They are cheap and consistent with the other samples in the folder, but I would rather flag that than let the diff imply I had reproduced them.

Every API-shape claim above was asserted against the botocore 1.43.59 service model — both enums for authorization, apiKeyCredentialProvider members, both status enums (including DELETE_FAILED's absence from the target enum), the InvokeHarness stream events, and the absence of any gateway parameter on CreateHarness.

Both CI gates pass. Ruff findings on this file go from 3 to 0 — the I001 came from #1863's own new import, and the two BLE001s were already failing on main.

…eaking the role

The gateway-integration sample runs correctly end to end, but its docs describe a
different sample than the one that ships, its gateway pollers cannot report the
failures the service actually returns, and cleanup leaves the IAM role behind.

Documentation
- The README walked the reader through a "Create routing rule" step. A gateway
  routes to its targets as soon as they report READY -- no rule is needed to run
  this sample. Replaced with a note explaining that rules are a separate,
  optional feature for shaping traffic across multiple targets.
- The Key Concepts heading read "IAM auth (`NONE` type)", which contradicts
  itself: NONE means no inbound authorizer, and AWS_IAM is a distinct value the
  sample does not use. Replaced with a "Two independent authorization sides"
  section and a table separating inbound (authorizerType on create_gateway) from
  outbound (credentialProviderConfigurations on create_gateway_target), so it no
  longer conflicts with 07-oauth, which configures both.
- The README flow and the module docstring were numbered 1-7 while the script
  prints Step 0-4 plus cleanup. Renumbered both to match the real output.
- Both described Step 3 as creating a harness "wired to the Gateway". No such
  binding exists: create_harness takes no gateway parameter, and the gateway ARN
  travels per-invoke in the `tools` argument to invoke_harness. Corrected, and
  stated explicitly, because the false coupling is what made the Clean Up
  section's ordering advice look justified.
- The Clean Up snippet listed the deletes in an order that contradicted its own
  "targets before gateway" note, omitted the role deletion entirely, and implied
  the harness must go first because it references the gateway. Only the
  target-before-gateway constraint is real.
- Added troubleshooting entries for HTTP 429 (the default Exa endpoint is keyless
  on a shared, rate-limited free tier, so a busy account can exhaust it; shows
  how to attach your own key as an outbound API_KEY credential provider), for the
  UPDATE_UNSUCCESSFUL / SYNCHRONIZE_UNSUCCESSFUL statuses, and for a target stuck
  in CREATE_PENDING_AUTH.

Code
- GATEWAY_POLL_TIMEOUT was 120s while the README told the reader to expect "2-3
  minutes" -- the same contradiction between a local constant and the documented
  wait that awslabs#1863 fixed for the harness poller. Raised to 300s, matching
  07-oauth's budget for these two operations. Measured: both the gateway and the
  target reach READY in about 5s, so the old ceiling only ever mattered on the
  slow runs it was least able to survive.
- The pollers checked for the wrong terminal statuses. A gateway reports
  UPDATE_UNSUCCESSFUL, not FAILED, when an update fails, and a target adds
  SYNCHRONIZE_UNSUCCESSFUL; neither was treated as terminal, so a failed resource
  was polled until the timeout and the service's statusReasons was never shown.
  The target poller also tested for DELETE_FAILED, which is not in the target
  status enum at all -- a dead branch. Both now use the real enums, and the three
  *_PENDING_AUTH states fail fast with an explanation instead of spinning on a
  state that cannot clear by itself.
- The stream handler surfaced only internalServerException. validationException
  and runtimeClientError are modelled stream events too, and falling through them
  printed nothing, so a rejected invoke looked like an agent with no answer.
- REGION was read from AWS_DEFAULT_REGION alone, while utils/client.py
  deliberately honours AWS_REGION as well and treats an empty string as unset.
  A shell exporting AWS_DEFAULT_REGION="" therefore built a working
  harness_control and a gw_control that died on "Invalid endpoint:
  https://bedrock-agentcore-control..amazonaws.com". Now imports the shared
  REGION so both clients resolve identically.
- _cleanup now deletes the execution role, guarded by created_role so that a role
  supplied via --role-arn is never touched. The role name is shared by every
  sample in this folder, so deleting a caller's role would destroy something the
  script does not own. Without this the role outlived every run.
- Annotated the create_gateway and create_gateway_target call sites with which
  authorization side each parameter controls, dropped the now-unused os import,
  and moved the utils.iam import back into alphabetical order.

Every API-shape claim added here was read off the botocore service model rather
than inferred: the authorizerType and credentialProviderType enums, the
apiKeyCredentialProvider members, the credentialLocation enum, the gateway and
target status enums, the InvokeHarness stream event members, and the absence of
any gateway parameter on CreateHarness.

Verified live in us-west-2, five end-to-end runs, the last on the exact bytes
committed here: gateway and target reach READY with no routing rule, the agent
calls exa-search___web_search_exa through the gateway and returns real results,
and teardown removes the harness, target, gateway, inline policy and role,
leaving the account clean. Unit-checked against stubbed clients: both pollers for
the happy, every terminal, every pending-auth, transient-passthrough and timeout
paths; the stream handler for all three error events, tool-use, raw mode and the
tools payload shape; and _cleanup for created_role=True, created_role=False, the
default argument, and failure before any resource exists. The empty-region
regression was reproduced before the fix and confirmed resolved after. Ruff
check and format both pass; findings on the file go from 3 to 0.
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Latest scan for commit: 9b959eb | Updated: 2026-08-01 00:23:09 UTC

Security Scan Results

Scan Metadata

  • Project: ASH
  • Scan executed: 2026-08-01T00:22:53+00:00
  • ASH version: 3.0.0

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

Column Explanations:

Severity Levels (S/C/H/M/L/I):

  • Suppressed (S): Security findings that have been explicitly suppressed/ignored and don't affect the scanner's pass/fail status
  • Critical (C): The most severe security vulnerabilities requiring immediate remediation (e.g., SQL injection, remote code execution)
  • High (H): Serious security vulnerabilities that should be addressed promptly (e.g., authentication bypasses, privilege escalation)
  • Medium (M): Moderate security risks that should be addressed in normal development cycles (e.g., weak encryption, input validation issues)
  • Low (L): Minor security concerns with limited impact (e.g., information disclosure, weak recommendations)
  • Info (I): Informational findings for awareness with minimal security risk (e.g., code quality suggestions, best practice recommendations)

Other Columns:

  • Time: Duration taken by each scanner to complete its analysis
  • Action: Total number of actionable findings at or above the configured severity threshold that require attention

Scanner Results:

  • PASSED: Scanner found no security issues at or above the configured severity threshold - code is clean for this scanner
  • FAILED: Scanner found security vulnerabilities at or above the threshold that require attention and remediation
  • MISSING: Scanner could not run because required dependencies/tools are not installed or available
  • SKIPPED: Scanner was intentionally disabled or excluded from this scan
  • ERROR: Scanner encountered an execution error and could not complete successfully

Severity Thresholds (Thresh Column):

  • CRITICAL: Only Critical severity findings cause scanner to fail
  • HIGH: High and Critical severity findings cause scanner to fail
  • MEDIUM (MED): Medium, High, and Critical severity findings cause scanner to fail
  • LOW: Low, Medium, High, and Critical severity findings cause scanner to fail
  • ALL: Any finding of any severity level causes scanner to fail

Threshold Source: Values in parentheses indicate where the threshold is configured:

  • (g) = global: Set in the global_settings section of ASH configuration
  • (c) = config: Set in the individual scanner configuration section
  • (s) = scanner: Default threshold built into the scanner itself

Statistics calculation:

  • All statistics are calculated from the final aggregated SARIF report
  • Suppressed findings are counted separately and do not contribute to actionable findings
  • Scanner status is determined by comparing actionable findings to the threshold
Scanner S C H M L I Time Action Result Thresh
bandit 0 0 0 0 0 0 525ms 0 PASSED MED (g)
cdk-nag 0 0 0 0 0 0 4.2s 0 PASSED MED (g)
cfn-nag 0 0 0 0 0 0 4ms 0 PASSED MED (g)
checkov 0 0 0 0 0 0 4.4s 0 PASSED MED (g)
detect-secrets 0 0 0 0 0 0 758ms 0 PASSED MED (g)
grype 0 0 0 0 0 0 34.8s 0 PASSED MED (g)
npm-audit 0 0 0 0 0 0 106ms 0 PASSED MED (g)
opengrep 0 0 0 0 0 0 <1ms 0 SKIPPED MED (g)
semgrep 0 0 0 0 0 0 <1ms 0 MISSING MED (g)
syft 0 0 0 0 0 0 1.3s 0 PASSED MED (g)

@evandrofranco evandrofranco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@evandrofranco
evandrofranco merged commit ceacf51 into awslabs:main Aug 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants