Creative-library storyboards run (and fail) against sellers declaring has_creative_library:false — gating leaks via tool auto-registration and schema defaults (both 3.1.2 and 3.1.5 packs)

[tej656]

Summary

Four media_buy_seller storyboards execute their sync_creatives steps against a seller that explicitly declares it has no creative library (creative.has_creative_library: false, media_buy.features.inline_creative_management: true, and sync_creatives omitted from tools/list). Instead of skipping, all four run and fail with UNSUPPORTED_FEATURE.

None of the four gate on has_creative_library, and the gates they do declare are each satisfied by something other than an affirmative capability claim from the agent.

Reproduces identically on the 3.1.2 and 3.1.5 compliance packs — this is not a recent regression in the pack, though one of the four causes does appear to trace to #2278 (see below).

Environment

• Runner: @adcp/sdk@12.1.1 (pack 3.1.5) and @adcp/sdk@12.0.4 (pack 3.1.2)
• Agent under test: inline-only sell-side agent built on @adcp/sdk@12.0.4
• Command: adcp storyboard run <url> media_buy_seller --allow-http --compliance-version <3.1.2|3.1.5>

Agent's declared capabilities (relevant subset)

{
  "supported_protocols": ["media_buy", "creative"],
  "media_buy": {
    "features": { "inline_creative_management": true },
    "creative_approval_mode": "auto_approve"
  },
  "creative": {
    "has_creative_library": false,
    "supports_compliance": false,
    "supports_generation": false,
    "supports_transformation": false
  }
}

tools/list returns 14 tools and does not include sync_creatives:

get_products, create_media_buy, update_media_buy, get_media_buys,
get_media_buy_delivery, list_creative_formats, list_creatives, list_accounts,
sync_accounts, report_usage, get_task_status, list_tasks, tasks_get,
get_adcp_capabilities

media_buy.propagation_surfaces is not declared at all.

Expected

Storyboards whose steps call sync_creatives should skip (capability_unsupported / missing_tool) for an agent declaring creative.has_creative_library: false.

Actual

Four storyboards run and fail. Both packs produce identical results — 119 steps, 36 pass / 4 fail / 79 skip, with zero per-step status differences between 3.1.2 and 3.1.5:

Failing step	Storyboard's declared gate
pending_creatives_to_start/sync_creative	requires_capability: media_buy.creative_approval_mode == auto_approve
creative_fate_after_cancellation/sync_creative_with_assignment	required_tools: [get_products, create_media_buy, update_media_buy, sync_creatives, list_creatives] (no requires_capability)
dependency_impairment/sync_creative	requires_capability: {path: media_buy.propagation_surfaces, contains: "snapshot"}
dependency_impairment_cardinality/sync_two_creatives	same as above

All four fail with the same error:

UNSUPPORTED_FEATURE: sync_creatives not supported by this sales platform

The failures also cascade: 26 of the 79 skips are prerequisite_failed downstream of these four setup steps.

Root causes

Four distinct gating gaps, each independently sufficient.

1. required_tools is OR-evaluated, not AND

runner.mjs:1238:

const hasAnyRequired = storyboard.required_tools.some((t) => options.agentTools.includes(t));

.some() means a storyboard listing five required tools runs when the agent advertises any one of them. creative_fate_after_cancellation requires [get_products, create_media_buy, update_media_buy, sync_creatives, list_creatives]; the agent advertises the first three, so the gate passes and the storyboard proceeds to call the tool it does not have. The failure message (agent does not advertise any of [...]) confirms the OR intent is deliberate, but it makes required_tools unusable as a capability precondition — it cannot express "this storyboard needs sync_creatives".

If OR semantics are intentional, the field is misnamed and there is no working way to require a specific tool; if a storyboard needs all of them, this should be .every().

2. Tool discovery reports a tool the agent omits from tools/list

The runner's own observation reports 15 tools including sync_creatives and list_creatives, while the agent's tools/list returns the 14 above without sync_creatives:

tool_discovery: Discovered 15 tools: [get_products, create_media_buy, update_media_buy,
get_media_buys, get_media_buy_delivery, list_creative_formats, sync_creatives,
list_creatives, list_accounts, sync_accounts, report_usage, get_task_status,
list_tasks, tasks_get, get_adcp_capabilities]

Verified directly against the running agent — a raw tools/list JSON-RPC call returns 14 entries with no sync_creatives. I haven't traced the mechanism (the runner may union tools/list with capability-implied or SDK-registered tool names), but the effect is that the discovered set includes a tool the agent deliberately withholds. This compounds #1: even under .every() semantics the gate would still pass.

3. requires_capability is satisfied by a JSON Schema default the agent never declared

dependency_impairment and dependency_impairment_cardinality gate on:

requires_capability:
  path: media_buy.propagation_surfaces
  contains: "snapshot"

The agent never declares propagation_surfaces. But resolveCapabilityPathForGate falls through to getSchemaDefaultByPath when the path is absent, and get-adcp-capabilities-response.json defines:

"propagation_surfaces": {
  "type": "array",
  "default": ["snapshot"],
  "items": { "enum": ["snapshot", "webhook", "out_of_band"] }
}

Because the agent declares some media_buy block, schemaDefaultShouldApply returns true, the runner substitutes ["snapshot"], and contains: "snapshot" passes. The storyboard-level gate therefore never short-circuits, setup runs, and sync_creatives fails.

The pack's own comment flags this behavior (dependency_impairment.yaml:25):

# materializes capability-gate schema defaults; earlier runners skip them as
# not_applicable (adcp-client#2278).

So this looks like an unintended consequence of #2278: before that change these storyboards graded not_applicable for agents that don't declare propagation_surfaces; now the schema default satisfies the gate and the run fails instead of skipping. A schema default describes what a field means when omitted — it is not an affirmative capability claim, and treating it as one inverts the fail-safe direction of every requires_capability gate that targets a defaulted field.

Worth noting the spec already draws exactly this distinction for a neighbouring field. From the creative_approval_mode description in the same schema:

When absent, approval behavior is legacy-unspecified; runners SHOULD NOT treat omission as an affirmative auto-approval claim.

4. pending_creatives_to_start conflates auto-approval with having a creative library

It gates only on media_buy.creative_approval_mode == auto_approve. A conformant inline-only auto-approve seller matches that gate legitimately, but the supply_creatives phase invokes sync_creatives with no inline fallback.

Its inline-only counterpart inline_creatives_without_sync (gated on media_buy.features.inline_creative_management == true) passes against this same agent, so the vocabulary to express this correctly already exists — this scenario just doesn't use it.

Suggested fixes

Scenario authoring — add an explicit creative.has_creative_library gate to every storyboard whose steps call sync_creatives:

• creative_fate_after_cancellation, dependency_impairment, dependency_impairment_cardinality:

  requires_capability:
    path: creative.has_creative_library
    equals: true

  (for the two dependency_impairment storyboards this needs to compose with the existing propagation_surfaces gate)
• pending_creatives_to_start: add the same gate, or branch supply_creatives to the inline-creative path when inline_creative_management == true and has_creative_library == false, mirroring inline_creatives_without_sync.

Runner — two behavioral fixes, both of which change fail-open to fail-closed:

• Do not apply JSON Schema defaults when resolving a requires_capability path. Absent should mean absent. If default-materialization is needed for some gates, make it opt-in per predicate (e.g. use_schema_default: true) rather than the global fallback.
• Either fix required_tools to .every(), or add a genuinely conjunctive field so a storyboard can state that it needs a specific tool. Also worth reconciling the discovered-tool set with what tools/list actually returns (Fix Fly.io deployment timeout issues #2).

Since both the 3.1.2 and 3.1.5 packs are affected, a scenario-side fix likely wants backporting across the 3.1.x cache line rather than landing only on tip.

Impact

Any spec-conformant inline-only seller — has_creative_library: false plus inline_creative_management: true, precisely the configuration inline_creatives_without_sync exists to validate — cannot obtain a clean media_buy_seller run on any 3.1.x pack. Four scenarios report fail where they should report skip, and 26 further steps cascade to prerequisite_failed.

Because requires_capability currently fails open on any defaulted field, the blast radius is wider than these four storyboards: every gate whose target path carries a JSON Schema default is affected the same way.

[bokelley]

Thanks for filing!

Triage

Classification: Bug (storyboard runner)
Bucket(s): conformance, cli
Status: ready-for-human

What the experts said:

• ad-tech-protocol-expert: required_tools OR semantics are documented in types.ts:45 — runner is correct, storyboard YAMLs are the bug (spec repo). Root Fix Fly.io deployment timeout issues #2 is multi-agent routing-union bleed at runner.ts:2057. Root Update documentation and production configuration #3 fix direction (absent ≠ affirmative claim) is protocol-correct per the spec's own "SHOULD NOT treat omission as affirmative claim" language, but partially reverts fix: apply schema defaults to capability gates #2282 with broad grade impact and needs explicit sign-off.
• code-reviewer: Root AdCP Testing Framework with Authentication, Hierarchical Logging & Security Features #1 is a confirmed comply.ts-vs-runner.ts inconsistency — comply.ts:1278 uses AND (any missing tool → skip) while runner.ts:2215 uses OR — fix direction is architectural and ambiguous. Root Fix Fly.io deployment timeout issues #2 mechanism is unconfirmed from source (may be tryListModernMCPTools path or cluster variance; a raw tools/list JSON-RPC capture from the runner would confirm). Root Update documentation and production configuration #3: types.ts:131–134 documents default-materialization as intentional per fix: apply schema defaults to capability gates #2282; the bug is in the storyboard YAML missing a creative.has_creative_library: { equals: true } gate, not the runner.

My take: Two decisions needed from @bokelley: (1) required_tools OR vs AND — comply.ts:1278 (AND) and runner.ts:2215 (OR) diverge and one must yield, with storyboard YAML auditing required if AND wins; (2) whether equals/contains gates should continue materializing schema defaults per the #2282 contract, or revert to "absent = not declared" for non-present predicates. For Root #4 and the storyboard-YAML portion of Root #1, filing on adcontextprotocol/adcp is the right path — pending_creatives_to_start, creative_fate_after_cancellation, dependency_impairment, and dependency_impairment_cardinality each need an explicit creative.has_creative_library: { equals: true } gate or a phase-level requires_tool: sync_creatives step.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01US1TFDHM4kXvYSHgwTqC9f

---

Generated by Claude Code