Skip to content

compile panics on unresolvable adapter config - #290

Merged
handcaught merged 1 commit into
mainfrom
compile-panics-on-unresolvable-adapter-config
Jul 31, 2026
Merged

compile panics on unresolvable adapter config#290
handcaught merged 1 commit into
mainfrom
compile-panics-on-unresolvable-adapter-config

Conversation

@handcaught

Copy link
Copy Markdown
Member

compile-panics-on-unresolvable-adapter-config

Repository: brokenbots/criteria

Confirmed behavior

When the workflow compiler has access to an adapter's ConfigSchema, an adapter config attribute of type string that is assigned from a workflow variable with no default produces a panic instead of a diagnostic.

Refs under test and results:

  • origin/main (0829f887968d42d8b5e4d53685c6f48086390caf) — reproduced.
  • v0.5.5 (6ac528757d2615d64b4b5f78949a12bdae1870fa) — reproduced.

Observed panic stack trace on both refs (root workflow path):

panic: value is unknown [recovered, repanicked]
...
github.com/zclconf/go-cty/cty.Value.AsString(...)
    cty/value_ops.go:1464
github.com/brokenbots/criteria/workflow.validateSchemaAttrs(...)
    workflow/compile_validation.go:222
github.com/brokenbots/criteria/workflow.resolveAdapterConfig(...)
    workflow/compile_adapters.go:180
github.com/brokenbots/criteria/workflow.compileOneAdapter(...)
    workflow/compile_adapters.go:88
github.com/brokenbots/criteria/workflow.compileAdapters(...)
    workflow/compile_adapters.go:51
github.com/brokenbots/criteria/workflow.CompileWithContext(...)
    workflow/compile.go:122

For subworkflows the same panic propagates through:

github.com/brokenbots/criteria/workflow.compileSingleSubworkflow(...)
    workflow/compile_subworkflows.go:75
github.com/brokenbots/criteria/workflow.compileSubworkflows(...)
    workflow/compile_subworkflows.go:27

Shapes that reproduced the panic on both refs:

  • A2 (root, schema-aware): workflow contains variable "missing_default" { type = string } and an adapter with config { system_prompt = var.missing_default }; the compiler is given a real ConfigSchema for the adapter. See evidence/run-main.md and evidence/run-stable.md, Shape A2.
  • C (subworkflow, schema-aware): same variable/adapter pattern inside a subworkflow reached by the parent through workflow blocks. See evidence/run-main.md and evidence/run-stable.md, Shape C.

Shapes that did not reproduce — already correct, do not regress:

  • A (CLI, permissive path): Without a locally installed criteria-adapter-copilot binary, the CLI falls back to schema-unverified decoding via decodeAttrsToStringMap, which already guards unknown/null values and returns a located diagnostic.
  • B (default present): A variable with a default compiles cleanly; the default value is folded into the adapter config.
  • E (no schema): An adapter with no ConfigSchema and an unresolved variable returns a normal located diagnostic, no panic.
  • F (step input): Step-level input {} referencing an unresolved variable is runtime-deferred and compiles cleanly.
  • G (type error): A true type error in a schema-aware adapter config (e.g., assigning a number to a string field) returns a normal located diagnostic, no panic.

Required behavior

  1. workflow.validateSchemaAttrs must not call cty.Value.AsString() on a cty.String value unless the value is known and non-null.
  2. When an adapter config attribute evaluates to an unknown or null cty.String, the compiler must emit a located diagnostic error that names the attribute and states that the value cannot be determined.
  3. The compiler must return the diagnostic (i.e., Diagnostics) to the caller instead of panicking.
  4. Requirements 1–3 apply both to root workflows compiled directly and to child workflows reached through compileSubworkflows / compileSingleSubworkflow.
  5. The permissive no-schema path (decodeAttrsToStringMap), the known-default path, the step-input path, and the type-error path must continue to behave exactly as they did before the fix.

The fix belongs in workflow/compile_validation.go inside validateSchemaAttrs, before the per-type coercion branch reads a concrete string from the value.

Exit criteria

  1. A regression test in workflow/ calls workflow.Compile (or workflow.CompileWithContext) with a real adapter ConfigSchema and a workflow containing a string variable with no default referenced from adapter "copilot" "bot" { config { system_prompt = var.missing_default } }. The test must fail before the fix by triggering the panic and pass after the fix by receiving a non-empty Diagnostics and no panic.
  2. A second regression test in workflow/ covers the subworkflow path: a parent workflow invokes a child workflow whose adapter config contains the same unresolved string variable pattern. The test must fail before the fix by triggering the panic and pass after the fix by receiving a non-empty Diagnostics and no panic.
  3. All existing tests in workflow/ continue to pass after the fix.
  4. make test passes at the repository root after the fix.
  5. The shapes listed under "already correct — do not regress these" in the Confirmed behavior section continue to produce their pre-fix results after the fix.

…string variable

In validateSchemaAttrs, cty.Value.AsString() was called on cty.String values
without first checking IsKnown()/IsNull(). When a schema-aware adapter
config attribute referenced a variable with no default, the resulting
unknown cty.String caused a panic.

Add an unknown/null guard before the per-type coercion so that:
- step-level input (evalCtx == nil) continues to defer the value to runtime;
- adapter-level config (evalCtx != nil) emits a located diagnostic naming
  the field and stating that the value cannot be determined.

Add regression tests for both root workflows and subworkflows reached via
compileSubworkflows/compileSingleSubworkflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@brokenbot brokenbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The fix adds an unknown/null guard in validateSchemaAttrs before the coercion switch that previously called AsString() on unresolved cty.String values, panicking. Adapter-config mode (non-nil evalCtx) now emits a located diagnostic naming the attribute; step-input mode (nil evalCtx) defers to runtime with an empty placeholder, preserving prior behavior. Type-check branch continues before the guard so the type-error path is unaffected. Two regression tests cover the root and subworkflow paths (child diagnostics propagate via compileSubworkflows). Full workflow suite passes; all CI gates green.

@handcaught
handcaught merged commit dff749c into main Jul 31, 2026
10 checks passed
@handcaught
handcaught deleted the compile-panics-on-unresolvable-adapter-config branch July 31, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants