fix: round-trip serdes result on first run across ops

[ayushiahjolia]

Issue #, if available:
#406
#544

Description of changes:
When a step, child context, or wait_for_condition finishes, we now serialize its result and deserialize it back before saving the SUCCEED checkpoint. The value the function returns on the first run is the deserialized one - exactly what it would return on replay.

Before this, the first run returned the raw in-memory result, but replay returned the value rebuilt from the checkpoint. With a custom serdes that changes the value in transit, those two could differ. Running the round-trip up front makes the first run and replay always agree, and guarantees a SUCCEEDED result is always reconstructable.

Serdes failures are now clearly split:

• Permanent failures raise SerDesError. The operation fails (no step retry) and user code can catch it.
• Transient failures raise RetryableSerDesError, which fails the invocation so the backend retries the whole thing (no step retry).

Behavior changes to note

• For wait_for_condition, the wait strategy now sees the round-tripped value, matching the Java and JS SDKs.
• A result that wasn't serializable before (for virtual or large-payload child contexts) will now raise SerDesError instead of silently passing through.
• Custom serdes authors should expect deserialize to be called right after serialize on the first run. For large payloads this means one extra round-trip.
• Python matches Java's round-trip-before-checkpoint; JS defers deserialize until after SUCCEED for step/child.
• Serdes failures are now a catchable SerDesError (permanent, no retry) or RetryableSerDesError (transient, backend retry)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[yaythomas]

should this be raising RetryableSerDesError or SerDesError in the new world?

[ayushiahjolia]

SerDesError is ExecutionError and RetryableSerDesError is InvocationError, so this is correct. However, we can rework filesystem serdes now that we have new error.

[yaythomas]

re SerDesError name clash raised by the bot reviewer...

A collision here means except StepError: misses a real step failure and except SerDesError: catches a non-serdes one

could be an idea to store the namepssced class name rather than the short/bare class name? i.e aws_durable_execution_sdk_python.exceptions.SerDesError?

collisions become difficult... To collide, a user would have to ship a module literally named aws_durable_execution_sdk_python.exceptions containing a class of the same name.

At which point they've shadowed the SDK's own import path and have gone to a fair degree of trouble.

[github-actions]

Codex AI review

1. P1 operation/step.py:273 — A retryable SerDes failure leaves an AT_MOST_ONCE_PER_RETRY step at STARTED. On backend retry, replay treats it as an interrupted step and invokes the step retry strategy, consuming a retry or failing permanently instead of retrying the SerDes operation. Preserve sufficient state for replay, or explicitly handle serialize/deserialize failures under at-most-once semantics; add coverage for this configuration.

2. P2 concurrency/executor.py:273 — Recording retryable_error does not stop the scheduling loop. With CompletionConfig.all_completed() or tolerated failures, every pending map/parallel branch can still start before the error is re-raised, delaying the backend retry and potentially reaching the Lambda timeout. Stop admitting pending branches once this field is set, drain only already-running branches, then re-raise.

Reviewed commit eebd0ac2320c65cca6a5ce981aa80683fb760640. Workflow run

[yaythomas]

f"{__module__}.{__qualname__}"?

Also, so only this one type gets a namespaced token and eight others get bare class names? So the ErrorType field now carries two conventions.

You could argue this is the only discriminant the SDK branches on to choose a raised type, so this bare-names for the others are fine and this one is special?

If you want to go the other way for consistency, a relatively light way of enforcing this for the others is something like this in ErrorObject.from_exception?

cls_ = type(exception)
module: str = "" if cls_.__module__ == "builtins" else f"{cls_.__module__}."
wire_type: str = f"{module}{cls_.__qualname__}"

type=wire_type"

This would give you ValueError rather than builtins.ValueError, and everything else full.y qualified.

this would also mean to change
_DURABLE_OPERATION_ERROR_REGISTRY to derived from the classes rather than typed by hand.

And raise_as_operation_error's comparison to use the same derived expression for SerDesError.

[ayushiahjolia]

We can do this to make it consistent, but should be a separate change.
And this change means we are accepting that python will differ in convention with other SDKs.

[github-actions]

Claude AI review

Review: fix: round-trip serdes result on first run across ops

I reviewed the complete SHA-anchored diff plus the surrounding source (step.py, child.py, wait_for_condition.py, concurrency/executor.py, serdes.py, exceptions.py, lambda_service.py, execution.py, map.py, examples).

Verified correct

• Round-trip before SUCCEED checkpoint in step.py, child.py, wait_for_condition.py: a permanent SerDesError in the deserialize half writes FAIL and never SUCCEED (no double-checkpoint), and the first-run return value is identical to the replay path. None-payload short-circuits mirror replay.
• Large-payload / ReplayChildren: returned value uses the full serialized_result; only the summary is checkpointed. First run and replay both return deserialize(serialize(raw)) — confirmed equal.
• Virtual / FLAT branches now round-trip results (intentional, documented change; previously-unserializable results now raise SerDesError); replay re-executes and round-trips identically.
• Error taxonomy: SerDesError (permanent, catchable DurableExecutionsError) vs RetryableSerDesError (transient, retryable InvocationError) correctly routed. serialize/deserialize propagate the retryable variant and wrap everything else as SerDesError. ErrorObject/raise_as_operation_error surface SerDesError as itself via WIRE_ERROR_TYPE (not bare class name) on first run and replay.
• Executor: retryable branch errors escape the batch (backend retry), take priority over suspension, and re-raise on the FLAT-replay path; permanent SerDesError becomes a per-operation failed item with the correct wire type. New InvocationError import is used, not duplicated.
• lambda_service str | None widening consistent across all create_*_succeed/_retry/_start sites; handles serdes returning None.
• Examples: deleted step_with_custom_serdes fully removed (source, test, catalog); consolidated serdes_roundtrip example is consistent with the namespace-package layout.

No actionable code findings.

Residual test risk

The map/parallel parent-level BatchResult round-trip (a changed behavior — these ops now return a round-tripped BatchResult) is covered only by mock-based unit tests (tests/operation/map_test.py:1224, parallel_test.py:1245). Unlike step/run_in_child_context/wait_for_condition, there is no end-to-end first-run-vs-replay equality test with a real non-identity serdes for map/parallel, and the serdes_roundtrip example omits them. A custom MapConfig/ParallelConfig serdes must now round-trip (not just serialize) a BatchResult; that path lacks integration coverage. Coverage gap, not a defect.

Reviewed commit eebd0ac2320c65cca6a5ce981aa80683fb760640. Workflow run