Skip to content

[ANCHOR-1162] Add optimistic locking to prevent TOCTOU race in transaction processing#1896

Merged
JiahuiWho merged 7 commits intodevelopfrom
add-optimistic-lock-to-transaction
Mar 18, 2026
Merged

[ANCHOR-1162] Add optimistic locking to prevent TOCTOU race in transaction processing#1896
JiahuiWho merged 7 commits intodevelopfrom
add-optimistic-lock-to-transaction

Conversation

@JiahuiWho
Copy link
Copy Markdown
Contributor

@JiahuiWho JiahuiWho commented Mar 4, 2026

Description

  • Add @Version optimistic locking to JdbcSepTransaction to prevent concurrent modifications from silently overwriting each other
  • Add Flyway migration (V25) to backfill version column on all transaction tables
  • Handle OptimisticLockingFailureException in RpcService and TransactionService with clear retry error messages, and retry logic in TransactionService and rpcService

tests

  • Fix flaky Sep31PlatformApiTests due to race between event delivery and DB write
  • Removed SEP-6 deposit with pending-external status (see comments)

Context

A bounty report identified a TOCTOU race condition: the RPC handler reads a transaction, validates in memory, then saves — with no concurrency control. Two concurrent requests can both pass validation against stale state, causing silent data corruption (e.g., double-refund). Optimistic locking makes the second concurrent save fail explicitly instead.

Testing

  • New RpcServiceTest — verifies OptimisticLockingFailureException returns JSON-RPC error code -32603 with retry message
  • New TransactionServiceTest — verifiesOptimisticLockingFailureException on save is caught and re-thrown as BadRequestException

@JiahuiWho JiahuiWho force-pushed the add-optimistic-lock-to-transaction branch 3 times, most recently from 083987b to 36fce97 Compare March 9, 2026 21:18
@JiahuiWho JiahuiWho force-pushed the add-optimistic-lock-to-transaction branch from 36fce97 to 874faff Compare March 9, 2026 22:21
*/
@Test
@Order(50)
fun `SEP-6 deposit with pending-external status`() {
Copy link
Copy Markdown
Contributor Author

@JiahuiWho JiahuiWho Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer a valid test after implement optimistic lock since step 2 and step 3 are essentially competing with each other. The realistic way is

pending_user_transfer_start -> notify_offchain_funds_sent
pending external -> notify_offchain_funds_received


Which has already been covered by other test cases

@JiahuiWho JiahuiWho marked this pull request as ready for review March 9, 2026 23:07
Copilot AI review requested due to automatic review settings March 9, 2026 23:07
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds optimistic locking and retry/error-handling to prevent TOCTOU races during transaction updates, ensuring concurrent modifications are detected and surfaced as retryable failures instead of silently overwriting state.

Changes:

  • Add @Version to JdbcSepTransaction and a Flyway migration (V25) to add a version column to SEP 6/24/31 transaction tables.
  • Add optimistic-lock retry loops and retryable error messaging in RpcService and TransactionService.
  • Add/adjust tests for optimistic locking handling and tweak reference server event processing to wait for transaction visibility.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
platform/src/main/java/org/stellar/anchor/platform/data/JdbcSepTransaction.java Adds JPA @Version field to enable optimistic locking.
platform/src/main/resources/db/migration/V25__add_optimistic_lock_version.sql Adds version column to transaction tables.
platform/src/main/java/org/stellar/anchor/platform/service/TransactionService.java Retries PATCH transaction updates on optimistic-lock conflicts and maps failures to retryable errors.
platform/src/main/java/org/stellar/anchor/platform/service/RpcService.java Retries RPC processing on optimistic-lock conflicts; clears persistence context between retries.
platform/src/test/kotlin/org/stellar/anchor/platform/service/TransactionServiceTest.kt Adds unit test coverage for optimistic-lock failure mapping to BadRequestException.
platform/src/test/kotlin/org/stellar/anchor/platform/service/RpcServiceTest.kt Adds unit test coverage for optimistic-lock failure mapping to JSON-RPC internal error with retry message.
kotlin-reference-server/src/main/kotlin/org/stellar/reference/event/processor/Sep31EventProcessor.kt Adds retry/wait to reduce flakiness when events arrive before transaction is readable via API.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread platform/src/main/resources/db/migration/V25__add_optimistic_lock_version.sql Outdated
Comment thread platform/src/main/java/org/stellar/anchor/platform/service/RpcService.java Outdated

// Retries on OptimisticLockingFailureException, which occurs when two concurrent requests
// modify the same transaction. All other exceptions propagate immediately.
private Object processRpcCallWithRetry(RpcRequest rpcCall) throws AnchorException {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd return a conflict error to the client instead of silently retrying. A optimistic lock failure means that the state has changed since the RPC call was made. That's not really a transient issue that should be retried automatically since the call can now be invalid. The client should reread the state if they observe this error and decide whether to retry.

Comment thread platform/src/main/resources/db/migration/V25__add_optimistic_lock_version.sql Outdated
marwen-abid and others added 2 commits March 16, 2026 16:56
…#1905)

Defer event publishing to after database transaction commit using a
TransactionSynchronization decorator, preventing race conditions where
consumers receive events before transaction data is visible. Remove the
retry/polling workaround in the reference server's Sep31EventProcessor
that was compensating for this timing issue.
@JiahuiWho JiahuiWho merged commit 7b1c054 into develop Mar 18, 2026
11 checks passed
@JiahuiWho JiahuiWho deleted the add-optimistic-lock-to-transaction branch March 18, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants