Add --move-tables mode - #1755
Open
ericyan wants to merge 23 commits into
Open
Conversation
Add move-table command parsing and validate the table list input. Initialize migration context from the CLI configuration. Refs: #1702
Build range-bounded copy queries and prepare target-table DML queries. Apply copy batches through the applier and invoke them from the migrator. Refs: #1703
Enable end-to-end target-table copying, route DML events to the target, and harden move-table initialization and table selection. Refs: #1705 Co-authored-by: Chris Kirkland <chriskirkland@github.com> Co-authored-by: Daniel Joos <danieljoos@github.com> Co-authored-by: womoruyi <womoruyi@github.com>
Implement the T0-T6 cooperative cutover workflow and cover atomic target-table cutover behavior. Refs: #1704 Co-authored-by: Zach Sierakowski <zacharysierakowski@github.com>
Create target tables in the destination database and abort safely when target tables already exist. Refs: #1710
Use target-cluster credentials and replica status for control-replica throttling during move-table migrations. Refs: #1709
Expose move-table source, target, and migration details to lifecycle hooks. Refs: #1711
Initialize move-table configuration consistently before migration setup. Refs: #1712
Enable the throttler in move-table mode with the move-table-specific configuration. Refs: #1713
Persist per-table checkpoints and drain GTIDs so copying and cutover can resume safely after interruption. Refs: #1708
Remove target artifacts and migration state after successful or failed move-table operations. Refs: #1717
Add local integration scenarios for single-table copying and concurrent writes. Refs: #1714
Restore the move-table drain GTID when resuming an interrupted migration. Refs: #1720
Introduce source-primary connection configuration and guard primary-required move-table operations. Refs: #1718
Update status reporting and lag measurement for move-table migrations. Refs: #1721
Add failpoint-driven integration coverage for copy, drain, hook, and cutover recovery. Refs: #1723
Extend migration state, copying, checkpointing, draining, and atomic rename to ordered sets of tables. Refs: #1726
Exclude generated columns from move-tables writable column lists during initial setup and resume. Use the filtered columns for row-copy queries while preserving the complete column layout for binlog event ordinals.
Populate writable column metadata for fresh and resumed move-table migrations so JSON, unsigned, and binary values are encoded correctly.
Exercise row copy and concurrent DML with signed, unsigned, and maximum BIGINT UNSIGNED values.
ericyan
force-pushed
the
feature/move-tables
branch
from
August 18, 2026 15:36
5acafd2 to
f830fe0
Compare
ericyan
added
feature/move-tables
ericyan
marked this pull request as ready for review
August 20, 2026 07:29
Contributor
There was a problem hiding this comment.
Pull request overview
Adds multi-table, GTID-based migration between MySQL clusters, including target-side copying, cooperative cutover, recovery, throttling, hooks, and integration coverage.
Changes:
- Adds move-table configuration, copy/application flow, and atomic cutover.
- Adds checkpoint-based recovery and multi-table runtime state.
- Adds dual-cluster integration tooling and failpoint tests.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/move-tables-tests.yml |
Runs move-table integration tests. |
.gitignore |
Ignores generated test artifacts. |
doc/hooks.md |
Documents move-table hook variables. |
go.mod |
Adds failpoint dependency. |
go.sum |
Records dependency checksums. |
go/base/context.go |
Defines move-table configuration and state. |
go/base/context_test.go |
Tests move-table context behavior. |
go/binlog/binlog_entry.go |
Extends binlog event handling. |
go/binlog/gomysql_reader.go |
Supports move-table GTID streaming. |
go/cmd/gh-ost/main.go |
Adds and validates CLI options. |
go/logic/applier.go |
Implements target writes, locking, and checkpoints. |
go/logic/applier_test.go |
Tests target-side application behavior. |
go/logic/checkpoint.go |
Adds multi-table checkpoint state. |
go/logic/checkpoint_test.go |
Tests checkpoint serialization. |
go/logic/hooks.go |
Exposes move-table hook context. |
go/logic/hooks_test.go |
Tests hook variables. |
go/logic/inspect.go |
Inspects each source table. |
go/logic/migrator.go |
Orchestrates copy, recovery, and cutover. |
go/logic/migrator_move_tables_cleanup_test.go |
Tests post-migration cleanup. |
go/logic/migrator_move_tables_cutover_test.go |
Tests cooperative cutover. |
go/logic/migrator_test.go |
Extends migrator coverage. |
go/logic/progress_snapshot.go |
Reports multi-table progress. |
go/logic/server.go |
Exposes move-table status. |
go/logic/test_utils_test.go |
Adds move-table test utilities. |
go/logic/throttler.go |
Adds target-cluster throttling. |
go/logic/throttler_test.go |
Tests target throttling. |
go/mysql/utils.go |
Adds MySQL connection support. |
go/sql/builder.go |
Builds split copy and checkpoint queries. |
go/sql/builder_test.go |
Tests new query builders. |
localtests/docker-compose-move-tables.yml |
Defines dual-cluster test topology. |
localtests/move-tables-test.sh |
Implements integration-test harness. |
localtests/move-tables/atomic-multi-table-cutover/create.sql |
Creates atomic-cutover fixtures. |
localtests/move-tables/atomic-multi-table-cutover/tables.txt |
Lists atomic-cutover tables. |
localtests/move-tables/atomic-multi-table-cutover/test.sh |
Exercises atomic cutover. |
localtests/move-tables/generated-columns/create.sql |
Creates generated-column fixtures. |
localtests/move-tables/generated-columns/tables.txt |
Lists generated-column tables. |
localtests/move-tables/json/create.sql |
Creates JSON fixtures. |
localtests/move-tables/json/tables.txt |
Lists JSON tables. |
localtests/move-tables/multiple-three-concurrent-writes/create.sql |
Creates concurrent multi-table fixtures. |
localtests/move-tables/multiple-three-concurrent-writes/on_test.sh |
Generates concurrent writes. |
localtests/move-tables/multiple-three-concurrent-writes/tables.txt |
Lists concurrent test tables. |
localtests/move-tables/multiple-three/create.sql |
Creates three-table fixtures. |
localtests/move-tables/multiple-three/tables.txt |
Lists three-table fixtures. |
localtests/move-tables/multiple-two/create.sql |
Creates two-table fixtures. |
localtests/move-tables/multiple-two/tables.txt |
Lists two-table fixtures. |
localtests/move-tables/resume-panic-before-drain-complete/create.sql |
Creates drain-recovery fixtures. |
localtests/move-tables/resume-panic-before-drain-complete/tables.txt |
Lists drain-recovery tables. |
localtests/move-tables/resume-panic-before-drain-complete/test.sh |
Tests recovery during drain. |
localtests/move-tables/resume-panic-before-on-success-hook/create.sql |
Creates hook-recovery fixtures. |
localtests/move-tables/resume-panic-before-on-success-hook/hooks/gh-ost-on-success |
Provides success hook fixture. |
localtests/move-tables/resume-panic-before-on-success-hook/tables.txt |
Lists hook-recovery tables. |
localtests/move-tables/resume-panic-before-on-success-hook/test.sh |
Tests recovery before success hook. |
localtests/move-tables/resume-panic-on-row-copy/create.sql |
Creates row-copy recovery fixtures. |
localtests/move-tables/resume-panic-on-row-copy/tables.txt |
Lists row-copy recovery tables. |
localtests/move-tables/resume-panic-on-row-copy/test.sh |
Tests row-copy recovery. |
localtests/move-tables/single-concurrent-writes/create.sql |
Creates concurrent single-table fixture. |
localtests/move-tables/single-concurrent-writes/on_test.sh |
Generates concurrent writes. |
localtests/move-tables/single-concurrent-writes/tables.txt |
Lists the concurrent table. |
localtests/move-tables/single-with-hooks/create.sql |
Creates hook test fixture. |
localtests/move-tables/single-with-hooks/hooks/gh-ost-on-before-cut-over |
Provides pre-cutover hook. |
localtests/move-tables/single-with-hooks/hooks/gh-ost-on-row-copy-complete |
Provides row-copy hook. |
localtests/move-tables/single-with-hooks/hooks/gh-ost-on-success |
Provides success hook. |
localtests/move-tables/single-with-hooks/hooks/util.sh |
Shares hook utilities. |
localtests/move-tables/single-with-hooks/tables.txt |
Lists hook test table. |
localtests/move-tables/single-with-hooks/test.sh |
Tests hook execution. |
localtests/move-tables/single/create.sql |
Creates basic fixture. |
localtests/move-tables/single/tables.txt |
Lists basic fixture table. |
localtests/move-tables/unsigned/create.sql |
Creates unsigned-column fixtures. |
localtests/move-tables/unsigned/tables.txt |
Lists unsigned-column tables. |
localtests/test.sh |
Integrates move-table tests. |
script/docker-gh-ost-move-tables-tests |
Manages CI test containers. |
script/move-tables/README.md |
Documents local test tooling. |
script/move-tables/insert-source-primary-loop |
Generates source writes. |
script/move-tables/mysql-source-primary |
Connects to source primary. |
script/move-tables/mysql-source-replica |
Connects to source replica. |
script/move-tables/mysql-target-primary |
Connects to target primary. |
script/move-tables/mysql-target-replica |
Connects to target replica. |
script/move-tables/reset |
Resets test clusters. |
script/move-tables/setup |
Creates local test environment. |
script/move-tables/teardown |
Removes local test environment. |
vendor/github.com/pingcap/failpoint/.codecov.yml |
Vendors failpoint coverage configuration. |
vendor/github.com/pingcap/failpoint/.gitignore |
Vendors failpoint ignore rules. |
vendor/github.com/pingcap/failpoint/CONTRIBUTING.md |
Vendors contribution guidance. |
vendor/github.com/pingcap/failpoint/LICENSE |
Vendors dependency license. |
vendor/github.com/pingcap/failpoint/MAINTAINERS.md |
Vendors maintainer metadata. |
vendor/github.com/pingcap/failpoint/Makefile |
Vendors failpoint build targets. |
vendor/github.com/pingcap/failpoint/README.md |
Vendors failpoint documentation. |
vendor/github.com/pingcap/failpoint/failpoint.go |
Vendors core failpoint implementation. |
vendor/github.com/pingcap/failpoint/failpoints.go |
Vendors failpoint registry. |
vendor/github.com/pingcap/failpoint/http.go |
Vendors failpoint HTTP support. |
vendor/github.com/pingcap/failpoint/marker.go |
Vendors failpoint markers. |
vendor/github.com/pingcap/failpoint/terms.go |
Vendors failpoint expression terms. |
vendor/modules.txt |
Registers vendored dependency. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 78/93 changed files
- Comments generated: 4
- Review effort level: Balanced
Comment on lines
+287
to
+294
| if apl.migrationContext.IsMoveTablesMode() { | ||
| lockTable = "movetables." + apl.migrationContext.MoveTablesRunToken() | ||
| lockSubject = fmt.Sprintf("tables %v", apl.migrationContext.MoveTables.TableNames) | ||
| } else { | ||
| lockTable = apl.originalTableName() | ||
| lockSubject = fmt.Sprintf("`%s`.`%s`", apl.migrationContext.DatabaseName, apl.originalTableName()) | ||
| } | ||
| lockName := buildMigrationLockName(apl.migrationContext.GetTargetDatabaseName(), lockTable) |
Comment on lines
+3072
to
+3077
| applied := coords.SmallerThanOrEquals(mgtr.applier.CurrentCoordinates) | ||
| idle := len(mgtr.applyEventsQueue) == 0 && (mgtr.eventsStreamer == nil || len(mgtr.eventsStreamer.eventsChannel) == 0) | ||
| if applied || idle { | ||
| if !applied { | ||
| mgtr.applier.CurrentCoordinates = coords.Clone() | ||
| } |
Comment on lines
+1506
to
+1509
| if mgtr.migrationContext.Checkpoint { | ||
| if err := mgtr.persistMoveTablesCutOverCheckpoint(drainGTID, false); err != nil { | ||
| return fmt.Errorf("failed to persist move-tables cutover checkpoint: %w", err) | ||
| } |
Comment on lines
+1284
to
+1286
| if err := mgtr.applier.CreateCheckpointTable(); err != nil { | ||
| mgtr.migrationContext.Log.Errorf("unable to create checkpoint table, see further error details") | ||
| } |
danieljoos
approved these changes
Aug 20, 2026
|
|
||
| // moveTablesCutOver orchestrates the cooperative cutover protocol for move-tables | ||
| // mode. It implements the T0-T6 transitions described in | ||
| // docs/learning/design-refs/coop_cutover.md §1.3. |
Contributor
There was a problem hiding this comment.
there are still some references to our internal poc docs in here. e.g. all the § refs.
Maybe it would also make sense to add a condensed version of the design document to the docs folder to describe what the move-tables mode actually does and how the cutover process is supposed to work.
Contributor
|
This is intriguing; can you perhaps elaborate on how this compares to ghostferry and the rationale for building this into gh-ost rather than as a standalone tool? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds support for a new move table mode, in which one or more tables are copied from a source MySQL cluster to a target MySQL cluster.
Related issue: #1681
script/cibuildreturns with no formatting errors, build errors or unit test errors.Component PRs
Most of the PRs for this feature were created before Stacked PR were available. In addition to the PR stack, this feature branch also consolidates the following move-tables PRs:
Core migration flow
Cutover and recovery
Throttling, hooks, and cleanup
Integration coverage and follow-up