Replace GORM + AutoMigrate with golang-migrate + sqlc

[iplay88keys]

Description

Warning

Upgrading from a version earlier than v0.8.0 is not supported. You must upgrade to v0.8.0+ before upgrading to this version.

GORM's AutoMigrate is additive-only, leaves no migration history, and has no rollback path. This PR replaces it with versioned SQL migrations (golang-migrate) and compile-time type-safe query generation (sqlc).

A new CI check fails if any existing migration file is modified. Migrations are immutable once merged.

What this changes for operators

• Golang-migrate takes over existing tables: existing tables use CREATE TABLE IF NOT EXISTS, so all data is preserved
• Rollback on failure: if a migration fails mid-run, changes are rolled back automatically before the controller exits non-zero
• Migration history: every schema change is now a numbered, named SQL file checked into the repo

Architecture

Migration logic lives in go/core/pkg/migrations and is exported for reuse:

// Apply all pending migrations. Safe to call on every startup — advisory lock
// ensures only one instance runs at a time; others block then find ErrNoChange.
func RunUp(ctx context.Context, url string, migrationsFS fs.FS, vectorEnabled bool) error

app.Start accepts an optional MigrationRunner as its second parameter. Passing nil uses the OSS default (migrations.RunUp with the embedded migrations.FS). Enterprise builds pass their own runner to apply additional tracks:

app.Start(extensions, func(ctx context.Context, postgresURL string, vectorEnabled bool) error {
    if err := migrations.RunUp(ctx, postgresURL, ossmigrations.FS, vectorEnabled); err != nil {
        return fmt.Errorf("oss migrations: %w", err)
    }
    return db.MigratePostgres(ctx, postgresURL, enterprisemigrations.FS)
})

Multi-instance safety

golang-migrate uses a session-level PostgreSQL advisory lock — only one instance runs migrations at a time. Others block, then find ErrNoChange. The lock releases automatically if the process crashes. If a crash leaves a dirty migration state, the next startup detects it and rolls back before retrying.

Testing

Manual upgrade testing was also performed against a live Kind cluster:

• Deployed from main with all default agents running against PostgreSQL
• Seeded data across all tables: sessions, events, tasks, feedback, tools, toolservers, LangGraph checkpoints, vector memories, and CrewAI flow state
• Deployed the currency-converter-agent (LangGraph) and poem-flow-agent (CrewAI Flow) samples to populate lg_checkpoint, lg_checkpoint_write, and crewai_flow_state
• Upgraded to this branch. Migrations ran at controller startup (core migrations applied, vector migrations applied)
• Verified all row counts matched exactly across all 12 tables after upgrade

Schema validation

To validate the migration produces a correct schema in both upgrade and fresh-install paths, pg_dump --schema-only was captured after each scenario and compared:

• Upgrade path: deployed v0.8.0 (GORM) to Kind, captured the baseline schema, then upgraded to this branch and re-dumped. All 12 tables, columns, types, and indexes were preserved exactly; only schema_migrations, vector_schema_migrations, and memory.id DEFAULT gen_random_uuid() were added
• Fresh install path: reset the PVC and ran make helm-install from scratch. The resulting schema was functionally identical to the upgraded schema (column ordering within tables differs only where GORM's struct-field ordering diverged from the migration file, which has no runtime impact)

[chromatic-com]

Important

UI Tests need review – Review now

🟡 UI Tests: 5 visual and accessibility changes must be accepted as baselines
🟡 UI Review: Go review the new and updated UI
Storybook Publish: 108 stories published