Meilisearch sink: Task verification timed out

[PierreMesure]

Related issue

#2056

Description

When applying a backfill with approximately 10 000 rows to a meilisearch index, the latter gets populated very quickly but task verification seems to fail many times for most rows.

I have tried many variants of batch_size (from 2 to 10 000), the sweet spot seem to be around 10. I have also tried to strip my records from almost all their data so meilisearch's indexing is as fast as possible. Still, even after Sequin claims the backfill is completed, after Meilisearch shows the 10 000 records, I still get messages that fail to be delivered for some reason.

Config

• Sequin v0.14.6
• Meilisearch latest

config.yml for Sequin

databases:
  - name: "test"
    username: "postgres"
    password: "xxx"
    hostname: "db"
    database: "postgres"
    port: 5432
    slot_name: "sequin_slot"
    publication_name: "sequin_pub"

functions:
  - name: "test-transform"
    description: "Index a document in Meilisearch"
    type: "transform"
    code: |-
      def transform(_action, record, _changes, _metadata) do
        %{
          "id" => Kernel.to_string(record["id"]),
          "document_id" => Kernel.to_string(record["document_id"]),
          "document_type" => record["document_type"]
        }
      end

sinks:
  - name: "sink"
    database: "athena"
    transform: "test-transform"
    source:
      include_tables:
        - "public.test_table"
    actions:
      - insert
      - update
      - delete
    destination:
      type: "meilisearch"
      endpoint_url: "http://meilisearch:7700"
      index_name: "test_index"
      primary_key: "id"
      api_key: "yyy"
      batch_size: 5
      timeout_seconds: 300

docker-compose.yml

name: supabase

services:
  meilisearch:
    container_name: meilisearch
    image: getmeili/meilisearch:latest
    restart: unless-stopped
    ports:
      - ${MEILISEARCH_PORT:-7700}:7700
    environment:
      MEILI_ENV: ${MEILI_ENV:-development}
      MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-xxx}
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - meilisearch-data:/meili_data
    healthcheck:
      test:
        [
          "CMD",
          "wget",
          "--no-verbose",
          "--tries=1",
          "--spider",
          "http://localhost:7700/health"
        ]
      interval: 10s
      timeout: 5s
      retries: 5

  sequin-redis:
    container_name: sequin-redis
    image: redis:7-alpine
    restart: unless-stopped
    command:
      [
        "redis-server",
        "--appendonly",
        "yes"
      ]
    volumes:
      - sequin-redis-data:/data
    healthcheck:
      test:
        [
          "CMD",
          "redis-cli",
          "ping"
        ]
      interval: 10s
      timeout: 5s
      retries: 5

  sequin:
    container_name: sequin
    image: sequin/sequin:latest
    restart: unless-stopped
    ports:
      - ${SEQUIN_PORT:-7376}:7376
    depends_on:
      sequin-redis:
        condition: service_healthy
    environment:
      SERVER_PORT: 7376
      PG_PORT: ${POSTGRES_PORT}
      PG_HOSTNAME: ${POSTGRES_HOST}
      PG_DATABASE: ${SEQUIN_PG_DATABASE:-${POSTGRES_DB}}
      PG_USERNAME: ${SEQUIN_PG_USERNAME:-postgres}
      PG_PASSWORD: ${SEQUIN_PG_PASSWORD:-${POSTGRES_PASSWORD}}
      REDIS_URL: redis://sequin-redis:6379
      SECRET_KEY_BASE: ${SEQUIN_SECRET_KEY_BASE:-xxx}
      VAULT_KEY: ${SEQUIN_VAULT_KEY:-xxx}
      FEATURE_PROVISION_DEFAULT_USER: ${SEQUIN_FEATURE_PROVISION_DEFAULT_USER:-true}
      FEATURE_ACCOUNT_SELF_SIGNUP: ${SEQUIN_FEATURE_ACCOUNT_SELF_SIGNUP:-true}
      MAX_MEMORY_MB: ${SEQUIN_MAX_MEMORY_MB:-512}
      CONFIG_FILE_PATH: "/config/config.yml"
    volumes:
      - ./sequin/config.yml:/config/config.yml
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:7376/health"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  meilisearch-data:
    name: meilisearch-data
  sequin-redis-data:
    name: sequin-redis-data

Screenshots

Backfill completed within 2.5 minutes:

But some messages are still stuck after 10 attempts:

[dosubot]

This looks like the same bug reported in #2056 [1]. The root cause is a conflict between two layers of retry configuration in the Meilisearch client introduced in PR #2038 [2].

Specifically, the base HTTP request in client.ex sets max_retries: 1 with retry: :transient, which overrides the custom wait_for_task/2 logic that expects 5 retries with exponential backoff [3]. The result is that task verification makes only 1 GET request to /tasks/{task_id} (~1ms) and then gives up — even though Meilisearch tasks typically complete within ~100ms [1].

This is why:

• Your data appears correctly in Meilisearch (the PUT/POST succeeds and Meilisearch indexes it)
• But Sequin reports delivery failures (verification times out before even checking properly)
• Adjusting batch_size or timeout_seconds has minimal effect — timeout_seconds only controls the HTTP receive timeout, not the verification polling window [4]

The regression was introduced between v0.13.9 and v0.13.13 [1].

Workaround: As noted in #2056, you can use a webhook sink with batch_mode=false pointed at your Meilisearch instance instead of the native Meilisearch sink. This bypasses the broken task verification entirely [1].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}