Skip to content

feat(chat-media): archive media this account sent, behind CHAT_MEDIA_ARCHIVE_OUTBOUND - #1182

Merged
rmyndharis merged 1 commit into
mainfrom
feat/outbound-media-archive
Aug 9, 2026
Merged

feat(chat-media): archive media this account sent, behind CHAT_MEDIA_ARCHIVE_OUTBOUND#1182
rmyndharis merged 1 commit into
mainfrom
feat/outbound-media-archive

Conversation

@rmyndharis

Copy link
Copy Markdown
Owner

Summary

The chat-media archive only ever wrote inbound media — archive() had a single call site, on the inbound persist path — so a sent attachment had no durable copy, no S3 portability and no TTL retention. #1172 made sent media retrievable from the inline row copy; this makes it archivable.

Archiving now runs for outbound rows when CHAT_MEDIA_ARCHIVE_OUTBOUND=true is set alongside CHAT_MEDIA_ARCHIVE_ENABLED. It is a sub-flag rather than a mode of its own, so the retention purge and orphan sweep — which only run while archiving is enabled — are guaranteed to be maintaining whatever it writes. Off by default: it doubles storage again for the outbound half.

Refs #1165.

The guards this needed first

Pointing the existing archive() at outbound rows would have been actively harmful, so three guards land with it, all inside archive() so every current and future caller inherits them:

  • URL pointers are refused. A URL-based send stores the URL string, not bytes. Buffer.from('https://example.com/cat.png','base64') does not throw — it yields 18 bytes of noise — and the read endpoint consults the archive before the inline copy. Archiving one would have replaced the correct 404 with a 200 carrying garbage labelled image/png, in a file the orphan sweep treats as referenced and therefore never reaps. This is the third copy of a discriminator the send path and the export controller already apply to this value.
  • Idempotence. An outbound row has two possible writers (the REST/bulk persist and the engine echo), so the same row can reach archive() twice; a row already pointing at a file is left alone rather than orphaning the first one.
  • getMedia matches chatId dialects. An outbound row stores the caller's literal chatId or the engine-neutral form depending on which writer won the persist race — the same duality the inline fallback already resolves. Without this, archived outbound media would be unreadable exactly when the inline copy cannot stand in for it.

Also fixed

Merging a URL-pointer payload onto the engine's own-send echo replaced bytes the gateway had already downloaded (whatsapp-web.js enriches its echo with the real payload), leaving a row that rendered as a bare marker after a reload. Both merge sites now keep the richer copy. This is opportunistic — whatsapp-web.js only, and only when the echo wins the race — not a general fix for URL sends.

Deliberately not included

Pre-fetching a URL send's bytes in MessageService so they could be stored. It is not a localised change: it moves the fetch outside the failSend funnel, so an SSRF block would surface as a raw 500 — potentially leaking the resolved internal address the current path explicitly redacts — instead of the sanitised 400; it bypasses the whatsapp-web.js sticker trustDeclaredType:false anti-forgery branch; it drops the URL-basename filename default; it inflates every URL-send row from ~40 bytes to the full base64 payload, which GET /messages returns verbatim and the export budget charges for; and it would make ~18 unmocked unit tests hit the network. That deserves its own decision, not a footnote in this one.

Surfaces

configuration.ts, env.validation.ts (strict boolean — a typo silently leaves outbound rows unarchived), env-precedence.ts, both compose files, .env.example, env.validation.spec.ts, docs/06 (the "outbound messages are never archived" sentence was rewritten, not patched), docs/07, and the changelog. The controller's OpenAPI descriptions and the five SDK docstrings describe the serving order rather than the configuration, and already name the URL-send case, so they stay accurate and openapi.json is unchanged.

Verification

Nine new tests: three URL-pointer spellings including HTTPS://, idempotence, the dialect-matched lookup, both merge sites keeping downloaded bytes, and the flag/SENT gating on the archive call. Each guard was mutation-checked — removing the flag check, the SENT gate, the compose forward and the checkBool entry each make their own test fail.

Full gate on the rebased tree: lint, format:check, build, tsc --noEmit, 5206 unit tests (run twice), test:scripts (55), and 148 e2e tests.

…ARCHIVE_OUTBOUND

The chat-media archive only ever wrote inbound media: archive() had a single
call site on the inbound persist path, so a sent attachment had no durable
copy, no S3 portability and no TTL retention. It now runs for outbound rows
too when CHAT_MEDIA_ARCHIVE_OUTBOUND is set alongside CHAT_MEDIA_ARCHIVE_ENABLED
(off by default, and a sub-flag rather than a mode of its own so the retention
purge and orphan sweep are guaranteed to be maintaining whatever it writes).

Three guards make that safe, all inside archive() so every caller inherits them:

- A URL-based send stores the URL STRING, not bytes. Buffer.from(url,'base64')
  does not throw — it yields ~18 bytes of noise — and the read endpoint consults
  the archive BEFORE the inline copy, so archiving one would have served garbage
  in place of the correct 404, in a file the orphan sweep treats as referenced.
- The same row can reach archive() from two writers, so a row already pointing
  at a file is left alone rather than orphaning the first one.
- getMedia now matches the caller's chatId dialects: an outbound row stores the
  literal or the neutral form depending on which writer won the persist race,
  the same duality the inline fallback already resolves.

Separately, merging a URL-pointer payload onto the engine's own-send echo
replaced bytes the gateway had already downloaded (wwjs enriches its echo with
the real payload), so those two merge sites now keep the richer copy.
@rmyndharis
rmyndharis force-pushed the feat/outbound-media-archive branch from b9001ab to 0aa7708 Compare August 9, 2026 09:31
@rmyndharis
rmyndharis merged commit 52feebd into main Aug 9, 2026
16 checks passed
@rmyndharis
rmyndharis deleted the feat/outbound-media-archive branch August 9, 2026 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant