Skip to content

feat(db): compress Receipts rows with zstd#465

Merged
kariy merged 5 commits intomainfrom
feat/receipt-compression
Mar 11, 2026
Merged

feat(db): compress Receipts rows with zstd#465
kariy merged 5 commits intomainfrom
feat/receipt-compression

Conversation

@kariy
Copy link
Copy Markdown
Member

@kariy kariy commented Mar 9, 2026

Summary

  • Store new Receipts rows as an explicit envelope plus zstd(postcard(receipt)) while keeping reads compatible with legacy raw-postcard rows.
  • Treat malformed enveloped rows as corruption instead of silently falling back to the legacy decoder.
  • Add focused codec and Receipts table round-trip tests, and document the mixed-format receipt storage in docs/database.md.

Compatibility

  • No db.version bump in this phase.
  • New Katana code can read existing databases that still contain legacy raw-postcard receipt rows.
  • Older Katana binaries are not guaranteed to read databases after compressed receipt rows have been written.

@kariy kariy changed the title Compress receipt rows with zstd feat(db): compress Receipts rows with zstd Mar 9, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 9, 2026

Codecov Report

❌ Patch coverage is 99.29078% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 68.71%. Comparing base (9bde0ae) to head (b76ca30).
⚠️ Report is 300 commits behind head on main.

Files with missing lines Patch % Lines
crates/storage/db/src/models/receipt.rs 99.05% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #465      +/-   ##
==========================================
- Coverage   73.32%   68.71%   -4.62%     
==========================================
  Files         209      280      +71     
  Lines       23132    34143   +11011     
==========================================
+ Hits        16961    23460    +6499     
- Misses       6171    10683    +4512     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kariy kariy force-pushed the feat/receipt-compression branch from 8c779e4 to b76ca30 Compare March 10, 2026 22:24
@kariy kariy merged commit cb2071c into main Mar 11, 2026
10 of 11 checks passed
@kariy kariy deleted the feat/receipt-compression branch March 11, 2026 18:38
@kariy
Copy link
Copy Markdown
Member Author

kariy commented Mar 12, 2026

Preliminary test done on a database filled with mainnet data of about ~3M blocks, we're able to reduce the Receipts table size by about ~30%.

kariy added a commit that referenced this pull request Mar 12, 2026
Extracts the envelope logic introduced in #465 for `ReceiptEnvelope` into a generic `Envelope<T>` parameterized by an `EnvelopePayload` trait, then applies the same zstd compression to the `Transactions` table via a new `TxEnvelope` type alias.

Each payload type supplies a 4-byte magic (`KRCP` for receipts, `KTXN` for transactions), a human-readable name for error messages, and a `from_legacy_bytes` function for backward-compatible deserialization of pre-envelope rows. The shared `Compress`/`Decompress` impls handle the header, zstd encode/decode, version/encoding validation, and legacy fallback.

The `Transactions` table value type changes from `VersionedTx` to
`TxEnvelope`. Existing uncompressed rows are transparently decoded via
the legacy fallback in `VersionedTx::from_legacy_bytes`, so no migration
is needed. However, any code that reads directly from the `Transactions`
table outside of the provider (eg. custom tooling) will need to update
to handle `TxEnvelope` instead of bare `VersionedTx`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant