Storage-backed mode for LoroDoc — compaction without full in-memory materialization?

[agronbajraktari]

We use Loro for a collaborative editor with a Rust snapshotter that periodically compacts journal entries into a full snapshot. The problem is memory amplification — a ~5 MB serialized document expands to 600+ MB of RAM when loaded into LoroDoc for compaction, making it impractical in memory-constrained environments (e.g., Cloudflare Durable Objects at 128 MB).

Is there any consideration for a storage-backed mode (e.g., backed by SQLite or a trait Storage) where compaction could operate in a streaming/chunked fashion without materializing the entire document in RAM?

Thanks for the amazing work!

[zxch3n]

Can you share the doc that expands from 5MB to 600+ MB of RAM? I'll try to fix this

[agronbajraktari]

Yes ofc. I store the LoroDoc snapshot as a loro blob. Or did you mean sth else?

20k_rectangles+tree.loro.zip

PS: is it possible to batch at the WASM boundary? The CRDT commit itself is batched via doc.commit(), but each individual mutation still crosses the boundary separately. Or am i doign sth wrong?

thanks for the help!

[agronbajraktari]

Also:

Committing 1 shape to a 12k-shape document (1MB export) takes ~35ms of pure WASM time. The WebAssembly.Memory grows to 343MB. The cost appears to scale with document size, not operation size.