Conversation
…ferred-block-proving
bobbinth
left a comment
There was a problem hiding this comment.
Looks good! Thank you! I left some comments inline - most are pretty small and some should be left for follow-up PRs.
There was a problem hiding this comment.
Maybe not for this PR, but it would be great to add doc comments (maybe at the module level), explaining how we save raw block data. I believe currently it is something like this:
- Raw block data is saved under
[store_dir]/[epoch]/block_[block_num].datfile. - For proven blocks, the proof data is saved under
[store_dir]/epoch[epoch_num]/proof_[block_num].datfile.
Where epoch_num is the 16 most significant bits of the block number, and both epoch_num and block_num are formatted as hex strings.
| pub async fn apply_block( | ||
| &self, | ||
| signed_block: SignedBlock, | ||
| proving_inputs: Option<BlockProofRequest>, |
There was a problem hiding this comment.
We should probably explain this in the doc comment above.
Also, not from this PR, but is the TODO on line 42 still relevant?
| BlockRange block_range = 1; | ||
|
|
||
| // The finality level to use when clamping the upper bound of the block range. | ||
| // | ||
| // When set to FINALITY_COMMITTED (default), the upper bound is clamped to the chain tip. | ||
| // When set to FINALITY_PROVEN, the upper bound is clamped to the latest proven block. | ||
| Finality finality = 2; |
There was a problem hiding this comment.
I still think it would be good to make finality and block_to mutually exclusive - but I would leave this for a follow-up PR.
To illustrate the idea, in Rust, it would look something like this:
struct SyncChainMmrRequest {
block_from: u32,
block_to: UpperBlockBound,
}
// the name is just illustrative
enum UpperBlockBound {
Block(u32),
LastCommitted,
LastProven,
}…ferred-block-proving
bobbinth
left a comment
There was a problem hiding this comment.
Looks good! Thank you! I left a few comments inline. The main one is about encapsulating more logic in the mark_block_proven operation and making it "atomic".
bobbinth
left a comment
There was a problem hiding this comment.
Looks good! Thank you! I left a couple more small comments inline.
Also, I didn't review in depth the test code and some of the tokio-related scheduling code - so, another set of eyes on that would be good.
Context
We are adding deferred (asynchronous) block proving for the node, as described #1592. Currently, block proving happens synchronously during
apply_block, which means block commitment is blocked until the proof is generated.Blocks will now exhibit committed (not yet proven) and proven states. A committed block is already part of the canonical chain and fully usable. Clients that require proof-level finality can opt into it via the new
finalityparameter onSyncChainMmr.Changes
proving_inputs BLOBto theblock_headerstable, with partial index for querying proven (proving_inputs = NULL) blocks. Addedproven_in_sequence BOOLEANcolumn to track whether a block and all its ancestors have been proven, with a partial index for efficient lookups.BlockStore(following the existing block file pattern) rather than as BLOBs in SQLite.select_block_proving_inputs(returns deserializedBlockProofRequest),clear_block_proving_inputs,select_proven_not_in_sequence_blocks,mark_blocks_as_proven_in_sequence,select_unproven_blocks, andselect_latest_proven_in_sequence_block_num. Block proving business logic (tip walking, contiguity detection) lives inDb::mark_proven_and_advance_sequenceindb/mod.rs, composing these queries within a single transaction.apply_block: TheBlockProofRequestis now serialized and persisted alongside the block duringapply_block. Removed theApplyBlockDatastruct, replacing it with a&SignedBlockparameter plusnotesandproving_inputs.proof_scheduler.rs) that drives deferred proving. It queries unproven blocks on startup (restart recovery), listens for new block commits via awatchchannel, and proves blocks concurrently up to a configurable limit. Because blocks may be proven out of order, the scheduler maintains aBTreeSetof proven-but-not-yet-in-sequence blocks and advances a contiguous watermark as gaps fill, updating theproven_in_sequencecolumn for exactly the newly in-sequence blocks.SyncChainMmr: Added aFinalityenum (COMMITTED,PROVEN) to the protobuf and afinalityfield onSyncChainMmrRequest. WhenPROVENfinality is requested, the effective tip is the highest contiguously proven block.