Skip to content

fix(compliance): return null instead of panicking for unavailable blocks - #144

Merged
sprites0 merged 1 commit into
node-builderfrom
fix/compliance-unknown-block-panic
Jul 25, 2026
Merged

fix(compliance): return null instead of panicking for unavailable blocks#144
sprites0 merged 1 commit into
node-builderfrom
fix/compliance-unknown-block-panic

Conversation

@sprites0

@sprites0 sprites0 commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Fixes #143.

Root cause

system_tx_count_for_block assumed the caller had validated block_id, but it runs before any
block lookup on raw user input, so header_by_id returns Ok(None) and the second unwrap()
panics a tokio worker.

The reported trigger turned out not to be pruned/below-earliest blocks. Those return null
via the existing block_by_id guard, and a testnet datadir carries real headers from block 0
anyway (init-state --without-evm writes placeholder headers for every block below the start
height — verified against a public testnet snapshot: static_file_headers_* are populated for
segments 0-99, while transactions/receipts are empty below 34000000_34499999).

The actual trigger is eth_getBlockByNumber("pending"). reth's default PendingBlockKind is
Full, so the EthApi builds a local pending block numbered latest + 1, and adjust_block then
asks the provider for a header that by definition is not in the database.

Fix

system_tx_count_for_block now returns Result<Option<usize>, Eth::Error>: provider errors
propagate as RPC errors and a missing header makes the caller report the block as unknown.
Returning null rather than unfiltered data matters here, since the system tx count is what makes
the response compliant.

Two neighbouring panics on the same paths are fixed as well:

  • adjust_transaction_receipt underflowed meta.index - system_tx_count and panicked on
    .nth(..).unwrap() for a receipt request on any system tx hash, which is hidden in compliant
    mode. Those now return null, as does the block-vanished race that was unreachable!().
  • adjust_log unwrapped the receipts_by_block provider result; the log is now dropped instead
    of panicking the subscription task.

safe/finalized before the CL sets them hit ProviderError::FinalizedBlockNotFound, which
panicked on the first unwrap; that now surfaces as an RPC error.

Why the index arithmetic is not clamped

An earlier revision of this branch wrapped every subtraction in saturating_sub. That was wrong:
clamping would emit transactionIndex: 0 for a transaction that is not index 0, in the one module
whose job is emitting correct indices, and adjust_transaction_receipt would then nth() the
wrong receipt entirely. overflowing_sub is no better unless the flag is acted on, at which point
it is a wordier checked_sub. (count - U256::from(..) never panicked in the first place — ruint
implements Sub as wrapping_sub.)

The second commit removes the need for clamping instead. adjust_block_receipts decided what to
drop with cumulative_gas_used == 0 but computed the adjusted index from
header.extras.system_tx_count. Those are two different definitions of "system transaction",
and there are three in the codebase:

notion definition used by
A header.extras.system_tx_count, a prefix length adjust_block, getBlockTransactionCount*, block_receipts_with_system_txs, the offset in adjust_transaction_receipt
B receipt.cumulative_gas_used == 0 adjust_block_receipts' filter, adjust_log
C gas_price == 0, anywhere in the block evm/config.rs when building the header, eth_getEvmSystemTxsByBlock*

A receipt reads zero only because HlBlockExecutor::execute_transaction skips accumulating gas
for system txs, so the value stays zero for as long as the block's system tx prefix lasts. If a
zero-gas-price tx ever landed after a gas-consuming one, B would not drop it while A counted it —
leaking a system tx receipt to compliant clients and shifting every following index by one.
Partitioning on idx < system_tx_count (what block_receipts_with_system_txs already does) makes
the arithmetic non-negative by construction and stops eth_getBlockReceipts and
eth_getBlockReceiptsWithSystemTx from disagreeing about the same block.

adjust_log (notion B) and eth_getEvmSystemTxsByBlock* (notion C) are left alone here; those
inconsistencies predate this PR and deserve their own change.

Verification

Release builds of nb-20260613 and this branch, run against the same testnet snapshot datadir
with --chain testnet --hl-node-compliant, head at block 49675750.

Both panics reproduce on the unpatched binary and are gone on this branch:

request unpatched this branch
eth_getBlockByNumber("pending") panic at rpc.rs:671:59, client gets Empty reply from server null, 0 panics under 200 concurrent calls
eth_getTransactionReceipt(<system tx>) panic at rpc.rs:662:91 null, 0 panics

The first is identical in line and column to the report in #143. Every other candidate — unknown
hash, future number, block 0x1, safe, finalized, eth_getBlockReceipts,
eth_getTransactionReceipt on an unknown hash — returned null with zero panics on both
binaries.

No regression on the normal path, over two response sets captured from both binaries:

  • 28 blocks that contain system transactions (found by scanning 50k blocks; they are sparse on
    testnet), covering eth_getBlockReceipts, eth_getBlockReceiptsWithSystemTx,
    eth_getBlockByNumber(full), getBlockTransactionCountByNumber, eth_getEvmSystemTxsByBlockNumber,
    eth_getTransactionReceipt for every regular tx, and per-block eth_getLogs — 171 responses,
    byte-identical (d3c69d2062d5dc043cf9772614f43985). This is the set that exercises the
    partitioning change.
  • 16 blocks without system transactions, 109 responses, byte-identical
    (b00c8311855560e8fe0fcefbcb398a92). Note this set says nothing about the partitioning change:
    with system_tx_count == 0 both the old and new filters are no-ops.

tests/run_tests.sh gains the issue-143 checks: seven methods against unknown-hash/future-number
asserting result: null, a pending no-panic check, and a system tx receipt check that discovers
a system tx by walking back from the head. The last one fails on nb-20260613 and passes here.

@sprites0
sprites0 force-pushed the fix/compliance-unknown-block-panic branch from a0ea72b to 4899ea1 Compare July 25, 2026 18:48
`system_tx_count_for_block` assumed the caller had validated `block_id`,
but it is called with raw user input before any block lookup, so
`header_by_id` returns `Ok(None)` for any block the node does not have
locally (unknown, or a locally built pending block numbered `latest + 1`),
and the `unwrap()` panicked a tokio worker.

It now returns `Result<Option<usize>>`: provider errors propagate as RPC
errors, and a missing header makes the caller report the block as
unknown. Returning `null` rather than unfiltered data matters here, since
the system tx count is what makes the response compliant.

Two neighbouring panics on the same paths are fixed as well:

- `adjust_transaction_receipt` underflowed `meta.index - system_tx_count`
  and panicked on `.nth(..).unwrap()` for a receipt request on any system
  tx hash, which is hidden in compliant mode. Those now return null, as
  does the block-vanished race that was `unreachable!()`.
- `adjust_log` unwrapped the `receipts_by_block` provider result; the log
  is now dropped instead of panicking the subscription task.

The index arithmetic is left unclamped. Saturating the subtractions would
serve a wrong transaction or log index instead of failing, which is worse
in a layer whose job is emitting correct indices. Instead the underflow is
made impossible: `adjust_block_receipts` decided what to drop with
`cumulative_gas_used == 0` but computed the adjusted index from
`header.extras.system_tx_count`, two different definitions of "system
transaction". A receipt reads zero only because
`HlBlockExecutor::execute_transaction` skips accumulating gas for system
txs, so the value stays zero for as long as the block's system tx prefix
lasts; a zero-gas-price tx landing after a gas-consuming one would leak a
system tx receipt to compliant clients and shift every following index by
one. It now partitions on `idx < system_tx_count`, which is what
`block_receipts_with_system_txs` already does, so the two methods can no
longer disagree about the same block.

`tests/run_tests.sh` gains the issue-143 checks: seven methods against
unknown-hash/future-number asserting `result: null`, a `pending` no-panic
check, and a system tx receipt check that discovers a system tx by walking
back from the head, skipping when the window has none.

Verified against a testnet snapshot at head 49675750, comparing release
builds of nb-20260613 and this branch: `eth_getBlockByNumber("pending")`
panicked at rpc.rs:671:59 and `eth_getTransactionReceipt` on a system tx
hash at rpc.rs:662:91, both now returning null with zero panics under
repeated and concurrent calls. Responses are byte-identical between the
two binaries over 28 blocks containing system transactions (171 responses)
and 16 blocks without them (109 responses).

Fixes #143
@sprites0
sprites0 force-pushed the fix/compliance-unknown-block-panic branch from 4899ea1 to 7bc699b Compare July 25, 2026 18:55
@sprites0
sprites0 merged commit 07af3bb into node-builder Jul 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic in hl_node_compliance/rpc.rs:671 (system_tx_count_for_block): Option::unwrap() on None for pruned/unknown blocks; node eventually stops ingesting

1 participant