Skip to content

[FIX] [RPC] Align getrawtransaction with Bitcoin Core#985

Merged
Davidson-Souza merged 7 commits into
getfloresta:masterfrom
moisesPompilio:get_raw_transaction
Jun 30, 2026
Merged

[FIX] [RPC] Align getrawtransaction with Bitcoin Core#985
Davidson-Souza merged 7 commits into
getfloresta:masterfrom
moisesPompilio:get_raw_transaction

Conversation

@moisesPompilio

Copy link
Copy Markdown
Collaborator

Description and Notes

Brings Floresta’s getrawtransaction RPC closer to Bitcoin Core compatibility.

Fixes the RPC method call so the client correctly uses getrawtransaction instead of the incorrect gettransaction, and standardizes verbosity handling to match Bitcoin Core’s numeric behavior.

The response format was aligned with Bitcoin Core as well:

  • RawTx fields such as blockhash, confirmations, blocktime, and time are now optional and only serialized when present.
  • txid encoding was corrected.
  • TxOut.value now uses f64 in BTC instead of u64 in sats, matching Bitcoin Core’s denomination.
  • TxOut.address is now optional and only included when the script can be converted into a valid address.
  • TxIn now handles coinbase inputs properly, including the coinbase field only when applicable.
  • ASM formatting was improved to follow Bitcoin Core’s rules more strictly.
  • TxOut script types were expanded to cover the script types supported by Bitcoin Core.

Note: the desc field was intentionally left out, since matching Bitcoin Core’s descriptor logic requires additional investigation and a more complex implementation. That can be handled in a future PR.

How to verify the changes you have done?

Compare the output of getrawtransaction between Floresta and Bitcoin Core for both verbosity = 0 and verbosity = 1.

The integration test added here already covers this behavior and is the best reference for validation.

  • Check regular transactions across multiple script types, including P2PKH, P2WPKH, P2SH, and P2TR.
  • Check coinbase transactions and confirm the special input/output fields are handled correctly.
  • Review the optional field behavior to ensure fields are only present when they should be.
  • Inspect the integration test to confirm field-by-field validation against Bitcoin Core.

@moisesPompilio

Copy link
Copy Markdown
Collaborator Author

This PR is in draft because it depends on #821. That PR introduces typos for spell checking, allowing more flexibility when using randomly generated addresses without triggering spelling errors

@moisesPompilio moisesPompilio self-assigned this Apr 23, 2026
@moisesPompilio moisesPompilio added the ecosystem Enable interoperability, compatibility and practical integration with the broader Bitcoin ecosystem label Apr 23, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in Floresta Apr 23, 2026
@moisesPompilio moisesPompilio moved this from Backlog to In progress in Floresta Apr 23, 2026
@jaoleal jaoleal self-requested a review April 23, 2026 10:46
@moisesPompilio moisesPompilio force-pushed the get_raw_transaction branch 3 times, most recently from 9547e4a to c9ac410 Compare May 15, 2026 01:55
@moisesPompilio moisesPompilio marked this pull request as ready for review May 15, 2026 01:59
@moisesPompilio

Copy link
Copy Markdown
Collaborator Author

22e29a2 I rebased and converted the added integration test to the pytest format.

@jaoleal jaoleal moved this from In progress to Needs review in Floresta May 15, 2026
Comment thread bin/floresta-cli/src/main.rs Outdated
#[command(name = "gettransaction")]
GetTransaction { txid: Txid, verbose: Option<bool> },
#[command(name = "getrawtransaction")]
GetRawTransaction { txid: Txid, verbose: Option<u32> },

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think that Bitcoin Core will appear with 4 294 967 295 levels of verbosity

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but u32 is the default for numeric verbosity that we use.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u32 occupies too much space for something simple as a verbosity level.

Also: #1076

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced the u32 with a u8.

Comment on lines +99 to 106
match verbosity {
0 => serde_json::to_value(serialize_hex(&tx.tx))
.map_err(|e| JsonRpcError::Decode(e.to_string())),
1 => serde_json::to_value(self.make_raw_transaction(tx))
.map_err(|e| JsonRpcError::Decode(e.to_string())),
_ => return Err(JsonRpcError::InvalidVerbosityLevel),
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return a structured type instead of a generic

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


/// The information returned by a get_raw_tx
#[derive(Deserialize, Serialize)]
pub struct RawTxJson {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you tried to use corepc ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the GetRawTransaction of corepc is not good, the coinbase field of the input is missing, and in addition, old fields are present.

@csgui csgui added this to the Q2/2026 milestone May 18, 2026
@Davidson-Souza

Copy link
Copy Markdown
Member

nit: the bang (!) should come before the column (:) in your commit titles. fix(rpc)!:

@Davidson-Souza Davidson-Souza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first code review

Comment on lines +90 to 92
if verbosity > 1 {
return Err(JsonRpcError::InvalidVerbosityLevel);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

03bae40

nit: Since there's a wildcard inside the match below, this check is useless.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I didn’t want the verbosity check to be after getting the transaction, but it doesn’t really matter, since it would only save a few milliseconds in the process. So I removed this if; the validation is in the match now.

Comment on lines +81 to +85
pub enum GetRawTransactionRes {
Zero(String),

One(Box<RawTxJson>),
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b07a232

nit:

Suggested change
pub enum GetRawTransactionRes {
Zero(String),
One(Box<RawTxJson>),
}
pub enum GetRawTransactionRes {
Zero(String),
One(Box<RawTxJson>),
}

Also, you Box-ed it because it is too big?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we are adopting Box for the response enums because the verbosity objects tend to be very large.

}

/// The information returned by a get_raw_tx
#[derive(Debug, Deserialize, Serialize)]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b07a232

Suggested change
#[derive(Debug, Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
/// The information returned by a get_raw_tx

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jaoleal jaoleal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 7331f65

@Davidson-Souza

Copy link
Copy Markdown
Member

Needs rebase

@moisesPompilio

Copy link
Copy Markdown
Collaborator Author

d0b8750 I’ve rebased

@Davidson-Souza Davidson-Souza requested a review from jaoleal June 5, 2026 13:09
Comment thread bin/floresta-cli/src/main.rs Outdated
#[command(name = "gettransaction")]
GetTransaction { txid: Txid, verbose: Option<bool> },
#[command(name = "getrawtransaction")]
GetRawTransaction { txid: Txid, verbose: Option<u8> },

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verbosity (in other places as well).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

let raw_tx = self.make_raw_transaction(tx)?;
serde_json::to_value(raw_tx).map_err(|e| JsonRpcError::Decode(e.to_string()))
}
_ => Err(JsonRpcError::InvalidVerbosityLevel),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think core always return the maximum known level if the number isn't defined — if I pass 3, it returns the same as 1

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, in getrawtransaction, Bitcoin Core returns 0 as the default value, according to the documentation.
https://bitcoincore.org/en/doc/30.0.0/rpc/rawtransactions/getrawtransaction/

}

#[derive(Debug, Deserialize, Serialize)]
/// The information returned by a get_raw_tx

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't corepc have those types?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, but the GetRawTransaction of corepc is not good, the coinbase field of the input is missing, and in addition, old fields are present.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps one an issue and leave a TODO here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the newer versions of corepc, and the getrawtransaction structure has been improved. So I updated to the latest version and used its struct here.

/// The weight of this transaction, as defined by the segwit soft-fork
pub weight: u32,

/// This transaction's version. The current bigger version is 2

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, three (TRUC transactions)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

#[derive(Debug, Deserialize, Serialize)]
/// The locking script inside a txout
pub struct ScriptPubKeyJson {
/// A ASM representation for this script

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An, no?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

.witness
.iter()
.map(|w| w.to_hex_string(bitcoin::hex::Case::Upper))
.map(|w| w.to_hex_string(bitcoin::hex::Case::Lower))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hex::Case::Lower

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


/// TODO: This function is not compliant with Bitcoin Core.
/// See: <https://github.com/getfloresta/Floresta/issues/987>
pub(super) fn get_script_type_descriptor(script: &Script, address: &Option<Address>) -> String {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These methods could be an extension trait for Address or Script

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I’m planning to work on that in issue #1108, so this will be a separate PR to add extensions for transaction items, since some parts are shared between floresta-node and floresta-electrum.

Comment thread tests/floresta-cli/getrawtransaction.py Outdated
block = self.bitcoind.rpc.get_block(bestblockhash)
txids.append(block["tx"][0]) # Get the coinbase transaction txid

self.utreexod = utreexod_node

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, Utreexo is only used here to provide transaction proofs to Floresta, so it basically just connects to other peers and performs the IBD. That’s why it is only called in this lower part.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why the bind?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I removed it.

@moisesPompilio moisesPompilio force-pushed the get_raw_transaction branch 2 times, most recently from d4c3dc2 to b99a917 Compare June 17, 2026 22:19
@moisesPompilio

Copy link
Copy Markdown
Collaborator Author

b99a917 Added a helper function to compare fields, which helps in cases where we need to compare responses against Bitcoin Core. Also rebased the branch.

Comment thread crates/floresta-node/src/json_rpc/blockchain.rs Outdated
Comment thread crates/floresta-node/src/json_rpc/server.rs
@moisesPompilio moisesPompilio force-pushed the get_raw_transaction branch 2 times, most recently from a09ec26 to 20b0dad Compare June 18, 2026 19:19
@jaoleal

jaoleal commented Jun 19, 2026

Copy link
Copy Markdown
Member

@moisesPompilio CI broke

@jaoleal jaoleal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions

Comment thread tests/floresta-cli/getrawtransaction.py Outdated
Comment on lines +78 to +85
with pytest.raises(HTTPError):
self.florestad.rpc.get_raw_transaction("nonexistingtxid")

with pytest.raises(HTTPError):
self.florestad.rpc.get_raw_transaction("nonexistingtxid", verbose=0)

with pytest.raises(HTTPError):
self.florestad.rpc.get_raw_transaction("nonexistingtxid", verbose=1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now we have tooling for the framework to precisely assert against rpc errors, dont you wanna use it here ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


txids = []
value = 6.4242521
for address in [

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see other tests that can use a chain with transactions, perhaps we could extract this into a shared helper ? Same thing we did with the wait_until wrapper

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think it's better to leave this for later. I'd like this helper function to be implemented using Floresta's wallet functionality rather than Bitcoin Core's

raise TimeoutError(f"{error_msg} after {timeout} seconds")


def compare_fields(candidate, reference, ignore_fields=None, float_tol=1e-8):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment on lines +165 to +169
for key, ref_value in reference.items():
if key in ignore_fields:
continue
assert key in candidate, f"Missing key in candidate: {key}"
compare_fields(candidate[key], ref_value, ignore_fields=ignore_fields)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THis method doesnt need to be recursive but okay

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it is necessary. We have cases with dicts nested inside other dicts, as well as dicts inside lists that are themselves nested within dicts. That's why the recursion is needed. A direct comparison would not work for those cases.

return

# scalar
assert candidate == reference

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This have the "failfast" behavior right ? itll stop comparing at the first difference, dont we want to know all fields that differ (in a single run)?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is fail-fast because it relies on assertions. As soon as a comparison fails, the entire comparison chain stops.

In this case, collecting all differences is not really feasible because the function is generic and does not know the structure of the data beforehand. For example, a getrawtransaction response contains a top-level dict, a vin field that is a list, and each item in that list is another dict. Since the structure can vary depending on the RPC response being compared, the function recursively traverses the data and fails as soon as it encounters the first mismatch.

Comment on lines +149 to +155
for height in range(initial_block_coinbase_wallet, best_block_height):
block_hash = self.bitcoind.rpc.get_blockhash(height)

block = self.bitcoind.rpc.get_block(block_hash, verbosity=1)
coinbase_tx = block["tx"][0] # Get the coinbase transaction txid

self.compare_getrawtransaction(coinbase_tx)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are checking coinbases twice ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

- `InvalidVerbosityLevel` - Verbosity parameter is not 0 or 1
- `MissingParameter` - The txid parameter was not provided
- `InvalidParameterType` - The txid is not a valid hex string
- `Decode` - Error during transaction decoding or serialization

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I would like to see a note about the requirement of the wallet to track the transaction or we will not find it

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@moisesPompilio moisesPompilio force-pushed the get_raw_transaction branch 2 times, most recently from 0396af7 to b2e9a69 Compare June 22, 2026 21:50
@moisesPompilio moisesPompilio requested a review from jaoleal June 22, 2026 21:59
@moisesPompilio

moisesPompilio commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

b2e9a69 Applied the suggestions made by @jaoleal

…osity to numeric type

The RPC client was incorrectly calling 'gettransaction' (which doesn't exist on the server)
instead of 'getrawtransaction'.

Additionally, the verbosity parameter handling was inconsistent. While the method signature
correctly accepted Option<u32>, it needed better standardization to match Bitcoin Core's
RPC specification.
Bump `corepc-types` to v0.14.0, bringing various updates and fixes to Bitcoin Core RPC type definitions and improving compatibility with newer RPC fields and responses.
…format

This commit aligns the getrawtransaction RPC response with Bitcoin Core's
format and behavior. The following changes were made:

RawTx struct:
- Made blockhash, confirmations, blocktime, and time optional fields
  They are now only serialized when present, avoiding null values
- Fixed txid encoding that was previously inverted

TxOut struct:
- Changed value from u64 (sats) to f64 (BTC) for proper denomination
- Made address field optional, only serialized when script can be
  converted to a valid address

TxIn struct:
- Added coinbase field that only appears in coinbase transactions,
  containing the hex-encoded script from scriptSig
- Coinbase inputs have: coinbase, sequence, witness (optional)
- Regular inputs have: txid, vout, script_sig, sequence, witness (optional)

Script handling:
- Improved ASM (assembly) format conversion to strictly follow Bitcoin Core
  rules and formatting standards

TxOut types:
- Updated to include all script types that Bitcoin Core supports

Note: The `desc` field was not implemented due to complex matching rules.
This will be added in a future PR.
Add a helper for recursively comparing RPC responses while ignoring selected
fields. Use it in the `getblockchaininfo`, `getblockheader`, `gettxout` and `getblock` tests to simplify response
validation between Floresta and Bitcoin Core.
Merge `noraise_raw_request` and `noraise_raw_data_request` into a single
`noraise_raw_request` helper. The new implementation automatically handles
dictionary payloads using the previous JSON-RPC behavior and falls back to
the raw data request path for non-dictionary payloads.

Move commonly used RPC helper functions into `BaseRPC` so they are available
to all nodes without requiring additional imports. Also remove redundant
helper methods from the RPC base implementation.
Add comprehensive integration test comparing Floresta's `getrawtransaction`
RPC output with Bitcoin Core for verbosity levels 0 and 1, covering:
- Regular transactions (`P2PKH`, `P2WPKH`, `P2SH`, `P2TR types`)
- Coinbase transactions
- Input/output field validation
- Optional field handling

New BaseRPC methods:
- `get_raw_transaction()`

New BitcoinRPC methods:
- `create_wallet()`
- `get_new_address()`
- `generate_block_to_wallet()`
- `send_to_address()`
@moisesPompilio

Copy link
Copy Markdown
Collaborator Author

eb13fe8 I rebased

@jaoleal jaoleal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK eb13fe8

@Davidson-Souza Davidson-Souza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK eb13fe8

@luisschwab luisschwab left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK eb13fe8

@Davidson-Souza Davidson-Souza merged commit 73e748e into getfloresta:master Jun 30, 2026
14 checks passed
@github-project-automation github-project-automation Bot moved this from Needs review to Done in Floresta Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem Enable interoperability, compatibility and practical integration with the broader Bitcoin ecosystem

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants