bench: replace embedded raw block with configurable block generator by l0rinc · Pull Request #32554 · bitcoin/bitcoin

l0rinc · 2025-05-18T21:39:14Z

Problem: The hardcoded benchmark block 413567 has several issues:

embeds 1.1MB of binary data in the repository (we can't remove it from history, but we can from HEAD)
mined in 2016, predates SegWit v0, Taproot, and modern script types
benchmarks cannot test different transaction patterns or script mixes
updating to a newer block (e.g., #32457) would still add binary bloat and would remain inflexible

Fix: Added configurable bench/block_generator.{h,cpp} that builds fully valid blocks at runtime, providing blocks or raw serialized streams.

Reproducer: You can run all affected tests and benchmarks with a sanity check:

rm -rf build && \
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON && \
cmake --build build -j$(nproc) && \
build/bin/test_bitcoin -t block_generator_deterministic_seeded_output,block_generator_serialization_roundtrip,block_generator_seed_perturbation,block_generator_multiple_seed_sanity && \
build/bin/bench_bitcoin -sanity-check -filter='DeserializeBlockTest|DeserializeAndCheckBlockTest|LoadExternalBlockFile|WriteBlockBench|ReadBlockBench|ReadRawBlockBench|BlockToJsonVerbosity1|BlockToJsonVerbosity2|BlockToJsonVerbosity3|BlockToJsonVerboseWrite|HexStrBench'

This supersedes #32457 by eliminating the need for any hardcoded block data while enabling flexible benchmark scenarios.

DrahtBot · 2025-05-18T21:39:17Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32554.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj, yuvicc, Raimo33, hodlinator

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#35646 (RFC: Separate out runtime errors from BlockValidationState using util::Expected by yuvicc)
#34075 (fees: Introduce Mempool Based Fee Estimation to reduce overestimation by ismaelsadeeq)
#32729 (test, refactor: extract script template helpers and expand sigop coverage by l0rinc)
#29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

laanwj · 2025-05-19T09:18:41Z

Concept ACK

yuvicc · 2025-06-29T15:44:33Z

Concept ACK

This one is pretty useful as I can test with up-to date test_data.

Raimo33 · 2025-10-19T14:37:15Z

Concept ACK for more realistic benchmarks

l0rinc · 2025-12-20T23:19:52Z

Rebased and applied the new block everywhere, removing the binary of block 413567 embedded in the benchmarks.
Let me know what you think.

hodlinator

Concept ACK b5b57cd

Strong problem A-C-K - It's quite bad that we are only benchmarking against a pre-segwit block.

The general direction is promising, I'd mainly prefer slightly more realistic transactions.

Also, since removing the old block data doesn't shrink the size of the repo, maybe we could keep it around together with some of the older benchmarks?

Suggestions branch: master...hodlinator:bitcoin:pr/32554_suggestions
(Corresponds pretty much to the inline comments).

hodlinator · 2025-12-22T20:55:46Z

+DataStream GetBlockData(
+    const CChainParams& chain_params = *CChainParams::RegTest(CChainParams::RegTestOptions{}),
+    const ScriptRecipe& = kWitness,
+    const uint256& seed = {}
+);
+CBlock GetBlock(


Naming should probably indicate work/computation:

Suggested change

DataStream GetBlockData(

const CChainParams& chain_params = *CChainParams::RegTest(CChainParams::RegTestOptions{}),

const ScriptRecipe& = kWitness,

const uint256& seed = {}

);

CBlock GetBlock(

DataStream GenerateBlockData(

const CChainParams& chain_params = *CChainParams::RegTest(CChainParams::RegTestOptions{}),

const ScriptRecipe& = kWitness,

const uint256& seed = {}

);

CBlock GenerateBlock(

Done, extracted this to a separate commit where we're wrapping the previous behavior first to separate the introduction of the generator from applying it.

hodlinator · 2025-12-22T21:01:28Z

 static void LoadExternalBlockFile(benchmark::Bench& bench)
 {
-    const auto testing_setup{MakeNoLogFileContext<const TestingSetup>(ChainType::MAIN)};
+    const auto testing_setup{MakeNoLogFileContext<const TestingSetup>(ChainType::REGTEST)};


My guess is that we are changing networks in order to get a powLimit which allows for trivially finding a valid nonce. Could the commit message spell this out?

Could also replace this assert...

bitcoin/src/bench/block_generator.cpp

Lines 76 to 78 in 0fbbefd

CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec, const uint256& seed)

{

assert(params.IsTestChain());

...with a more direct assert further down, closer to where it's important:

block.hashPrevBlock = params.GenesisBlock().GetHash(); assert(params.GetConsensus().powLimit == uint256{"7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"}); // lowest difficulty block.nBits = UintToArith256(params.GetConsensus().powLimit).GetCompact();

Yeah, that's why I switched to REGTEST. GenerateBlock* does nonce search in setup, and we don't want to do that on MAIN difficulty - updated the code and commit message to spell that out.

I have played with your suggestions but it seemed to me the assert(chain_params.IsTestChain()) in BuildBlock should make that clear - and it's not the place to validate GetConsensus().powLimit values here.

hodlinator · 2025-12-22T21:21:46Z

+    block.nTime = params.GenesisBlock().nTime;
+    block.hashPrevBlock.SetNull();
+    block.nBits = UintToArith256(params.GetConsensus().powLimit).GetCompact(); // lowest difficulty


Maybe have genesis block as previous block to be slightly more regular, instead of being a second genesis block (due to null hashPrevBlock)?

Suggested change

block.nTime = params.GenesisBlock().nTime;

block.hashPrevBlock.SetNull();

block.nBits = UintToArith256(params.GetConsensus().powLimit).GetCompact(); // lowest difficulty

block.nTime = params.GenesisBlock().nTime + 10 * 60;

block.hashPrevBlock = params.GenesisBlock().GetHash();

block.nBits = params.GenesisBlock().nBits;

Being the genesis block also presents a mystery as non-coinbase transactions shouldn't be possible in genesis as outputs cannot be spent for 100 blocks. (I guess making us the second block doesn't help too much in that respect as the first 100 blocks should only have a coinbase transaction).

Added genesis as hashPrevBlock, thanks.

hodlinator · 2025-12-22T21:30:11Z

-    auto const& data = benchmark::data::block413567;
-    bench.batch(data.size()).unit("byte").run([&] {
+    FastRandomContext rng{/*fDeterministic=*/true};
+    auto data{rng.randbytes<uint8_t>(CPubKey::COMPRESSED_SIZE)};


nit: Why not go for something in the same order of magnitude in size as on master, and use more modern std::bytes?

Suggested change

auto data{rng.randbytes<uint8_t>(CPubKey::COMPRESSED_SIZE)};

auto data{rng.randbytes(MAX_BLOCK_WEIGHT)};

Good idea, applied those and split it into a separate PR, and added you as co-author in #34598

hodlinator · 2025-12-22T21:35:02Z

+    const auto test_block{benchmark::GetBlock(params)};
    const auto& expected_hash{test_block.GetHash()};
-    const auto& pos{blockman.WriteBlock(test_block, 413'567)};
+    const auto& pos{blockman.WriteBlock(test_block, /*nHeight=*/1)};


I think height should technically be 0 if we don't have a prev block - then again, we have a bunch of random non-coinbase transactions, which should disqualify us from being a genesis block.

1 would be correct if our prev is genesis though, see my other comment.

we have a genesis now so kept the 1 - thanks!

hodlinator · 2026-02-09T22:02:41Z

+            return GetScriptForMultisig(required, keys);
+        }},
+        {rec.null_data_prob, [&] {
+            const auto len{1 + rng.randrange<size_t>(90)}; // sometimes exceed policy rules


Might want to make OP_RETURNs very large sometimes, probably using non-linear randomness?

With a fixed tx_count, even a few rare “huge” OP_RETURNs can push the block overweight and make the generator fail deterministically for the default seed.
But I've bumped it a tiny bit and added better comment:

const auto len{1 + rng.randrange<size_t>(100)}; // can exceed pre-v30 OP_RETURN 83-byte policy limits

hodlinator · 2026-02-10T10:00:17Z

+        const bool checked{CheckBlock(block, validationState, params.GetConsensus())};
+        assert(checked);


nit: Naming:

Suggested change

const bool checked{CheckBlock(block, validationState, params.GetConsensus())};

assert(checked);

const bool valid{CheckBlock(block, validationState, params.GetConsensus())};

assert(valid);

hodlinator · 2026-02-10T10:44:46Z

+    double nonstandard_prob{0};
+
+    //! tuning
+    size_t tx_count{0};


nit: Could document field (unless the approach is switched to min_tx_occupancy, see other comment):

Suggested change

size_t tx_count{0};

size_t tx_count{0}; //!< Does not include coinbase tx. Leave at 0 to pick a random, valid number.

hodlinator · 2026-02-11T11:37:42Z

+        const auto raw{rng.randbytes<uint8_t>(1 + rng.randrange(100))};
+        return CScript(raw.begin(), raw.end());


I think it would be better to require that ScriptRecipe probabilities add up to ~1.0 and not generate random scripts as a fallback.

If we want to still generate random scripts it should be it's own ScriptRecipe::random_prob field IMO.

Diff which applies on top off the tx_occupancy_limit-suggestion from other comment

diff --git a/src/bench/block_generator.cpp b/src/bench/block_generator.cpp index 1bedfb5f9d..1891400d20 100644 --- a/src/bench/block_generator.cpp +++ b/src/bench/block_generator.cpp @@ -68,7 +68,9 @@ auto createScriptFactory(FastRandomContext& rng, const benchmark::ScriptRecipe& double sum{}; for (const auto& p : table | std::views::keys) sum += p; + // Verify that probabilities add up to ~1.0. assert(sum <= 1); + assert(sum + 0.01 > 1.0); return table; } @@ -100,8 +102,7 @@ CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec if (probability < p) return factory(); probability -= p; } - const auto raw{rng.randbytes<uint8_t>(1 + rng.randrange(100))}; - return CScript(raw.begin(), raw.end()); + return scriptFactory.back().second(); }}; // Add 2 bytes to account for compact size representation of vtx vector increasing from 1 to 3 bytes. diff --git a/src/bench/block_generator.h b/src/bench/block_generator.h index 4ce71f9c24..9ce7ea5ad8 100644 --- a/src/bench/block_generator.h +++ b/src/bench/block_generator.h @@ -35,16 +35,16 @@ struct ScriptRecipe /* purely legacy outputs, useful for pre-SegWit baselines */ inline constexpr ScriptRecipe kLegacy{ .anchor_prob = 0.00, - .multisig_prob = 0.05, - .null_data_prob = 0.005, - .pubkey_prob = 0.10, - .pubkeyhash_prob = 0.20, - .scripthash_prob = 0.10, - .witness_v1_taproot_prob = 0.00, - .witness_v0_keyhash_prob = 0.00, - .witness_v0_scripthash_prob = 0.00, - .witness_unknown_prob = 0.00, - .nonstandard_prob = 0.005, + .multisig_prob = 0.10, + .null_data_prob = 0.01, + .pubkey_prob = 0.20, + .pubkeyhash_prob = 0.43, + .scripthash_prob = 0.23, + .witness_v1_taproot_prob = 0, + .witness_v0_keyhash_prob = 0, + .witness_v0_scripthash_prob = 0, + .witness_unknown_prob = 0, + .nonstandard_prob = 0.025, .tx_occupancy_limit = 0.8, }; @@ -53,13 +53,13 @@ inline constexpr ScriptRecipe kLegacy{ inline constexpr ScriptRecipe kWitness{ .anchor_prob = 0.001, .multisig_prob = 0.005, - .null_data_prob = 0.002, + .null_data_prob = 0.01, .pubkey_prob = 0.001, .pubkeyhash_prob = 0.02, .scripthash_prob = 0.01, - .witness_v1_taproot_prob = 0.20, - .witness_v0_keyhash_prob = 0.25, - .witness_v0_scripthash_prob = 0.05, + .witness_v1_taproot_prob = 0.35, + .witness_v0_keyhash_prob = 0.40, + .witness_v0_scripthash_prob = 0.16, .witness_unknown_prob = 0.02, .nonstandard_prob = 0.02,

I think it would be better to require that ScriptRecipe probabilities add up to ~1.0 and not generate random scripts as a fallback.

Good idea, added random_script_prob to make this explicit.

hodlinator · 2026-02-11T11:49:09Z

+        for (size_t in{0}; in < in_count; ++in) {
+            auto& tx_in{tx.vin[in]};
+            tx_in.prevout = {Txid::FromUint256(rng.rand256()), uint32_t(GeomCount(rng, rec.geometric_base_prob))};
+            tx_in.scriptSig = rand_script();


How about generating semi-realistic script sigs / unlock scripts?

WIP diff

diff --git a/src/bench/block_generator.cpp b/src/bench/block_generator.cpp index 1891400d20..4ee174f8f5 100644 --- a/src/bench/block_generator.cpp +++ b/src/bench/block_generator.cpp @@ -40,39 +40,164 @@ CPubKey RandPub(FastRandomContext& rng) return CPubKey(pubkey.begin(), pubkey.end()); } -auto createScriptFactory(FastRandomContext& rng, const benchmark::ScriptRecipe& rec) +struct FactoryEntry { + const double prob; + const std::function<CScript()> lock_script; + const std::function<CScript()> unlock_script; + const std::function<CScript()> witness_script; // TODO write + use field +}; + +auto createScriptFactory(FastRandomContext& rng, const CExtKey& xprv, const benchmark::ScriptRecipe& rec) { - std::array<std::pair<double, std::function<CScript()>>, 11> table{ - std::pair{rec.anchor_prob, [&] { return GetScriptForDestination(PayToAnchor{}); }}, - std::pair{rec.multisig_prob, [&] { - const size_t keys_count{1 + rng.randrange<size_t>(15)}; - const size_t required{1 + rng.randrange<size_t>(keys_count)}; - std::vector<CPubKey> keys; - keys.reserve(keys_count); - for (size_t i{}; i < keys_count; ++i) keys.emplace_back(RandPub(rng)); - return GetScriptForMultisig(required, keys); - }}, - {rec.null_data_prob, [&] { - const auto len{1 + rng.randrange<size_t>(90)}; // sometimes exceed policy rules - return CScript() << OP_RETURN << rng.randbytes<unsigned char>(len); - }}, - std::pair{rec.pubkey_prob, [&] { return GetScriptForRawPubKey(RandPub(rng)); }}, - std::pair{rec.pubkeyhash_prob, [&] { return GetScriptForDestination(PKHash(RandPub(rng))); }}, - std::pair{rec.scripthash_prob, [&] { return GetScriptForDestination(ScriptHash(CScript() << OP_TRUE)); }}, - std::pair{rec.witness_v1_taproot_prob, [&] { return GetScriptForDestination(WitnessV1Taproot(XOnlyPubKey(RandPub(rng)))); }}, - std::pair{rec.witness_v0_keyhash_prob, [&] { return GetScriptForDestination(WitnessV0KeyHash(RandPub(rng))); }}, - std::pair{rec.witness_v0_scripthash_prob, [&] { return GetScriptForDestination(WitnessV0ScriptHash(CScript() << OP_TRUE)); }}, - std::pair{rec.witness_unknown_prob, [&] { return CScript() << OP_2 << rng.randbytes<uint8_t>(32); }}, - std::pair{rec.nonstandard_prob, [&] { return CScript() << OP_TRUE; }}, + FactoryEntry table[] = { + { + .prob = rec.anchor_prob, + .lock_script = [&] { return GetScriptForDestination(PayToAnchor{}); }, + .unlock_script = {}, // TODO: confirm correctness + .witness_script = {}, // TODO: confirm correctness + }, + { + .prob = rec.multisig_prob, + .lock_script = [&] { + const size_t keys_count{1 + rng.randrange<size_t>(MAX_PUBKEYS_PER_MULTISIG)}; + const size_t required{1 + rng.randrange<size_t>(keys_count)}; + std::vector<CPubKey> keys; + keys.reserve(keys_count); + for (size_t i{}; i < keys_count; ++i) keys.emplace_back(RandPub(rng)); + return GetScriptForMultisig(required, keys); + }, + .unlock_script = [&] { + const size_t keys_count{1 + rng.randrange<size_t>(MAX_PUBKEYS_PER_MULTISIG)}; + const size_t required{1 + rng.randrange<size_t>(keys_count)}; + std::vector<CKey> priv_keys; + priv_keys.reserve(keys_count); + std::vector<CPubKey> pub_keys; + pub_keys.reserve(keys_count); + + for (size_t i{}; i < keys_count; ++i) { + CExtKey child_xprv; + Assert(xprv.Derive(child_xprv, rng.rand<unsigned int>())); + const CKey priv{child_xprv.key}; + priv_keys.push_back(priv); + pub_keys.push_back(priv.GetPubKey()); + } + + const CScript prevout_lock_script{GetScriptForMultisig(required, pub_keys)}; + + CScript s{OP_0}; // Extra required dummy value for CHECKMULTISIG. + const uint8_t sighash_type{SIGHASH_ALL}; + CMutableTransaction tx; + tx.vin.emplace_back(COutPoint{}, CScript{}, rng.rand32()); + const uint256 sighash{SignatureHash(prevout_lock_script, tx, 0, sighash_type, CAmount{0}, SigVersion::BASE)}; + for (size_t i{}; i < required; ++i) { + std::vector<uint8_t> sig; + Assert(priv_keys[i].Sign(sighash, sig)); + sig.push_back(sighash_type); + s << sig; + } + + return s; + }, + .witness_script = {}, // TODO: confirm correctness + }, + { + .prob = rec.null_data_prob, + .lock_script = [&] { + const auto len{1 + rng.randrange<size_t>(90)}; // sometimes exceed policy rules + return CScript{} << OP_RETURN << rng.randbytes(len); + }, + .unlock_script = {}, // Unspendable + .witness_script = {}, // Unspendable + }, + { + .prob = rec.pubkey_prob, + .lock_script = [&] { return GetScriptForRawPubKey(RandPub(rng)); }, + .unlock_script = [&] { + CExtKey child_xprv; + Assert(xprv.Derive(child_xprv, rng.rand<unsigned int>())); + const CKey priv{child_xprv.key}; + const CPubKey pub{priv.GetPubKey()}; + const CScript prevout_lock_script{GetScriptForRawPubKey(pub)}; + + const uint8_t sighash_type{SIGHASH_ALL}; + CMutableTransaction tx; + tx.vin.emplace_back(COutPoint{}, CScript{}, rng.rand32()); + const uint256 sighash{SignatureHash(prevout_lock_script, tx, 0, sighash_type, CAmount{0}, SigVersion::BASE)}; + std::vector<uint8_t> sig; + Assert(priv.Sign(sighash, sig)); + sig.push_back(sighash_type); + + return CScript{} << sig; + }, + .witness_script = {}, + }, + { + .prob = rec.pubkeyhash_prob, + .lock_script = [&] { return GetScriptForDestination(PKHash(RandPub(rng))); }, + .unlock_script = [&] { + CExtKey child_xprv; + Assert(xprv.Derive(child_xprv, rng.rand<unsigned int>())); + const CKey priv{child_xprv.key}; + const CPubKey pub{priv.GetPubKey()}; + const CScript prevout_lock_script{GetScriptForRawPubKey(pub)}; + + const uint8_t sighash_type{SIGHASH_ALL}; + CMutableTransaction tx; + tx.vin.emplace_back(COutPoint{}, CScript{}, rng.rand32()); + const uint256 sighash{SignatureHash(prevout_lock_script, tx, 0, sighash_type, CAmount{0}, SigVersion::BASE)}; + std::vector<uint8_t> sig; + Assert(priv.Sign(sighash, sig)); + sig.push_back(sighash_type); + + return CScript{} << sig << pub; + }, + .witness_script = {}, + }, + { + .prob = rec.scripthash_prob, + .lock_script = [&] { return GetScriptForDestination(ScriptHash(CScript() << OP_TRUE)); }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, + { + .prob = rec.witness_v1_taproot_prob, + .lock_script = [&] { return GetScriptForDestination(WitnessV1Taproot(XOnlyPubKey(RandPub(rng)))); }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, + { + .prob = rec.witness_v0_keyhash_prob, + .lock_script = [&] { return GetScriptForDestination(WitnessV0KeyHash(RandPub(rng))); }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, + { + .prob = rec.witness_v0_scripthash_prob, + .lock_script = [&] { return GetScriptForDestination(WitnessV0ScriptHash(CScript() << OP_TRUE)); }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, + { + .prob = rec.witness_unknown_prob, + .lock_script = [&] { return CScript() << OP_2 << rng.randbytes<uint8_t>(32); }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, + { + .prob = rec.nonstandard_prob, + .lock_script = [&] { return CScript() << OP_TRUE; }, + .unlock_script = {}, // TODO + .witness_script = {}, // TODO + }, }; double sum{}; - for (const auto& p : table | std::views::keys) sum += p; + for (const auto& e : table) sum += e.prob; // Verify that probabilities add up to ~1.0. assert(sum <= 1); assert(sum + 0.01 > 1.0); - return table; + return std::to_array(table); } CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec, const uint256& seed) @@ -80,6 +205,10 @@ CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec assert(params.IsTestChain()); FastRandomContext rng{seed}; + CExtKey xprv; + constexpr auto xprv_seed{std::to_array({std::byte{'2'}, std::byte{'1'}})}; + xprv.SetSeed(xprv_seed); + assert(rec.geometric_base_prob >= 0 && rec.geometric_base_prob <= 1); const auto tx_occupancy_limit{rec.tx_occupancy_limit != 0.0 ? rec.tx_occupancy_limit : MakeUnitDouble(rng.rand64())}; @@ -95,14 +224,33 @@ CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec block.vtx.push_back(MakeTransactionRef(std::move(cb))); } - auto scriptFactory{createScriptFactory(rng, rec)}; - auto rand_script{[&] { + auto scriptFactory{createScriptFactory(rng, xprv, rec)}; + auto rand_lock_script{[&] { double probability{MakeUnitDouble(rng.rand64())}; - for (const auto& [p, factory] : scriptFactory) { - if (probability < p) return factory(); - probability -= p; + for (const auto& entry : scriptFactory) { + if (probability < entry.prob) return entry.lock_script(); + probability -= entry.prob; + } + return scriptFactory.back().lock_script(); + }}; + const double unlock_script_prob{[&] { + double sum{0.0}; + for (const auto& entry : scriptFactory) { + if (entry.unlock_script) sum += entry.prob; + } + return sum; + }()}; + auto rand_unlock_script{[&] { + const FactoryEntry* last_unlock_entry{nullptr}; + double probability{MakeUnitDouble(rng.rand64()) * unlock_script_prob}; + for (const auto& entry : scriptFactory) { + if (!entry.unlock_script) continue; + last_unlock_entry = &entry; + if (probability < entry.prob) return entry.unlock_script(); + probability -= entry.prob; } - return scriptFactory.back().second(); + assert(last_unlock_entry); + return last_unlock_entry->unlock_script(); }}; // Add 2 bytes to account for compact size representation of vtx vector increasing from 1 to 3 bytes. @@ -120,7 +268,7 @@ CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec for (size_t in{0}; in < in_count; ++in) { auto& tx_in{tx.vin[in]}; tx_in.prevout = {Txid::FromUint256(rng.rand256()), uint32_t(GeomCount(rng, rec.geometric_base_prob))}; - tx_in.scriptSig = rand_script(); + tx_in.scriptSig = rand_unlock_script(); const size_t witness_count{GeomCount(rng, rec.geometric_base_prob)}; tx_in.scriptWitness.stack.reserve(witness_count); @@ -136,7 +284,7 @@ CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec for (size_t out{0}; out < out_count; ++out) { auto& tx_out{tx.vout[out]}; tx_out.nValue = rng.randrange(GeomCount(rng, rec.geometric_base_prob) * COIN); - tx_out.scriptPubKey = rand_script(); + tx_out.scriptPubKey = rand_lock_script(); } block_size_no_witness += ::GetSerializeSize(TX_NO_WITNESS(tx)); diff --git a/src/bench/checkblock.cpp b/src/bench/checkblock.cpp index 2bc03818ac..23354ba39c 100644 --- a/src/bench/checkblock.cpp +++ b/src/bench/checkblock.cpp @@ -7,6 +7,7 @@ #include <chainparams.h> #include <common/args.h> #include <consensus/validation.h> +#include <key.h> #include <primitives/block.h> #include <primitives/transaction.h> #include <streams.h> @@ -16,7 +17,6 @@ #include <cassert> #include <cstddef> #include <memory> -#include <vector> // These are the two major time-sinks which happen after we have fully received // a block off the wire, but before we can relay the block on to peers using @@ -24,6 +24,7 @@ static void DeserializeBlockTest(benchmark::Bench& bench) { + ECC_Context ec_context; auto stream{benchmark::GetBlockData()}; std::byte a{0}; stream.write({&a, 1}); // Prevent compaction @@ -37,6 +38,7 @@ static void DeserializeBlockTest(benchmark::Bench& bench) static void DeserializeAndCheckBlockTest(benchmark::Bench& bench) { + ECC_Context ec_context; const auto& chain_params{CChainParams::RegTest(CChainParams::RegTestOptions{})}; auto stream{benchmark::GetBlockData(*chain_params)}; std::byte a{0};

I'm happy to continue working in this direction of more realistic scripts, but wanted to see what you thought before "helping" with something you might disagree with.

Good point, split rand_lock_script and rand_unlock_script with very simple differentiation.
But the above implementation scares me, not sure we need that at this stage.
For this PR I want to keep generation self-contained and cheap, but I can imagine a bench+test helper in a follow-up where we can gradually make them more and more realistic.
Thanks for spending so much time on this!

l0rinc

Thanks a lot for the excellent review (I see you started it in December already).
I have applied almost every suggestion in one shape or form - let's iterate on a few remaining ones, let's first see if there's an appetite for removing the pre-segwit block and having a configurable generator.

Split the change up to smaller commits, first migrating the existing benchmarks to a common interface, followed by introducing the new generator (and the deletion as a separate commit since that huge binary is breaking the diff tools for me)
Added you as coauthor.

l0rinc · 2026-02-16T12:00:16Z

-    auto const& data = benchmark::data::block413567;
-    bench.batch(data.size()).unit("byte").run([&] {
+    FastRandomContext rng{/*fDeterministic=*/true};
+    auto data{rng.randbytes<uint8_t>(CPubKey::COMPRESSED_SIZE)};


Good idea, applied those and split it into a separate PR, and added you as co-author in #34598

l0rinc · 2026-02-16T13:06:24Z

+DataStream GetBlockData(
+    const CChainParams& chain_params = *CChainParams::RegTest(CChainParams::RegTestOptions{}),
+    const ScriptRecipe& = kWitness,
+    const uint256& seed = {}
+);
+CBlock GetBlock(


Done, extracted this to a separate commit where we're wrapping the previous behavior first to separate the introduction of the generator from applying it.

l0rinc · 2026-02-16T13:21:26Z

 static void LoadExternalBlockFile(benchmark::Bench& bench)
 {
-    const auto testing_setup{MakeNoLogFileContext<const TestingSetup>(ChainType::MAIN)};
+    const auto testing_setup{MakeNoLogFileContext<const TestingSetup>(ChainType::REGTEST)};


Yeah, that's why I switched to REGTEST. GenerateBlock* does nonce search in setup, and we don't want to do that on MAIN difficulty - updated the code and commit message to spell that out.

I have played with your suggestions but it seemed to me the assert(chain_params.IsTestChain()) in BuildBlock should make that clear - and it's not the place to validate GetConsensus().powLimit values here.

l0rinc · 2026-02-16T13:22:32Z

+    block.nTime = params.GenesisBlock().nTime;
+    block.hashPrevBlock.SetNull();
+    block.nBits = UintToArith256(params.GetConsensus().powLimit).GetCompact(); // lowest difficulty


Added genesis as hashPrevBlock, thanks.

l0rinc · 2026-02-16T13:23:34Z

+    const auto test_block{benchmark::GetBlock(params)};
    const auto& expected_hash{test_block.GetHash()};
-    const auto& pos{blockman.WriteBlock(test_block, 413'567)};
+    const auto& pos{blockman.WriteBlock(test_block, /*nHeight=*/1)};


we have a genesis now so kept the 1 - thanks!

l0rinc · 2026-02-16T14:35:01Z

+    FastRandomContext rng{seed};
+
+    assert(rec.geometric_base_prob >= 0 && rec.geometric_base_prob <= 1);
+    const auto tx_count{rec.tx_count ? rec.tx_count : 1000 + rng.randrange(2000)};


It seems more precise to have rec.tx_occupancy_limit between 0.0-1.0

The occupancy limit would be a cool solution to make sure this always generates a valid block, but it's not a real property of the blocks. I have also simplified generation a bit more so that a single seed generates all values now.

I would expect users to create a block by setting exact values or just a seed and experiment until they get one that fits their needs - which should deterministically work after that. For example find heavier scripts to temporarily stress their optimizations or set the values manually.

When experimenting with changing the scripts I hit block size limits with just 2000 transactions due to large scripts.

I have added a block_generator_multiple_seed_sanity to help us discover these invalid setups anyway.

l0rinc · 2026-02-16T14:35:38Z

+        ++block.nNonce;
+    }
+
+    // Make sure we've generated a valid block


Reworded it, thanks

l0rinc · 2026-02-16T15:45:49Z

+            return GetScriptForMultisig(required, keys);
+        }},
+        {rec.null_data_prob, [&] {
+            const auto len{1 + rng.randrange<size_t>(90)}; // sometimes exceed policy rules


With a fixed tx_count, even a few rare “huge” OP_RETURNs can push the block overweight and make the generator fail deterministically for the default seed.
But I've bumped it a tiny bit and added better comment:

const auto len{1 + rng.randrange<size_t>(100)}; // can exceed pre-v30 OP_RETURN 83-byte policy limits

l0rinc · 2026-02-16T15:58:55Z

+        const auto raw{rng.randbytes<uint8_t>(1 + rng.randrange(100))};
+        return CScript(raw.begin(), raw.end());


I think it would be better to require that ScriptRecipe probabilities add up to ~1.0 and not generate random scripts as a fallback.

Good idea, added random_script_prob to make this explicit.

l0rinc · 2026-02-16T16:12:55Z

+        for (size_t in{0}; in < in_count; ++in) {
+            auto& tx_in{tx.vin[in]};
+            tx_in.prevout = {Txid::FromUint256(rng.rand256()), uint32_t(GeomCount(rng, rec.geometric_base_prob))};
+            tx_in.scriptSig = rand_script();


Good point, split rand_lock_script and rand_unlock_script with very simple differentiation.
But the above implementation scares me, not sure we need that at this stage.
For this PR I want to keep generation self-contained and cheap, but I can imagine a bench+test helper in a follow-up where we can gradually make them more and more realistic.
Thanks for spending so much time on this!

l0rinc · 2026-04-07T13:01:01Z

Rebased after the nanobench setup PR and applied #34208 (comment) to checkblock benchmarks to make them even simpler - ready for review again.

Bench files loaded `block413567.raw` directly, so moving off the fixture required touching each benchmark's serialization setup. Add shared helper entry points backed by the current fixture and switch benchmark callsites to those helpers. This keeps behavior stable while moving setup to `regtest`, where the follow-up generated-block path can find a valid nonce quickly. Co-authored-by: hodlinator <hodlinator@users.noreply.github.com> Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>

The helper migration still returned one fixed historical fixture block, which kept benchmark coverage narrow and blocked script-mix experiments. Replace the fixture-backed helper with runtime block generation and `ScriptRecipe` presets so callsites can tune output mixes. Add unit tests that lock in deterministic seed behavior and helper round-trip validity. Link `bench_bitcoin` with `secp256k1` so benchmark code can include `<secp256k1.h>` and use canonical pubkey tag constants. Co-authored-by: hodlinator <hodlinator@users.noreply.github.com>

After benchmarks switched to helper-generated data, the embedded `block413567.raw` fixture and its cmake raw-data wiring were no longer needed. Co-authored-by: hodlinator <hodlinator@users.noreply.github.com>

DrahtBot · 2026-06-15T22:28:55Z

🚧 At least one of the CI tasks failed.
_{Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/27576920948/job/81527532246}
_{LLM reason (✨ experimental): CI failed because the IWYU (include-what-you-use) check reported missing/wrong #includes and intentionally exited non-zero.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

c043b28 bench: use deterministic `HexStr` payload (Lőrinc) Pull request description: ### Context Split out of bitcoin/bitcoin#32554 Inspired by bitcoin/bitcoin#32457 (comment) ### Problem `HexStrBench` uses the bytes from the embedded block fixture as a random source of bytes to measure `HexStr` performance against. This coupling makes block benchmark migrations in #32554 slightly more work than necessary. ### Fix We can use deterministic pseudo-random bytes instead so this benchmark keeps stable input without fixture coupling. Use `MAX_BLOCK_WEIGHT` so the benchmark stays in the same size range and keeps measured work above harness overhead. This changes the benchmark baseline because input size moves from about 1 MB to 4 MB. ACKs for top commit: maflcko: lgtm ACK c043b28 sedited: ACK c043b28 hodlinator: ACK c043b28 Tree-SHA512: 5144b9b71761c581ff13c8f1163efed5e97f247f3c760dd7e513209c84d50f13253aa0d2ab3a431aaa51c204fea51bece41ac128b3d0e3a9ef02d1be11d6a6db

e4402ca bench: use deterministic `HexStr` payload (Lőrinc) Pull request description: ### Context Split out of bitcoin/bitcoin#32554 Inspired by bitcoin/bitcoin#32457 (comment) ### Problem `HexStrBench` uses the bytes from the embedded block fixture as a random source of bytes to measure `HexStr` performance against. This coupling makes block benchmark migrations in #32554 slightly more work than necessary. ### Fix We can use deterministic pseudo-random bytes instead so this benchmark keeps stable input without fixture coupling. Use `MAX_BLOCK_WEIGHT` so the benchmark stays in the same size range and keeps measured work above harness overhead. This changes the benchmark baseline because input size moves from about 1 MB to 4 MB. ACKs for top commit: maflcko: lgtm ACK e4402ca sedited: ACK e4402ca hodlinator: ACK e4402ca Tree-SHA512: 5144b9b71761c581ff13c8f1163efed5e97f247f3c760dd7e513209c84d50f13253aa0d2ab3a431aaa51c204fea51bece41ac128b3d0e3a9ef02d1be11d6a6db

l0rinc mentioned this pull request May 18, 2025

bench: replace benchmark block with more representative one (413567 → 784588) #32457

Closed

This was referenced May 19, 2025

[IBD] specialize CheckBlock's input & coinbase checks #31682

Open

Short-circuit GetLegacySigOpCount for known scripts #32532

Closed

l0rinc marked this pull request as ready for review May 21, 2025 13:44

l0rinc changed the title ~~RFC: bench: replace embedded raw block with configurable block generator~~ bench: replace embedded raw block with configurable block generator May 21, 2025

DrahtBot added the Tests label May 21, 2025

DrahtBot mentioned this pull request Jun 11, 2025

test, refactor: extract script template helpers and expand sigop coverage #32729

Open

l0rinc mentioned this pull request Sep 15, 2025

bench: unrealistic ConnectBlock benchmarks #33375

Open

l0rinc mentioned this pull request Nov 27, 2025

validation: fetch block inputs on parallel threads #31132

Closed

3 tasks

hodlinator mentioned this pull request Dec 18, 2025

log: avoid collecting GetSerializeSize data when compact block logging is disabled #33738

Merged

l0rinc force-pushed the l0rinc/bench-block-generator branch from a1f706b to 0fbbefd Compare December 20, 2025 23:18

DrahtBot mentioned this pull request Jan 6, 2026

bench: add fluent API for untimed setup steps in nanobench #34208

Merged

l0rinc mentioned this pull request Jan 14, 2026

Coins Cache Cleanup checklist #34280

Open

13 tasks

DrahtBot mentioned this pull request Jan 18, 2026

kernel, refactor: return error status on all fatal errors #29700

Draft

This was referenced Jan 29, 2026

ci, iwyu: Fix warnings in src/util and treat them as errors #34448

Merged

refactor: Use SpanReader over DataStream #34483

Merged

DrahtBot added the Needs rebase label Feb 7, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from 0fbbefd to b5b57cd Compare February 7, 2026 19:16

DrahtBot removed the Needs rebase label Feb 7, 2026

hodlinator reviewed Feb 11, 2026

View reviewed changes

l0rinc mentioned this pull request Feb 16, 2026

bench: use larger payload in HexStrBench #34598

Merged

l0rinc commented Feb 16, 2026

View reviewed changes

l0rinc force-pushed the l0rinc/bench-block-generator branch from b5b57cd to ea001f6 Compare February 16, 2026 17:09

DrahtBot added the CI failed label Feb 16, 2026

DrahtBot mentioned this pull request Feb 19, 2026

kernel: Replace leveldb-based BlockTreeDB with WAL and .dat file based store #32427

Draft

l0rinc force-pushed the l0rinc/bench-block-generator branch from 406f143 to 3793ae9 Compare March 25, 2026 11:35

DrahtBot added the Needs rebase label Apr 6, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from 3793ae9 to e208869 Compare April 7, 2026 12:36

DrahtBot removed the Needs rebase label Apr 7, 2026

DrahtBot mentioned this pull request Apr 8, 2026

refactor: use SpanReader in deserialization benchmarks #35025

Merged

DrahtBot added the Needs rebase label Apr 23, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from e208869 to d974667 Compare April 23, 2026 07:21

DrahtBot removed the Needs rebase label Apr 23, 2026

This was referenced May 23, 2026

blockstorage: Remove cs_LastBlockFile recursive mutex #35359

Merged

iwyu: Fix warnings in src/bench and treat them as error #35414

Merged

DrahtBot added the Needs rebase label Jun 8, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from d974667 to a187808 Compare June 9, 2026 10:32

DrahtBot removed the Needs rebase label Jun 9, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from a187808 to 31bae5e Compare June 9, 2026 11:22

DrahtBot added CI failed Needs rebase and removed CI failed labels Jun 9, 2026

l0rinc force-pushed the l0rinc/bench-block-generator branch from 31bae5e to 64f9ac6 Compare June 15, 2026 21:15

DrahtBot removed the Needs rebase label Jun 15, 2026

l0rinc and others added 3 commits June 16, 2026 00:28

bench: remove embedded block413567.raw fixture

9bd4f18

After benchmarks switched to helper-generated data, the embedded `block413567.raw` fixture and its cmake raw-data wiring were no longer needed. Co-authored-by: hodlinator <hodlinator@users.noreply.github.com>

l0rinc force-pushed the l0rinc/bench-block-generator branch from 64f9ac6 to 9bd4f18 Compare June 15, 2026 22:28

DrahtBot added the CI failed label Jun 15, 2026

DrahtBot removed the CI failed label Jun 16, 2026

DrahtBot mentioned this pull request Jul 16, 2026

RFC: Separate out runtime errors from BlockValidationState using util::Expected #35646

Draft

	CBlock BuildBlock(const CChainParams& params, const benchmark::ScriptRecipe& rec, const uint256& seed)
	{
	assert(params.IsTestChain());

	auto data{rng.randbytes<uint8_t>(CPubKey::COMPRESSED_SIZE)};
	auto data{rng.randbytes(MAX_BLOCK_WEIGHT)};

		const bool checked{CheckBlock(block, validationState, params.GetConsensus())};
		assert(checked);

	size_t tx_count{0};
	size_t tx_count{0}; //!< Does not include coinbase tx. Leave at 0 to pick a random, valid number.

		const auto raw{rng.randbytes<uint8_t>(1 + rng.randrange(100))};
		return CScript(raw.begin(), raw.end());

Uh oh!

Conversation

l0rinc commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

Uh oh!

laanwj commented May 19, 2025

Uh oh!

yuvicc commented Jun 29, 2025

Uh oh!

Raimo33 commented Oct 19, 2025

Uh oh!

l0rinc commented Dec 20, 2025

Uh oh!

hodlinator left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0rinc Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hodlinator Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hodlinator Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0rinc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0rinc Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

l0rinc commented May 18, 2025 •

edited

Loading

DrahtBot commented May 18, 2025 •

edited

Loading

l0rinc Feb 16, 2026 •

edited

Loading

hodlinator Feb 11, 2026 •

edited

Loading

hodlinator Feb 11, 2026 •

edited

Loading

l0rinc Feb 16, 2026 •

edited

Loading