[draft] Refactor some validation.cpp methods to return BlockValidationState#7
Closed
optout21 wants to merge 178 commits into
Closed
[draft] Refactor some validation.cpp methods to return BlockValidationState#7optout21 wants to merge 178 commits into
optout21 wants to merge 178 commits into
Conversation
add comment explaining coinstatsindex cache exclusion update cache allocations to 10%/5%/5%
When setting the fd limit to unlimited, the node fails to start: ulimit -n unlimited build/bin/bitcoind Error: Not enough file descriptors available. -1 available, 160 required. This was caused by RaiseFileDescriptorLimit() casting limitFD.rlim_cur to int, which for RLIM_INFINITY overflows to -1. Fix it by returning std::numeric_limits<int>::max() instead. This commit also adds a functional test, which is skipped on environments with a hard limit below infinity. Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org> Co-authored-by: winterrdog
When setting the fd limit to 1 >> 31, the node fails to start: ulimit -n 214748364 build/bin/bitcoind Error: Not enough file descriptors available. -2147483648 available, 160 required. Similar to the previous commit, this is fixed by capping the limit to std::numeric_limits<int>::max(). Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
On 32-bit systems we build with _FILE_OFFSET_BITS=64 (see CMakeLists.txt), which makes rlim_t 64-bit when building against glibc (see bits/resource.h). Since size_t could be 32-bit, clamp RLIMIT_MEMLOCK to std::numeric_limits<size_t>::max() in PosixLockedPageAllocator::GetLimit(). Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
Cherry-pick include-what-you-use commit 52f85e1f4d99 onto the clang_22 branch tracked by ci/test/00_setup_env_native_iwyu.sh, fixing the false positive where std::hash was mapped to <string_view>/<variant> instead of its real provider (<functional>, <memory>, etc).
The default std::hash for shared_ptr compares by pointer. CTransactionRefHash or a custom hasher should be used instead. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
LoadBlockIndex() adds to m_blocks_unlinked based only on nTx > 0, without checking BLOCK_HAVE_DATA. Pruning preserves nTx but clears BLOCK_HAVE_DATA, so a pruned block whose parent was header-only gets re-added on every restart, causing the CheckBlockIndex() assertion that entries must have data on disk to fail. Check that BLOCK_HAVE_DATA is set before inserting into m_blocks_unlinked. Fixes bitcoin#35050.
d78b817 to
44d3197
Compare
CHMAC_SHA256 and CHMAC_SHA512 leave two stack buffers populated on return: rkey[] holds K' XOR ipad after the constructor, and temp[] holds the inner-hash output after Finalize(). When the HMAC is keyed with sensitive material (chain code in BIP32Hash() in hash.cpp for BIP32 child key derivation; PRK in HKDF-Expand in hkdf_sha256_32.cpp, used for BIP324 transport keying), rkey is one constant XOR from that key, and temp is a one-way digest covering it. Cleanse both buffers with memory_cleanse(), matching the convention in chacha20.cpp and chacha20poly1305.cpp. No observable change for callers.
A pruned stale-fork block whose parent doesn't have any transactions shouldn't be added to m_blocks_unlinked when starting up a node.
HMAC primitives cleanse their internal stack buffers, but a caller's ChainCode remains populated in memory after use. Promote ChainCode from `typedef uint256` to a `base_blob<256>` subclass with a memory_cleanse() destructor, so chain codes in CExtKey, CExtPubKey, and local variables are cleansed on scope exit. Retype MUSIG_CHAINCODE from `constexpr uint256` to `const ChainCode` to match its BIP328 semantic role. Dropping `constexpr` (ChainCode is no longer a literal type) also removes the GCC-14 consteval lambda workaround. Remove the duplicate typedef in pubkey.h (which includes hash.h transitively). Two fuzz-test call sites in test/fuzz/key.cpp now construct the chain-code argument explicitly rather than relying on the typedef.
a52e467 to
fd9b639
Compare
Owner
Author
|
Note: bitcoin#33856 has been merged |
aa67944 to
771a265
Compare
The variable is never set and will always be unbound.
The only way to set it correctly is via this hack:
.github/workflows/ci.yml- # In the image build step, no external environment variables are available,
.github/workflows/ci.yml- # so any settings will need to be written to the settings env file:
.github/workflows/ci.yml: run: sed -i "s|\${INSTALL_BCC_TRACING_TOOLS}|true|g" ./ci/test/00_setup_env_native_asan.sh
So just silence the warning, which happens when running the task
locally:
```
./ci/test/00_setup_env_native_asan.sh: line 12: INSTALL_BCC_TRACING_TOOLS: unbound variable
```
Shellcheck marked the script as violating SC2044. Instead of re-writing the Bash from scratch, just use Python.
build: exclude mptest target from compile commands
We are currently sometimes backlogged on waiting for runner selection. Selecting Warp or GitHub-hosted runners directly from the repository context avoids serializing all CI jobs behind a metadata job. This keeps forks on public runners while allowing upstream jobs to schedule immediately on the intended runner labels.
The seed ID is calculated from a pubkey produced by treating the seed as a private key. This calculation includes a pubkey compression parameter, even thought that compression is completely irrelevant for the usage of the seed as a BIP 32 seed. Thus migration should detect if a seed has been used multiple times by checking if the computed master key was already processed. The spkm_migration fuzzer needs to have it's added descriptors accounting to be updated for this fix.
Signed overflow on nScore updates is undefined behavior. Use SaturatingAdd in AddLocal() and SeenLocal() so increments saturate at INT_MAX instead of overflowing. Add unit test coverage for saturation in both code paths.
Instead we fully reset CoinsResult-instances without forgetting any fields. Fixes: bitcoin#35449 Variant resetting the other fields in Clear(): ₿ hyperfine --warmup 2 "./build/bin/test_bitcoin -t coinselector_tests/knapsack_solver_test" Benchmark 1: ./build/bin/test_bitcoin -t coinselector_tests/knapsack_solver_test Time (mean ± σ): 4.289 s ± 0.008 s [User: 4.252 s, System: 0.027 s] Range (min … max): 4.279 s … 4.301 s 10 runs This variant: ₿ hyperfine --warmup 2 "./build/bin/test_bitcoin -t coinselector_tests/knapsack_solver_test" Benchmark 1: ./build/bin/test_bitcoin -t coinselector_tests/knapsack_solver_test Time (mean ± σ): 4.279 s ± 0.005 s [User: 4.244 s, System: 0.026 s] Range (min … max): 4.271 s … 4.287 s 10 runs
…ngs.json 0654511 util: Check write failures before renaming settings.json (Shrey) Pull request description: This PR Fixes bitcoin#35373. The error message was updated from: "This is probably caused by disk corruption or a crash" to: "This is probably caused by a full disk, disk corruption or a crash" The following files were updated: 1. src/common/settings.cpp -- Updated the main string formatting for the parse failure. 2. src/test/settings_tests.cpp -- Updated the exact string expectation in the C++ unit test. 3. test/functional/feature_settings.py -- Updated the expected error message in the Python test suite. Configured the Bitcoin Core project natively on Windows using Visual Studio (cmake --preset vs2026-static -DBUILD_GUI=OFF) and it compiled with 100% of the C++ tests passed successfully (347 out of 347). ACKs for top commit: maflcko: review ACK 0654511 💧 winterrdog: ACK 0654511 sedited: ACK 0654511 ryanofsky: Code review ACK 0654511 with commit message and error message improved since last review Tree-SHA512: d7b49860f2081a5a9d1d44917b0cf372a77f10cf21773c9b3291871c788a122ec5dde063fdf9e1052cf45b8d2667e15585b952e9f037e84f98fb41546bdd1fa9
ff0af9e to
811adb0
Compare
811adb0 to
6232800
Compare
…nReserver reserve function 2818a17 test: add abortscan unit test (Pol Espinasa) bc30e95 wallet: move fAbortRescan reset into WalletRescanReserver reserve() (Pol Espinasa) Pull request description: Follow-up of bitcoin#35179 For extra context refer to the conversations bitcoin#35179 (comment) and comments bellow it. Long story short: currently `ScanForWalletTransactions()` resets the value of `fAbortRescan` before starting the rescan loop. This can cause a race condition where some function (e.g. `importdescriptors`) starts a rescan and at the same time the user aborts it manually. Could happen that the `abortrescan` call returns True (success) but the rescan continues running as the value is overwritten. This PR fixes this by resetting the value of `fAbortRescan` at the very beginning, when the wallet rescan is reserved, removing the race condition. Also adds a test for it. ACKs for top commit: achow101: ACK 2818a17 w0xlt: ACK 2818a17 nebula-21: ACK 2818a17 pablomartin4btc: ACK 2818a17 pinheadmz: ACK 2818a17 Tree-SHA512: 5b64b9a16a209dd145ccf7b2d7c3a7205038b3b214b02c3d9f1cbda75a6a1c9a41ad5c42cbeda43af6141793965613c47e44300d758149be9bfdf458e85a5520
1f3f0a4 Squashed 'src/secp256k1/' changes from 7262adb4b4..bd0287d650 (fanquake) Pull request description: Includes: * bitcoin-core/secp256k1#1846 * bitcoin-core/secp256k1#1848 * bitcoin-core/secp256k1#1849 * bitcoin-core/secp256k1#1851 * bitcoin-core/secp256k1#1852 * bitcoin-core/secp256k1#1854 * bitcoin-core/secp256k1#1855 * bitcoin-core/secp256k1#1859 * bitcoin-core/secp256k1#1860 * bitcoin-core/secp256k1#1861 * bitcoin-core/secp256k1#1864 * bitcoin-core/secp256k1#1865 * bitcoin-core/secp256k1#1867 The most interesting change to call out / that reviewers might want to test probably being bitcoin-core/secp256k1#1859. ACKs for top commit: theStack: ACK 9caae50 hebasto: ACK 9caae50. sedited: ACK 9caae50 andrewtoth: ACK 9caae50 Tree-SHA512: 187e76d7f7ed603d81b7c355abfb184f37658fdb5109a2c78f72c615cdf862dfc9c3e2ff4556eb047d59348ba1a7030ca38342682c56dbfdb51610d2cc1de405
…sult fields 2447385 rpc: remove unused RPCResult::Type::ELISION (satsfy (Renato Britto)) 7a85118 rpc: expand decodepsbt output script with explicit fields (satsfy (Renato Britto)) 88e2a6a rpc: expand getaddressinfo embedded with explicit fields (Renato Britto) a9f9e7d rpc: extract fee estimate result helpers (Renato Britto) 8a615a8 rpc: extract ListSinceBlockTxFields() helper (Renato Britto) 372ac28 rpc: extend TxDoc() for getblock verbosity 2/3 (Renato Britto) 0380a1c rpc: extend TxDoc() for getrawtransaction verbosity 2 (Renato Britto) 44fc3a2 rpc: introduce HelpElision variant and ElideGroup helper (Renato Britto) Pull request description: Partially addresses bitcoin#29912. Motivated by bitcoin#34683, which exports OpenRPC from existing `RPCHelpMan` metadata. [Sample OpenRPC](https://gist.github.com/natobritto/8c4a1da04968d2325082ac4bca7d2408). Some RPC help definitions rely on `RPCResult::Type::ELISION` entries whose structure is only described in prose. This keeps human-readable help concise, but leaves parts of the result layout implicit and prevents tools from deriving complete machine-readable schemas from `RPCHelpMan` metadata. This PR replaces ELISION-based reuse with shared structured definitions, so result layouts are represented directly in metadata rather than only in text. At the same time, human-readable help remains compact via explicit help-rendering elision using `HelpElision`, so previously elided sections stay abbreviated without losing schema completeness. Affected RPCs: `getrawtransaction`, `getblock`, `listsinceblock`, `estimaterawfee`, `getaddressinfo`. RPC return values are unchanged. Human-readable help remains compact, while structured result metadata becomes explicit enough to derive complete machine-readable schemas. A related `RPCResult::Type::ELISION` use in `importdescriptors` was split out into the follow-up PR bitcoin#34867 because it changes the generated help output, per [this review comment](bitcoin#34764 (comment)). Changes: - Introduce `HelpElision` (`NONE`, `START`, `SKIP`) and `ElideGroup()`, replacing the tri-state `print_elision` - Add an `RPCResult` copy-with-replacement-options constructor to support applying elision while keeping `m_opts` const - Extend `TxDoc()` / `TxDocOptions` to support reusable transaction layouts with optional `prevout`, `fee`, `hex`, and elision behavior - Replace ELISION-based reuse in `getrawtransaction` and `getblock` with explicit structured definitions - Factor shared result layouts into `GetBlockFields()`, `ListSinceBlockTxFields()`, `FeeRateBucketDoc()`, `GetAddressInfoEmbeddedFields()` and `FeeEstimateHorizonDoc()` - Expand `listsinceblock.removed`, `estimaterawfee` horizons/buckets and `getaddressinfo.embedded` into explicit metadata while preserving concise help output ACKs for top commit: achow101: ACK 2447385 w0xlt: reACK 2447385 janb84: re ACK 2447385 willcl-ark: ACK 2447385 Tree-SHA512: 8dc03c45c388ebdb4f8f1613af2576fc127a8d5425efe518cd0d0ed2439a38b2ed1236413471672c85f52ee22c4a17677c18fd4689bc6570496fc3af4cd4112f
735b255 support: clamp RLIMIT_MEMLOCK to size_t (Sjors Provoost) 8ab4b9f init: clamp fd limits to int (Sjors Provoost) 4afbabd Fix startup failure with RLIM_INFINITY fd limits (Sjors Provoost) Pull request description: When setting the fd limit to unlimited, the node fails to start: ```sh ulimit -n unlimited build/bin/bitcoind Error: Not enough file descriptors available. -1 available, 160 required. ``` This was caused by `RaiseFileDescriptorLimit()` (introduced in bitcoin#2568) casting `limitFD.rlim_cur` to `int`, which for `RLIM_INFINITY` overflows to `-1`. Fix it by returning `std::numeric_limits<int>::max()` instead. Some platforms implement `RLIM_INFINITY` as the maximum uint64, others as int64 (-1). So simply changing the return type to `uint64_t` wouldn't work. Similarly, though unlikely to actually happen: ```sh ulimit -n 214748364 build/bin/bitcoind Error: Not enough file descriptors available. -2147483648 available, 160 required. ``` The second commit expands the fix by clamping all values above `std::numeric_limits<int>::max()` instead of letting them overflow. This PR also expands `test/functional/feature_init.py` to cover these, using `resource.setrlimit`. The check is skipped on environments with a hard limit below infinity (or that don't have the Python [Resource module](https://docs.python.org/3/library/resource.html)). macOS by default has a hard limit of infinity, but on e.g. Ubuntu the default hard limit is 524288. The third commit applies a similar fix to `PosixLockedPageAllocator::GetLimit()` for 32-bit systems, but without a test. ACKs for top commit: winterrdog: Re-ACK 735b255 achow101: ACK 735b255 sedited: Re-ACK 735b255 pinheadmz: ACK 735b255 Tree-SHA512: 0ce0292ecd61456bdec6943b06cbb9ecfc5180ee6dce850f8496ef54af22c1fae6ea473085202f5ba6f72e4dc51a29247620c9a0eae31e96658adc77b293129f
7ad0822 to
12a93b8
Compare
Return BlockValidationState by value instead of using an out-parameter, similar to the TestBlockValidity refactoring in 74690f4. Remove redundant int return from btck_chainstate_manager_process_block_header. Previously returned both an int result and an output validation state parameter, creating ambiguity where non-zero could mean either invalid header or processing failure. Since ProcessNewBlockHeaders already provides complete validation info, the int return was redundant. Co-authored-by: stringintech <stringintech@gmail.com> Co-authored-by: stickies-v <stickies-v@protonmail.com>
In `ChainstateManager::AcceptBlock()`, don't ignore the return value from `FlushStateToDisk`.
12a93b8 to
cc4bac4
Compare
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary. Refactor validation result to be a return value instead of an output parameter in several
validation.cppmethods.Motivation. The benefits of the change are:
stateare inconsistentThis has grown out from bitcoin#33856, mentioned in comment here and here.
Details. Many methods follow the scheme where the validation state is returned in an output parameter (
BlockValidationState& state), and and additionalboolreturn value indicating success. In success case the convention is thatstate.IsValid()and the return value are both true. After the change there is only aBlockValidationStatereturn value, which is either success (state.IsValid() == true), or an invalid/error case.This change is a highly localized refactor, but touching a sensitive file.
Relevant methods called by
ProcessNewBlockHeadersandAcceptBlock(directly and indirectly) are touched.Changes are separated into commits by touched methods, ordered by bottom-to-top in the call hierarchy.