Skip to content

backport: assumeutxo M3 — crash-safe background completion (bitcoin#25740, #27862, #28050, #27746)#52

Open
PastaPastaPasta wants to merge 9 commits into
assumeutxo/m2-evodb-rolesfrom
assumeutxo/m3-background-completion
Open

backport: assumeutxo M3 — crash-safe background completion (bitcoin#25740, #27862, #28050, #27746)#52
PastaPastaPasta wants to merge 9 commits into
assumeutxo/m2-evodb-rolesfrom
assumeutxo/m3-background-completion

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Issue being fixed or feature implemented

M2 (#51) wired up dual-chainstate EvoDB roles but left two integration
items open: promoting snapshot markers to the legacy key at completion, and
an independent (non-shared-cache) comparison of background vs. snapshot
state before a background chainstate is allowed to complete and replace the
snapshot. Without crash-safe completion, a power loss mid-promotion can
destroy a valid, already-validated snapshot. This is milestone 3/7. The
execution plan and per-unit ledger (M3 section) are tracked outside the repo
in a private gist, not checked in: https://gist.github.com/PastaPastaPasta/aa52b1f89fb74a0566ba3b5e15afab6a

PR 3/7 in the stacked series — base is M2 (assumeutxo/m2-evodb-roles).

What was done?

  • bitcoin#27862 (moved here from M1 since it depends on M3's
    SnapshotCompletionResult) and bitcoin#28050FAILURE_FATAL bypasses
    the reindex prompt in Dash's init else-if structure;
    FAILURE_INSUFFICIENT_DBCACHE intentionally absent until bitcoin#27596
    lands in M5.
  • bitcoin#25740 — crash-safe background-validation completion. Round 1
    adversarial review found four issues: crash-unsafe promote-before-swap
    ordering (including missing fsync gaps), MN-list comparison that wasn't
    actually independent of the shared cache, a nullopt base-height skip that
    could mask on-disk corruption, and unsynced activation metadata — all
    fixed. Dash adaptation: crash-safe completion lifecycle (renames +
    DirectoryCommit before a synced, idempotent PromoteSnapshotMarkers;
    RecoverSnapshotCleanup state machine run at load); independent MN-list
    verification via a per-connect NORMAL-identity captured hash slot compared
    against the activation-recorded snapshot hash at completion (not read from
    shared cache). Both M2 integration TODOs (marker promotion, holistic
    base-state comparison) are discharged here.
  • bitcoin#27746 (13 upstream commits) — per-chainstate candidate admission.
    Round 1 review found ResetBlockFailureFlags bypassed per-chainstate
    admission (background could admit non-base-path forks and starve the
    active set) and that InvalidateBlock's cache could reinsert
    ChainLock-conflicting candidates — both fixed, including a
    BLOCK_CONFLICT_CHAINLOCK exclusion in TryAddBlockIndexCandidate and a
    dual-chainstate regression test.

How Has This Been Tested?

Breaking Changes

None. Background validation completion is now fully wired (UTXO-hash +
MN-list-hash + EvoDB-marker checks, crash-safe promotion/cleanup with
load-time recovery) but is not yet reachable — loadtxoutset itself is
still unexposed until M5.

Checklist:

  • I have performed a self-review of my own code
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (docs rewrite lands in M7/B8)
  • I have assigned this pull request to a milestone

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 640284bf-9281-4f3e-bcb9-66be9ee94853

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch assumeutxo/m3-background-completion

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 552e404 to 2a37be9 Compare July 12, 2026 14:17
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m3-background-completion branch from 32d5966 to 4f9d716 Compare July 12, 2026 14:21
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 2a37be9 to 77a5910 Compare July 12, 2026 15:59
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m3-background-completion branch from 4f9d716 to f08bce5 Compare July 12, 2026 16:00
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch 4 times, most recently from 6a803ac to 014afb4 Compare July 13, 2026 22:45
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m3-background-completion branch 2 times, most recently from a6e49d6 to eb757a0 Compare July 14, 2026 05:10
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 014afb4 to 06b5c6f Compare July 14, 2026 05:10
PastaPastaPasta and others added 9 commits July 14, 2026 11:31
Discard Dash snapshot lifecycle markers only after the invalid snapshot directory rename succeeds. If the rename fails, preserving the markers keeps the existing restart recovery state recognizable while the upstream rename error is propagated to the fatal shutdown message.
Keep Dash’s mock shutdown callback while asserting the expected fatal diagnostic. The default callback reaches StartShutdown(), whose unit-test guard aborts the process before this Dash test can complete.
a733dd7 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11ab Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b85 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b Documentation improvements for assumeutxo (Ryan Ofsky)
768690b Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1 Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc753 test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc3 Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c0571 Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7c Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd7
  jamesob:
    reACK a733dd7 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd7.
  ryanofsky:
    Code review ACK a733dd7. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
Preserve ChainLock candidate exclusions in the new admission wrapper and keep Dash background-notification and EvoDB fixtures consistent with the tightened multi-chainstate candidate invariants.
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 06b5c6f to 1a95697 Compare July 14, 2026 17:01
@PastaPastaPasta PastaPastaPasta force-pushed the assumeutxo/m3-background-completion branch from eb757a0 to 2bc543c Compare July 14, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants