Skip to content

POC: IPC tracing interface#32898

Closed
ryanofsky wants to merge 4 commits into
bitcoin:masterfrom
ryanofsky:pr/trace
Closed

POC: IPC tracing interface#32898
ryanofsky wants to merge 4 commits into
bitcoin:masterfrom
ryanofsky:pr/trace

Conversation

@ryanofsky

@ryanofsky ryanofsky commented Jul 7, 2025

Copy link
Copy Markdown
Contributor

This a is proof-of-concept adding an IPC interface for tracing as discussed in bitcoin-core/libmultiprocess#185. As mentioned there, this is a naive implementation that would need to be optimized to have good performance, but it does show an end-to-end demo of how tracing could work using IPC.

The can be tested by building with -DENABLE_IPC=ON and running:

build/bin/bitcoin-node -regtest -ipcbind=unix  # Start node listening on IPC socket
build/bin/bitcoin-trace -regtest -debug        # Run bitcoin-trace and connect to socket
build/bin/bitcoin-cli -regtest -generate       # Create a block to generate sample traces

ryanofsky and others added 4 commits July 3, 2025 19:45
@ryanofsky
ipc: Add MakeBasicInit function
09733c2 
Add a MakeBasicInit() function so simpler standalone IPC clients like
bitcoin-mine in bitcoin#30437 and bitcoin-cli in bitcoin#32297 that only initiate IPC
connections without exposing any IPC interfaces themselves can to avoid needing
to implement their own specialized interfaces::Init subclasses.
@ryanofsky
interface_ui: move from node to common library
c348507 
Allows InitError and InitWarning to be used by other executables beside
bitcoind and bitcoin-node.
@ryanofsky
util: Add kernel::Traces struct
f880817 
Add kernel::Traces struct and pass it places where it will be used next commit
to support emitting IPC traces.

Similar structs like wallet::Traces could be added to enable traces from other
parts of the code.
@ryanofsky
POC: IPC tracing interface
8e09fbe 
This a is proof-of-concept adding an IPC interface for tracing as discussed in
bitcoin-core/libmultiprocess#185. As mentioned there,
this is a naive implementation that would need to be optimized to have good
performance, but it does show an end-to-end demo of how tracing could work
using IPC.

The can be tested by building with -DENABLE_IPC=ON and running:

build/bin/bitcoin-node -regtest -ipcbind=unix  # Start node listening on IPC socket
build/bin/bitcoin-trace -regtest -debug        # Run bitcoin-trace and connect to socket
build/bin/bitcoin-cli -regtest -generate       # Create a block to generate sample traces
@DrahtBot

DrahtBot commented Jul 7, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32898.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #32575 (refactor: Split multithreaded case out of CheckInputScripts by fjahr)
  • #32317 (kernel: Separate UTXO set access from validation functions by TheCharlatan)
  • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)
  • #31679 (cmake: Move internal binaries from bin/ to libexec/ by ryanofsky)
  • #31132 (validation: fetch block inputs on parallel threads 10% faster IBD by andrewtoth)
  • #30437 (ipc: add bitcoin-mine test program by ryanofsky)
  • #29641 (scripted-diff: Use LogInfo over LogPrintf [WIP, NOMERGE, DRAFT] by maflcko)
  • #29409 (multiprocess: Add capnp wrapper for Chain interface by ryanofsky)
  • #29060 (Policy: Report reason inputs are non standard by ismaelsadeeq)
  • #26966 (index: initial sync speedup, parallelize process by furszy)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ryanofsky ryanofsky marked this pull request as draft July 7, 2025 20:50
@ryanofsky ryanofsky mentioned this pull request Jul 7, 2025
Closed
@ryanofsky

ryanofsky commented Jul 8, 2025

Copy link
Copy Markdown
Contributor Author

Notes on current implementation of this:

  • The third commit f880817 would be a lot simpler if it added a Traces* g_traces or similar global variable, which would be a reasonable thing to do. Initially, I wanted to avoid globals because they are hard to get rid of, and avoiding them could have benefits like being able to attach different tracers to different CCoinsViewCache instances, but this may not be worth extra complexity.
  • In the main commit 8e09fbe, the duplication between src/interfaces/tracing.h and src/ipc/capnp/tracing.capnp isn't very nice. Maybe some code generation could be done to generate one file from the other. This would be difficult to do for general IPC calls but might be easier for simple tracing definitions.
  • In the main commit 8e09fbe, I made the coins.cpp traces verbose and explicit for clarity, but a real implementation should replace all of that tracing code with single Trace() calls like Trace<&UtxoCache::add>(UtxoInfo{outpoint.hash, ...}) which would reduce verbosity, and could be optimized to avoid locking and blocking for efficiency.

This was referenced Jul 8, 2025
Open
Open
Merged
Merged
Closed
Open
Merged
Open
Merged
Draft
@0xB10C

0xB10C commented Jul 8, 2025

Copy link
Copy Markdown
Contributor

Awesome, thanks! I'll play around with this over the next few weeks.

@0xB10C 0xB10C mentioned this pull request Jul 8, 2025
Open
@ryanofsky

ryanofsky commented Jul 8, 2025

Copy link
Copy Markdown
Contributor Author

Awesome, thanks! I'll play around with this over the next few weeks.

Great! Will close this PR since I'm not planning to do more work on it, and we can continue to discuss in bitcoin-core/libmultiprocess#185

@ryanofsky ryanofsky closed this Jul 8, 2025
@0xB10C 0xB10C mentioned this pull request Jul 10, 2025
Open
7 tasks
This was referenced Aug 19, 2025
Merged
Closed
@ryanofsky ryanofsky mentioned this pull request Sep 8, 2025
Open
@0xB10C 0xB10C mentioned this pull request Sep 15, 2025
Open
@ryanofsky

ryanofsky commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Note: @willcl-ark did some followup based on this PR:

https://github.com/bitcoin-dev-tools/ipc-exporter-rust/blob/master/report.md
https://github.com/bitcoin-dev-tools/ipc-exporter-rust
https://github.com/bitcoin-dev-tools/ipc-dash

And these seems to be more discussion about something similar tracing or monitoring here:

peer-observer/peer-observer#141
peer-observer/peer-observer#370

I haven't been following these efforts very much but they look promising and wanted to add the links.

@xyzconstant xyzconstant mentioned this pull request Mar 23, 2026
Merged
@0xB10C 0xB10C mentioned this pull request Apr 23, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ryanofsky @DrahtBot @0xB10C