Skip to content

Don't send full transactions with a merkleblock if already known#125

Closed
dgenr8 wants to merge 1 commit into
bitcoinclassic:0.12from
dgenr8:keep_filtering_inventoryknown
Closed

Don't send full transactions with a merkleblock if already known#125
dgenr8 wants to merge 1 commit into
bitcoinclassic:0.12from
dgenr8:keep_filtering_inventoryknown

Conversation

@dgenr8

@dgenr8 dgenr8 commented Mar 2, 2016

Copy link
Copy Markdown

In merkleblock response, don't include transactions if they were already seen on the peer connection.

This long-standing optimization was removed in bitcoin#7133. The reason for the removal was that tracking of inventory already known to the peer was expanded and converted to a bloom filter with false-positive rate of 1e-6, whereas previously it was a simple set.

The use of a bloom filter in the already-seen check introduces a 1e-6 chance that a transaction will not be sent along with the merkleblock, even when it has not actually been seen before. Nevertheless, the optimization should be reinstated for the following reasons:

  • Consistency with RelayTransaction, which retains the optimization
  • It is all that's needed for Classic to fully serve XT-style thin blocks (client support is separate)
  • SPV clients should be prepared to handle a merkleblock which is not accompanied by full transactions for every txid they may need

I have corresponded with the maintainers of the most popular P2P SPV clients, here is a summary:

@gavinandresen

Copy link
Copy Markdown

Supporting thin blocks is important, but even a one-in-a-million chance of an SPV client not hearing about a transaction is annoying. How about doing this:

    if ((!pfrom->nServices & NODE_NETWORK) || 
                !pfrom->filterInventoryKnown.contains(pair.second))
       pfrom->PushMessage(NetMsgType::TX, block.vtx[pair.first]);

... so SPV clients end up getting extra data, but full nodes don't.

@dgenr8 dgenr8 force-pushed the keep_filtering_inventoryknown branch from c4b01ab to 5edcee5 Compare March 17, 2016 17:55
@dagurval

Copy link
Copy Markdown

I like the idea but the solution you propose will unfortunately not work for pruned nodes. Is there another way to signal that we want filtered merkleblocks?

@gavinandresen

Copy link
Copy Markdown

Right, you can be NODE_NETWORK and pruned and still wanna do the thin block thing...

This morning I checked, and (at least) bitcoinj doesn't do the BIP130 'sendheaders' thing, so SPV clients don't have the fPreferHeaders flag set. But checking that feels like a hack.

The right way is a new service bit for thin-block-preferring nodes-- and I think the thin blocks patch does have a new service bit, yes?

@dagurval

Copy link
Copy Markdown

Yes, there is a new service bit, but still the same problem.

Node supports bloom, but does it support filtered bloom or unfiltered bloom (or both)?
vs.
Node supports thin blocks, but is it bloom thin blocks or xthin blocks (or both)?

There should be sub-service bits :-)

@dgenr8

dgenr8 commented Mar 17, 2016

Copy link
Copy Markdown
Author

The service bit is for the "extreme" protocol changes. I think we're back to the SPV node being prepared to notice a txid being sent without the full tx. All it really needs to do is process merkleblocks from multiple peers, which it should be doing anyway to catch peers that withhold.

@dagurval

Copy link
Copy Markdown

If this is what you're thinking of @gavinandresen , then I'm happy with it. Not because it's a good solution but because it solves my problem :-)

 if ((!pfrom->nService & NODE_XTHIN) || !pfrom->filterInventoryKnown.contains(pair.second))
       pfrom->PushMessage(NetMsgType::TX, block.vtx[pair.first]);

In merkleblock response, re-enable suppression of transmission of
transactions already seen on the peer connection.
@dgenr8 dgenr8 force-pushed the keep_filtering_inventoryknown branch from 5edcee5 to ab94afb Compare March 17, 2016 19:16
@dgenr8

dgenr8 commented Mar 17, 2016

Copy link
Copy Markdown
Author

@dagurval Ok, no remote clients currently available will benefit from this, but we are working on one.

Before this PR is updated, we have another question to answer -- how to best distinguish a classic peer (which has this fix) from peer which lacks it.

@dgenr8

dgenr8 commented Jul 25, 2016

Copy link
Copy Markdown
Author

With the superior Xthin protocol supported, it will be used by the only client that supported XT-style thinblocks anyway. Closing.

@dgenr8 dgenr8 closed this Jul 25, 2016
@zander

zander commented Jul 26, 2016

Copy link
Copy Markdown

Thanks dgenr8!

jamoes pushed a commit to jamoes/bitcoinclassic that referenced this pull request Feb 3, 2017
…allowed

This feature was intended to allow high priority txns to always propagate
however there are issues with this as highlighted in issue bitcoinclassic#125

1) This feature is a magic number which is not a good idea to begin with
2) txns that have been floating around without getting mined for long periods of
   time end up rising in priority and avoiding getting rate limited which the
   consequence that our mempools are filling up much faster with un-mineable txns.
3) This value can and does override the minlimitertxfee value at times which makes
   it confusing for users who see that the minlimitertxfee is not being honoured.
@dgenr8 dgenr8 deleted the keep_filtering_inventoryknown branch September 19, 2018 14:21
@dgenr8 dgenr8 restored the keep_filtering_inventoryknown branch September 19, 2018 14:22
@dgenr8 dgenr8 deleted the keep_filtering_inventoryknown branch September 19, 2018 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants