Skip to content

Make all networking code mockable#21878

Closed
vasild wants to merge 9 commits into
bitcoin:masterfrom
vasild:Sock_all_over_the_place
Closed

Make all networking code mockable#21878
vasild wants to merge 9 commits into
bitcoin:masterfrom
vasild:Sock_all_over_the_place

Conversation

@vasild

@vasild vasild commented May 7, 2021

Copy link
Copy Markdown
Contributor

@DrahtBot

DrahtBot commented May 7, 2021

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK practicalswift, jonatack, promag, jnewbery, jamesob, josibake

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #27375 (net: support unix domain sockets for -proxy and -onion by pinheadmz)
  • #26812 (test: add end-to-end tests for CConnman and PeerManager by vasild)
  • #26312 (Remove Sock::Get() and Sock::Sock() by vasild)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@practicalswift

Copy link
Copy Markdown
Contributor

Strong Concept ACK: mockable means fuzzable.

@jonatack

Copy link
Copy Markdown
Member

Concept ACK

@promag promag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

Comment thread src/net.h Outdated
Comment thread src/net.h Outdated
@promag

promag commented May 12, 2021

Copy link
Copy Markdown
Contributor

@vasild I think 96bcc72 and refactors around it could easily be merged on its own.

@vasild vasild force-pushed the Sock_all_over_the_place branch from 0256cfd to e68bc63 Compare May 13, 2021 11:54
@vasild

vasild commented May 13, 2021

Copy link
Copy Markdown
Contributor Author

0256cfd34...e68bc633f: address review suggestions

@vasild

vasild commented May 13, 2021

Copy link
Copy Markdown
Contributor Author

@vasild I think 96bcc72 and refactors around it could easily be merged on its own.

Right! Submitted in #21943.

@jnewbery

Copy link
Copy Markdown
Contributor

Very nice. Strong concept ACK.

@vasild vasild force-pushed the Sock_all_over_the_place branch from e68bc63 to 5772edb Compare May 14, 2021 10:38
@vasild

vasild commented May 14, 2021

Copy link
Copy Markdown
Contributor Author

e68bc633f...5772edb36: add an explicit commit that shows we intentionally don't process newly accepted nodes, as discussed in #21943 (comment)

@vasild vasild force-pushed the Sock_all_over_the_place branch from 5772edb to ec5f887 Compare May 17, 2021 14:47
@vasild

vasild commented May 17, 2021

Copy link
Copy Markdown
Contributor Author

5772edb...ec5f887: replicate changes to #21943

@vasild vasild force-pushed the Sock_all_over_the_place branch from ec5f887 to a0b0030 Compare May 19, 2021 11:49
@vasild

vasild commented May 19, 2021

Copy link
Copy Markdown
Contributor Author

ec5f887543...a0b00304a8: rebase due to conflicts

@vasild vasild force-pushed the Sock_all_over_the_place branch from a0b0030 to 6bed261 Compare May 20, 2021 13:30
@vasild

vasild commented Jun 28, 2022

Copy link
Copy Markdown
Contributor Author

a6424d5023...12f848b5cd: rebase due to conflicts

@vasild

vasild commented Jul 20, 2022

Copy link
Copy Markdown
Contributor Author

12f848b5cd...f07b89e917: rebase due to conflicts

@achow101

Copy link
Copy Markdown
Member

Should this be marked as draft since it is a parent PR?

@jonatack

Copy link
Copy Markdown
Member

@vasild this may be able to be closed now that all of the child PRs have been merged.

@fanquake

Copy link
Copy Markdown
Member

vasild this may be able to be closed now that all of the child PRs have been merged.

Given that the majority of commits in this pull haven't been merged, I don't think so.

@vasild

vasild commented Oct 14, 2022

Copy link
Copy Markdown
Contributor Author

f07b89e917...9a27807453: rebase due to conflicts

Should this be marked as draft since it is a parent PR?

Could be, I am not sure. The reason I did not make it a draft is because it can be merged as it is - there is no danger that something bad will happen if it gets merged (assuming there are reviewers who ACK it). The reason I split it into smaller PRs is solely to ease review. Should I make it draft?

@vasild this may be able to be closed now that all of the child PRs have been merged.

Given that the majority of commits in this pull haven't been merged, I don't think so.

Right, there are two pieces left - #26312 which I just opened and the tests themselves (which depend on that PR, thus no separate PR for them yet).

@vasild

vasild commented Dec 12, 2022

Copy link
Copy Markdown
Contributor Author

9a27807453...77ab5e8e98: rebase due to conflicts

@vasild

vasild commented Feb 9, 2023

Copy link
Copy Markdown
Contributor Author

77ab5e8e98...b497200c7a: rebase due to conflicts

vasild added 9 commits August 16, 2023 14:31
Peeking at the underlying socket file descriptor of `Sock` and checkig
if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of
testing/mocking/fuzzing.

Instead use an empty unique_ptr to denote that there is no valid socket.
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `GetBindAddress()` is called.
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `ConnectSocketDirectly()` is called.
`Sock::Get()` was used only in `sock.{cpp,h}`. Remove it and access
`Sock::m_socket` directly.

Unit tests that used `Get()` to test for equality still verify that the
behavior is correct, indirectly, by testing whether the socket is closed
or not.
Now that all network calls done by `CConnman::OpenNetworkConnection()`
are done via `Sock` they can be redirected (mocked) to `FuzzedSocket`
for testing.
@vasild

vasild commented Aug 16, 2023

Copy link
Copy Markdown
Contributor Author

b497200c7a...80d9dfd0f0: rebase due to conflicts

@DrahtBot

DrahtBot commented Oct 3, 2023

Copy link
Copy Markdown
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

@vasild

vasild commented Oct 4, 2023

Copy link
Copy Markdown
Contributor Author

All the functional bits of this are now merged. Thank you all!

This enables extending the test framework to be able to call any method, without worries that it may end up doing OS syscalls (try to create sockets, open connections, etc). A PR that does some of that is in #28584 (= last 4 commits from this PR).

Closing as complete.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.