doc: Add Nix build notes#35375
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35375. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please copy-paste |
|
Thank you for your contribution. Some developers working on this project maintain https://github.com/bitcoin-dev-tools/bix , which defines a fairly comprehensive nix dev environment. I think the utility for documenting a slimmer setup, like done here, is rather limited, so I'm tending towards Concept NACK here. |
|
@sedited Thanks for the quick reply! I simply was not aware of that project. If that's the preferred way of building using Nix, I think it should be linked in the documentation. AFAICT, there are no references to bix as of now. When I wanted to build Core, I followed the README at the root of the project and only found info for various OSes. I've looked at their flake file. It's still somewhat "arbitrary" as it uses Clang and not GCC, for example. It's also way more complicated than the code I'm proposing here, which mainly aims to teach a person how to create their own env quickly. What do you think would be the best way to proceed?
I'm personally in favor of (3). Not because I've already written this, but because it's the most flexible:
WDYT? |
|
I think setting up a simple environment is pretty trivial. Anybody who is already motivated enough to read a nix documentation section, can probably do so with minimal effort anyway. Maybe @willcl-ark has an opinion on how and whether we should give some guidance to nix users. |
|
Sure, happy to hear @willcl-ark's opinion on this. In general, I'm OK with this not being merged if people don't want this. Not trying to be pushy in any way, just trying to clarify. I just think it is quite arbitrary not to include Nix if there are instructions on how to install via apt, etc. Yet Nix is a much more powerful tool allowing for use cases I mentioned above. So it's not simply about this basic recipe and more about having a place to collect future advanced use cases. But I'd be also happy to do so in my own repo somewhere. I also could have waited and collected those locally and then submitting, but I'm worried that it'll turn a simple doc change into a monumental task and it never gets done. In my experience, small incremental steps are easier. |
|
I am also thinking if the existing If we want to keep them, a simple table with the package manager name and package name should be sufficient (with or without nix). |
|
Something Nix related in this repo could be useful, but the current text is too verbose. If somebody wants to learn about Nix, what it is, how it works, or how to install it, they can go to https://nix.dev/ or https://nixos.org/. We don't need to repeat documentation/context from those websites here, content in this repo should stay specific to Bitcoin Core. |
|
Yes I think we should consider carfeully what we want to add here, nix-wise, as there are many possibilites.
These might cater to different crowds; it's quite nice to say to users "to build the binary run I suppose this comes down to "what problem are we trying to solve here".
In any case, I do like the idea of simpler table naming package manager and package names, and not repeating the instructions ~verbatim multiple times. |
|
There are several comments and proposals, let me try to summarize them to make them actionable:
Comments to the above:
My refined proposal:
Could you please react with 👍 or 👎 to this proposal? If 👎 , please share why. In general, I would like to make a small step to having better documentation instead of having a huge discussion about it. Let's make this first step and then we can separately discuss other things on other PRs. Again, not trying to be pushy, but it's very hard to make progress when we're discussing multiple things at the same time or have conflicting opinions. I hope that the plan presented above is a good compromise. |
|
Simplified according to the refined proposal above, confirmed that Bix works as of today. I think this is ready to be merged and looks way better than the original version. |
|
Ah, one second, let me edit the PR description as well to match the new doc structure. |
|
Updated, ready for review. |
| # Getting Nix | ||
|
|
||
| [Nix](https://nix.dev) is a declarative package manager from the | ||
| [NixOS](https://nixos.org) project. [Download Nix](https://nixos.org/download) | ||
| and follow the [manual](https://nix.dev/manual/nix/latest) to install it. | ||
|
|
||
| # Development environment | ||
|
|
||
| Some Bitcoin Core contributors maintain | ||
| [Bix](https://github.com/bitcoin-dev-tools/bix), which provides a Nix | ||
| development environment. Refer to the Bix documentation on how to use it. | ||
|
|
||
| This should allow you to build the project and run the tests, as specified in | ||
| [Developer Notes](developer-notes.md). Please follow the instructions there. |
There was a problem hiding this comment.
Not sure there is any value in this now. This mostly reads like a generic "ad" for Nix and NixOS and links to a separate repo. People running Nix(os) won't need to read an "ad" for it in the Bitcoin Core docs.
Also, I don't understand why this is not in the build-unix.md docs, like all other Linux distros.
There was a problem hiding this comment.
mostly reads like a generic "ad"
Is it the number of links? The links are there because the wiki, manual, and download instructions are all in different places.
There are notes for other package managers already, so it's not different. It's just that you don't have to link to the apt package manager website because it's already installed on popular systems. Nix won't be installed unless you're on NixOS, but it can be used on various systems.
It's also not like we're linking to something new. Nix has a community. There are NixOS meetups. The work on Nix started in 2003. The PhD thesis was defended in 2006. The NixOS paper was published in 2010. For comparison, the Bitcoin whitepaper was published in 2008.
Not sure there is any value in this now.
I was not aware of the Bix project because it's in a separate repo. So I spent some time creating my own config when I decided to contribute. It was not a lot of time, but it was significant, and definitely longer than copy-pasting. I had to add one dependency, wait for the build to fail, add another, and repeat until I could build the project.
Sometimes building with Nix can be a pain if some tricky dependency is not packaged. Having it explicitly mentioned for people who want to use it saves time, because you know from the start that things are likely to work.
I don't understand why this is not in the build-unix.md
Nix can be used on macOS, Linux distros, and Windows (via WSL). There are dedicated sections for all three. I could have linked my notes three times from each of these, but I'm not sure that's better.
There was a problem hiding this comment.
Sorry, with "ad" I mean a link to Nix. There are instructions for Ubuntu/Fedora/Alpine/etc, but none of them link to upstream. Maybe they can be skipped?
There was a problem hiding this comment.
Yep, I'll remove the links. The text provides enough context to understand which package manager this is about if you're not familiar, so the upstream docs and download page can be located with a web search. This way we're also not responsible if at some point they forget to renew a domain and it starts serving malware.
There was a problem hiding this comment.
I don't understand why this is not in the build-unix.md
Nix can be used on macOS, Linux distros, and Windows (via WSL).
I know, but they are all *nix (unix-like), so build-unix.md seems like the right place.
Also, the author of bix seems to recommend
| Nix (`nix-shell -p`) | `gcc gnumake cmake pkg-config python3 libevent boost` | `sqlite` | `capnproto` | `zeromq` | `linuxPackages.systemtap` | `qt6.qtbase qt6.qttools qt6.qtwayland qrencode` |
in #35402 (comment), which wouldn't require any external link at all.
But I am not sure what the difference is between bix and the above nix-shell -p
There was a problem hiding this comment.
@maflcko I'm okay with moving this to build-unix.md, because it's better than nothing, but I think it's bad for discoverability. I would not look there if I were on macOS, because there's a dedicated macOS file.
nix-shell -p is the old school way of creating a Nix environment with dependencies that are listed in the shell.nix file (no outside dependencies from the global environment).
Bix uses nix develop, which is a more recent take on nix-shell. It supports lock files, similar to pip and other package managers.
I'm okay with the idea of mentioning Nix in some way in that table, but I don't like the idea of just listing packages. Because it requires additional effort from the user to create a working environment. But it's better than nothing.
To match the format in that file, I would suggest we list the Nix packages as proposed above (in the table), but I think we should also link to Bix below.
There was a problem hiding this comment.
Ok, I see. So you want to add dev-specific docs here. Generally, the build-*.md docs are meant for end-users and dev stuff is listed in the dev notes (or other specific files like fuzzing.md)
Recall that Bix includes stuff like iwyu and ruff, which is not something that an end-user should have to deal with if they just want to compile Bitcoin Core.
There was a problem hiding this comment.
Understood, then let me maybe go and improve Bix so that it could be installed with minimal deps? Then we could link it here for end users and in the dev docs (for devs), with no duplication and with code reuse. Would that work?
Otherwise, we could list a shell.nix file in the docs here (or files that you would use with nix develop), but that was rejected before for being too basic and because it duplicates Bix.
My original motivation with this PR was to save people time. I would like an example that you could copy-paste, which would allow people to get started quickly. That’s why I don’t like the idea of listing dependencies in a table much because you then have to go and figure out how to use your package manager to install things, but it’s been already merged as part of the Unix notes change.
There was a problem hiding this comment.
That’s why I don’t like the idea of listing dependencies in a table much because you then have to go and figure out how to use your package manager to install things, but it’s been already merged as part of the Unix notes change.
Idk, the number of users that use a package manager (e.g. dnf) and don't know to type dnf install ... should be zero? In any case, it should be trivial to replace dnf (etc) with dnf install ... in the table.
Concept ACK on linking to bix. Fwiw, I also didn't know about it. I have been using uncomputable/bitcoin-nix-tools.
I think a minimal nix section with a link to bix would be nice to solve the problem of not knowing about it. |
Fixed in #35402 |
This PR adds notes on how to create a development environment using the Nix package manager, by recommending the user to use bitcoin-dev-tools/bix.
Rationale
Nix simplifies dependency management, which is especially relevant on systems without recent packages. It also allows you to install different compiler versions and libraries with ease, which is great for testing and development.
Why not add a package recipe at the root of the repo
Adding a configuration file for a package manager is an opinionated change. It also requires extensive testing and needs to work for a variety of users. For now, it's proposed to just link to the Bix documentation and focus on improving that project. This helps users who want to use Nix, but doesn't put burden on the main project to maintain relevant configs.