Re: packaging go-ethereum, and ultimately bee (of ethswarm.org)

[Attila Lendvai]

[with some reorderings]

> I hope my reply has been valueable. It seems like this discussion
> is going to end with ‘agreeing to disagree’ and I think I said all
> I've got to say about the philosophical part of the matter, so I
> don't think I'll sending further replies on the philosophical parts.
> But if you have any technical questions, feel free to ask me.


ok, noted. your feedback is much appreciated, and i don't mean to
argue against the guix philosopy. i'm converted after all! :)

so, i'll try to focus on only pointing out perspectives that you may
not be aware of, and proposing solutions.


> > this mechanism is part of how go builds are reproducible.
> > building go-ethereum locally should yield the same binary that they
> > have released.
>
> FWIW, ‘reproducibility’ in guix means that, if the build is repeated on 
> another
> machine, with the same architecture and guix version, the binaries will be
> exactly the same. Bit-for-bit reproducibility isn't guaranteed when the builds
> are performed on different distro's.


thank you for clarifying this! i'll use clearer language when talking
about reproducibility.


> the loader to be at /lib/lib(64)/ld-linux.[...].so.[...], but that doesn't 
> even
> exist on Guix, so binaries created on another distro cannot be run on guix,


FTR, the official binary seems to start up fine on my guix, and begins
syncing with some peers:

$ ~/.guix-profile/lib/ld-linux-x86-64.so.2 
./geth-linux-amd64-1.10.8-26675454/geth --syncmode light

it's not statically linked:

$ ldd ./geth-linux-amd64-1.10.8-26675454/geth
        linux-vdso.so.1 (0x00007fff26da6000)
        libpthread.so.0 => 
/gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libpthread.so.0 
(0x00007f86794af000)
        libm.so.6 => 
/gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libm.so.6 
(0x00007f867936e000)
        librt.so.1 => 
/gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/librt.so.1 
(0x00007f8679364000)
        libc.so.6 => 
/gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libc.so.6 
(0x00007f86791a7000)
        /lib64/ld-linux-x86-64.so.2 => 
/gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/ld-linux-x86-64.so.2 
(0x00007f86794d2000)

will this have any pitfalls? is there anything i'm not aware of?

if not, then i propose adding a package that downloads the official
release, authenticates it by checking its signature, and sets up a
wrapper script for it to be able to start it by its proper name.

so, maybe we should add two packages? one that fetches the official
release binary, and one that builds it from source (and doesn't
strictly follow the versions in the project's go.mod file)?

i, as a user, would be more relaxed running the authenticated official
binary release. maybe this is misguided, and in that case i'm happy to
be corrected.


> > building go-ethereum locally should yield the same binary that they
> > have released.
>
> ... why should it?


so that, ideally, we are able to use the trusted guix intrastructure
to rebuild the same binary locally that the project has
released... and with that double-check also the release process of the
upstream.


> > > Another problem is: if a go package has many (transitive) dependencies, 
> > > how do
> > > we check that it doesn't contain any malware or non-free components? That 
> > > needs
> >
> > go-ethereum is a software that handles ethereum wallets with several
> > zeroes. they are probably equally worried about this if not more.
>
> I'm sure the go-ethereum peope wouldn't mind if we double-check that no 
> malware
> slipt through their review process.

[...]

> > for projects like go-ethereum, it's not an option for the packager to
> > make decisions about the version of any of its dependencies.
>
> Why isn't this an option? Choosing different versions from upstream is already
> done in guix, see e.g. https://issues.guix.gnu.org/50217 (ok that's not merged
> yet, not the best example --- note, editing the version requirement was
> on my advice, see https://logs.guix.gnu.org/guix/2021-08-26.log).


extra auditing on top of the official release is certainly welcome by
each player. but, realisticly, i doubt that the hundreds of guix go
packages will get comparable amount of security-auditing-attention any
time soon. let alone -- and this is of key importance here -- auditing
any possible cross-interactions of altered versions of the
dependencies! who else is qualified to do that better than the
developers themselves?

for a program like the ethereum client, any such semantic surprises
may be disasterous. (losing money in various ways. e.g. their client
forking off of the consensus of the ethereum network, and if e.g. they
are paricipating in staking, then they will suffer penalties for not
adhering to the rules of the network, etc.)

this is why i would prefer to have a solution that somehow reproduces,
compares, and runs the same binary as the officially released one.

in the follwig days i'll play with reproducing the geth release binary
on my guix, and also planning to mock up a binary downloader package,
and see how it goes. i'll report back with anything worth mentioning.


> I see a third viable option (3): treat the "go.sum" as a mere
> ‘friendly suggestion’, and just use the latest version when feasible.
> Again, I don't see much difference with, say, haskell, python, ruby,
> guile, java ... packages.


my point here is not that all go projects should be packaged like this
(i.e. with exacly pinned dependencies), but that applications like
geth should be, regardless of the language they are written in.

in case of a random image viewer written in go, i'm all in for a
relaxed handling of dependencies.


> Have there been any problems in practice with just using the latest
> version (updating the version currently in guix where applicable)?


not that i'm aware of, but the first such identified issue could turn
out to be very expensive.

- attila