[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Go packaging
Re: Go packaging
Wed, 4 Oct 2017 06:19:18 +0200
Thanks Leo for the explanation. Now I understand why Go programs, such
as the IPFS implementation, have so many dependencies... What I
understand now is that packages get built 'lazily' and there is really
no way to force a build - other than running the target software. I
noticed the hash values in dependencies, so these are git checkouts
(i.e., they are versioned, but not in the standard sense).
It is an interesting approach - because it does guarantee people are
using the same (deep) software stack. I noticed that too:
On Tue, Oct 03, 2017 at 11:15:04AM -0400, Leo Famulari wrote:
> Based on my work creating a go-build-system and packaging a non-trivial
> Go application , I want to start a discussion on how we can
> efficiently package Go software in Guix.
> Go software is developed rather differently from most of what we
> package, and I think our package abstraction does not fit Go libraries
> The primary differences are 0) Go libraries are typically bundled as
> source code, 1) Go software is typically unversioned, 2) static archives
> (.a) are the norm, and 3) Go libraries are expected to be built as part
> of the build process of the calling application. That is, there is no
> standard way to build an entire Go library on its own.
> 0,1) The problem with every application bundling unversioned libraries
> is that we need to package a different Git commit of the library for
> every application we package, or risk our packages not working. Adding
> package variants per-version is a bit messy currently.
> 3) As an example of how Go libaries are built piecemeal, take the core
> networking library, 'golang.org/x/net' . It includes dozens of
> submodules such as bpf, icmp, ipv4, ipv6, etc. There is no way to build
> all these submodules with a single command. Instead, each one is built
> when it is needed during the build process of the calling application.
> There are no build scripts. The library compilation process is
> standardized as, for example, `go install golang.org/x/net/ipv4`.
> This means that the entire networking library would consist of several
> dozen Guix packages, multiplied by the number of different Git commits
> required by the calling applications. It's unreasonable, in my opinion.
> My suggestion is that we have two layers of Go library packages: a) a
> template layer that includes the source URI, unpack-path, and other
> package metadata, and b) a layer to create instances of the package
> within the inputs field of the calling package.
> Perhaps the instantiation layer could look like this in practice:
> (define-public my-go-program
> ,(go-package golang-org-x-net
> (version "ffcf1bedda")
> (import-paths '("golang.org/x/net/ipv4"
When I read the package.json file it includes things like:
indeed, there are even two versions in there for 'whyrusleeping'
deep dependencies ;). (would that be a reference to Ruby's Why?).
On the surface, similar to Rubygems, I think it is no problem to
distribute source packages from Guix and have them compile on the fly.
In a way that is also an interesting model for late optimizations -
something we are lacking in our current infrastructure. What I think
we should do is import above json file and generate GNU binary
packages that are GO source bundles. Provided GO can use a live build
directory outside the store it will only compile bundles once, on
demand. Ruby 'compiles' or interprets every time, so that is one up on
Go ;). That target directory would be mutable, so that is a downside,
a potential security risk.
What you are saying is that, inside the build system, we pull in all
packages as sources and do a complete compile - which flies in the
face of how dependencies are built independently today. One question
is, can you force that full compilation?
We ought to have a look at how Nix packaged Go builds because they are
already have a solution. Be interesting to see if they found a way to
compile packages 'greedily', the way Python does it.