Re: ./bootstrap --gnulib-srcdir and GNULIB_REVISION

[Simon Josefsson]

Bruno Haible <bruno@clisp.org> writes:

> Hi Simon,
>
>> Bug #2: ./bootstrap writes to the path indicated by --gnulib-srcdir with
>> the 'git checkout' command, and leaves the --gnulib-srcdir path at that
>> commit after ./bootstrap is finished.  This happens to work in my
>> example since I pointed it to a writable work tree, but I think altering
>> that path is unexpected and not documented.  Imagine pointing this to a
>> system-wide gnulib .git store like --gnulib-srcdir=/usr/share/src/gnulib
>> or similar read-only place.  Or imagine multiple ./bootstrap running at
>> the same time for different projects, both pointing to the same gnulib
>> .git work tree.  I think the path indicated by --gnulib-srcdir should be
>> read-only.
>> 
>> Should the 'git checkout' code be replaced with something like
>> 
>>   git clone --reference "$GNULIB_SRCDIR" "$gnulib_path" \
>>   && git checkout -C "$gnulib_path" $GNULIB_REVISION
>>   GNULIB_SRCDIR="$gnulib_path"
>> 
>> Discussion before suggesting patches would be useful, to establish some
>> agreement on how we want this to behave.
>
> You're right, --gnulib-srcdir and the $GNULIB_SRCDIR variable denote
>   "the local directory where gnulib
>    sources reside.  Use this if you already
>    have gnulib sources on your machine, and
>    you want to use these sources."
> (I introduced the distinction between GNULIB_SRCDIR and GNULIB_REFDIR
> in commit 2122284380cc0d1b3b6f11d92c04652616da79c7.)
>
> Thus the behaviour you observed is a bug. Even worse, 'bootstrap' does
> it even when the option --no-git is given!
>
> How to reproduce:
>   $ git clone git://git.savannah.gnu.org/make.git
>   $ cd make
>   $ ./bootstrap --no-git --gnulib-srcdir=$GNULIB_SRCDIR
>
> I think the use of --gnulib-srcdir when GNULIB_REVISION is specified
> in bootstrap.conf is a classical example of conflicting requests.
> Which one should take precedence?
>   - IMO --gnulib-srcdir is documented in such a way that it takes
>     precedence.
>   - But one may also argue that it should produce an error, to make
>     the user aware of the conflict. Something like
>     "The option --gnulib-srcdir cannot be honored together because the 
> package specifies a GNULIB_REVISION."
>     The user should be able to resolve the conflict either way,
>     by choosing different command-line options.

My reaction was initially exactly the same as yours, until I found this
piece of --help documentation, which actually is the first (and
presumably highest priority) rule:

 * If the environment variable GNULIB_SRCDIR is set (either as an
   environment variable or via the --gnulib-srcdir option), then sources
   are fetched from that local directory.  If it is a git repository and
   the configuration variable GNULIB_REVISION is set in bootstrap.conf,
   then that revision is checked out.

So I think this combination is intended to be supported, it is just not
working when a .gitmodules file is present in $CWD -- something that is
not mentioned as a requirement.

I would certainly agree that trying to understand the interaction
between:

   1) --gnulib-srcdir
   2) --gnulib-refdir
   3) GNULIB_REVISION
   4) --no-git
   4) building from a tarball with .gitmodules file
   5) building from a tarball without .gitmodules file
   6) building from a git clone with a .git sub-directory
   7) building from a git clone with an indirect .git file
   8) building with GNULIB_REVISION provided as an environment variable
      outside of bootstrap.conf

and maybe other factors is really complicated, and I have had to read
both --help and source code to feel close to understanding things.
Alas, GNULIB_REVISION is not documented in doc/gnulib.texi.

My impression is that the ./bootstrap script has gained a lot of
complexity that has evolved organically.  For some projects this
complexity is unwanted -- e.g., guile-gnutls is used early in the
bootstrapping of Guix and we eventually resolved to putting all needed
gnulib files in .git and used a naive ./bootstrap script:

https://gitlab.com/gnutls/guile/-/blob/master/bootstrap?ref_type=heads

/Simon

signature.asc
Description: PGP signature