Re: Failing to use gnulib bootstrap in libtool

[Gary V. Vaughan]

Hi Eric,

Thanks for your prompt response :)  The fog is clearing a little already!

On 3 Sep 2010, at 01:01, Eric Blake wrote:

> On 09/02/2010 10:36 AM, Gary V. Vaughan wrote:
>> Is gnulib bootstrap designed for reuse in other projects?
> 
> Yes; I know that it is already shared among coreutils, grep, libvirt, and 
> several other projects.  However, improvements are certainly welcome.
> 
> I'll tackle the questions I know an answer to, but since I don't (yet) use 
> bootstrap on m4, I'm not the best person to answer.
> 
>> 2. source_base, m4_base etc
>> ===========================
>> 
>> Why hardcode another copy of these locations into bootstrap, where they can 
>> fall out of sync.  Far better to either extract them from or inject them 
>> into `gnulib-cache.m4'.  Extracting them with `autom4te --language=Autoconf 
>> --trace=gl_SOURCE_BASE --trace=gl_M4_BASE ... configure.ac' seems like the 
>> most robust way of tackling this problem to me.
> 
> Seems reasonable to me, as well.  Do you have patches in mind?

Sure. I have some functions that parse the output of autom4te --trace that seem 
to be working better than an evaled sed script.

>> 4. bootstrap.conf
>> =================
>> 
>> Why is bootstrap.conf called below the definition of find_tool(), but the 
>> other functions
>> are redefined much later on.  It would be far more flexible to encapsulate 
>> most of the
>> functionality of `bootstrap' into shell functions (preferably well commented 
>> so that they
>> can be easily understood!) before sourcing `bootstrap.conf' so that those 
>> functions could
>> be overridden safely before they are called by the remaining code.
> 
> Indeed, that does sound like a more flexible approach for allowing 
> package-specific overrides.
> 
>> 
>> I don't like the way gnulib `bootstrap' forces a `gnulib-tool --import' 
>> invocation on every
>> run, when a `gnulib-tool --update' is quite sufficient most of the time.  If 
>> the calls
>> to gnulib-tool were in a function set before sourcing bootstrap.conf, I 
>> could redefine the
>> function for Libtool without upsetting other users of the script by fighting 
>> for a change
>> of semantics.
> 
> This is also my biggest pet peeve with bootstrap; and why I don't yet use it 
> for m4; I find the gnulib-tool --update to be much faster (especially on 
> Cygwin).

Then, I'd be very happy if you have time to test my patches.

> 
>> 5. option parsing
>> =================
>> 
>> The option parse loop is not as robust as it could be: For example, it 
>> doesn't support
>> `./bootstrap -fc --gnulib-srcdir ../gnulib' for a couple of reasons.
>> 
>> I wonder whether you would accept a generated script to check in to the 
>> gnulib repo,
>> along with a few lines of shell and an extra m4sh file to regenerate that 
>> file whenever
>> the source is edited?
> 
> Having bootstrap be generated seems reasonable (we maintain a couple other 
> generated files in gnulib, such as INSTALL being generated from install.texi 
> as copied from autoconf) - as long as the Makefile rules are in place, then 
> it seems okay to do commits that touch a master file and update bootstrap to 
> match; all other projects will just be copying bootstrap (and not any 
> upstream file).

Now that I look at it, a straight M4_GETOPTS expansion is not viable since it 
doesn't (yet) support split parsing.  However, hand coding a parse loop in the 
same style leaves more room for optimisation.


>> 7. buildreqs
>> ============
>> 
>> Why do we have to keep 2 copies of this information in sync?  `bootstrap' can
>> easily extract the prequisite versions of Autoconf, Automake, Libtool and
>> Gettext from configure.ac for us.  I didn't have this in my bootstrap script
>> so I don't have code to paste in, but it seems like it would just be a few 
>> more
>> lines in `extract_package_values'.
> 
> For the autotools, yes, bootstrap can extract prereq versions.  But for other 
> tools (think bison, help2man, etc.), that's something that bootstrap can't 
> extract.  So, as long as buildreqs have to support minimum versions (or - if 
> it is expected that any version will do), no one has bothered to automate 
> minimum version detection for the subset of tools where configure.ac already 
> contains that information.

I'll start with autoconf, automake, libtool and gettext then.

>> 8. gnulib submodule
>> ===================
>> 
>>> git_modules_config () {
>>>   test -f .gitmodules&&  git config --file .gitmodules "$@"
>>> }
>>> 
>>> gnulib_path=`git_modules_config submodule.gnulib.path`
>>> : ${gnulib_path=gnulib}
>> 
>> So when I don't have .gitmodules, gnulib_path is set to the empty string, 
>> which
>> prevents the follow line from falling back to a default value.
> 
> There's already outstanding issues that gnulib submodule handling needs to be 
> made smarter for people who do NOT have git installed, but DO have a snapshot 
> of the correct gnulib version accessible.  This whole area needs some loving 
> care.

Do people really want to rebootstrap a git checkout of, say libtool, without 
git?  I haven't seen any bug reports on bug-gnulib either.  I'm inclined not to 
worry too much about not being able to rebootstrap without the usual developer 
tools unless people actually trip over the problem.

>> Actually, I can't follow the logic of the case statement that follows, 
>> although
>> I think it is not supposed to do nothing at all when I have no `.gitmodules' 
>> file,
>> and pass `--gnulib-srcdir=../gnulib' to try and get a reference submodule 
>> installed.
>> 
>> The gnulib submodule code in GNU M4 is, by comparison, easy to follow... and 
>> it
>> works.  Aside from adding enough comments to make the logic understandable, 
>> and
>> fixing it to work in the common case of `--gnulib-srcdir=../gnulib' with no 
>> `.gitmodules'
>> file, I think the whole thing should be put inside a function before 
>> `bootstrap.conf',
>> so that I can override it with my preferred semantics.
> 
> To some degree, recent gnulib-srcdir changes in bootstrap are due to my 
> desire to copy m4's bootstrap logic.

I found the patches, and I think I understand what is required.  Although I'm 
not sure the existing code in this area can be made to work without being 
mostly rewritten.

>> 11. gnulib_tool_options
>> =======================
>> 
>> This is broken quite badly:
>> 
>>   `--import': Most of the time I'd rather use `--update', but there's no way
>>      to override it without using a gnulib-local patch (and I couldn't even
>>      get that to work for `bootstrap' since it's not even in a module).
>>   `--lib $gnulib_name' (and others): where `gnulib_name=lib$package' by 
>> default.
>>      Surely most people want to call it `libgnu' (not `liblibtool'!)?  And in
>>      any case, these are already stored in gnulib-cache.m4, so we shouldn't
>>      have to keep two references in sync.
> 
> That's one of the problems I see with bootstrap - currently, none of the 
> projects using gnulib's bootstrap are storing gnulib-cache.m4 in VCS. 
> gnulib-tool --update only works if gnulib-cache.m4 is stored in VCS (as it is 
> with M4).  If others don't mind the paradigm shift towards storing _both_ 
> bootstrap and gnulib-cache.m4 in their project, then we could make bootstrap 
> use the --update option.

How odd!  Why would you want to have to redo the entire `gnulib --import' from 
scratch every time something changes?  Still, it's easy enough to keep --import 
as the default option, as long as it is encapsulated in a function that can be 
overridden in `bootstrap.conf'.

> 
>> 12. AUTOHEADER and lack of AUTORECONF
>> =====================================
>> 
>> Autoreconf does a better job here too.  I suggest replacing this entire block
>> with:
>> 
>>     ${AUTORECONF-autoreconf} $autoreconf_flags
> 
> The biggest issue here is that gettextize and gnulib-tool do not (yet) play 
> well together; but there are also some libtool interplay issues. M4's 
> bootstrap script gets around this by running gettextize first, then 
> gnulib-tool, and finally AUTOPOINT=true autoreconf.  gnulib bootstrap gets 
> around this by running the tools separately into two different directories 
> (tmp1 for gnulib-tool, tmp2 for gettext), then merging the results while 
> favoring gnulib-tool.  I don't know if one approach is inherently better than 
> the other, but do know that M4's bootstrap is faster.

Ah yes, I remember this now.

I have to say that I find the whole tmp trees and file content mangling on copy 
quite scary.  But again, I'm happy to leave that as the default as long as I'm 
not stuck with only that option in my non-gettextize non-libtoolize projects!

Cheers,
-- 
Gary V. Vaughan (address@hidden)

PGP.sig
Description: This is a digitally signed message part