Re: Testsuite fails

[Akim Demaille]

Tom Tromey <address@hidden> writes:

> >>>>> "Pavel" == Pavel Roskin <address@hidden> writes:
> 
> Pavel> AC_CHECK_TOOL([STRIP],[strip])
> 
> Yes, I wasn't thinking.  I've removed this.
> 
> Ideally we'd have some way to tell autoconf, "if this package can be
> cross-compiled, then check for a cross `strip' as well".  Maybe
> there's an ugly way to do this.

There is still plenty to do to clean up the interface Autoconf
provides to its users :(

But I think your request is precisely AC_CHECK_TOOL.  What exactly do
you mean?

 - Macro: AC_CHECK_TOOL (VARIABLE, PROG-TO-CHECK-FOR,
          [VALUE-IF-NOT-FOUND], [PATH])
     Like `AC_CHECK_PROG', but first looks for PROG-TO-CHECK-FOR with a
     prefix of the host type as determined by `AC_CANONICAL_HOST',
     followed by a dash (*note Canonicalizing::).  For example, if the
     user runs `configure --host=i386-gnu', then this call:
          AC_CHECK_TOOL(RANLIB, ranlib, :)

     sets `RANLIB' to `i386-gnu-ranlib' if that program exists in
     `PATH', or to `ranlib' if that program exists in `PATH', or to `:'
     if neither program exists.


or do you mean

if $cross_compiling; then
  AC_CHECK_TOOL([STRIP], [strip])
fi

?


> Pavel> But what bothers me is the fact that very serious patches were
> Pavel> committed without running the testsuite.
> 
> Yeah, that's my fault.  Sometimes I forget to do this.  That's bad.

Yeah, you're a bad guy :)


> gcc uses a nice regression framework that sends email to people who
> committed patches in the window where a new bug could have been
> introduced.  That seems like overkill for automake.

Well, if it's easy to install, I would do it!  Yep, this is overkill,
but some comfort through more automation is somewhat attractive.


> I imagine we'll have to radically revamp the test suite when we move
> to using autoconf's tracing feature.  The current test suite is very
> dumb and doesn't usually generate a correct configure.in.

Well, not to say `never'.

I would really enjoy seeing Automake move to Autotest.

Also, my number one grief against the test suite is that it is
classified by alphabetical order, which looks cute, but is a stupid an
order as the random order.  We need to order them from the ``innermost
tests'' to the outermost, so that it gives us some good intuition of
*where* it fails, and what failures are actually the same only one.

> Perhaps we could also rewrite the test suite to be faster somehow.
> Maybe writing most of it in Perl would help.

I considered several times having several Autotest target languages,
and Perl indeed seems a good one.  But let's not invent yet another
testing framework, let's go to Autotest!