Re: Automake testsuite misuses DejaGnu

[Daniel Herring]

Hi Jacob,

It seems fragile for DejaGnu to probe for a testsuite directory and change 
its behavior as you describe.  For example, I could have a project without 
the testsuite dir, invoke the tester, and have it find and run some 
unrelated files in the parent directory.  Unexpected behavior (chaos) may 
ensue.

Is there an explicit command-line argument that could be added to the 
Automake invocation?

-- Daniel


On Mon, 12 Jul 2021, Jacob Bachmeyer wrote:

> Karl Berry wrote:
> >     DejaGnu has always required a DejaGnu testsuite to be rooted at a 
> > "testsuite" directory
> >
> > If something was stated in the documentation, but not enforced by the
> > code, hardly surprising that "non-conformance" is widespread.
>
> It is not widespread -- all of the toolchain packages correctly place their 
> testsuites in testsuite/ directories.  As far as I know, the Automake tests 
> are the only outlier.
>
> > Anyway, it seems like an unfriendly requirement for users. And even more
> > to incompatibly enforce something now that has not been enforced for
> > previous decades. Why? (Just wondering.) -k
>
> Previous versions of DejaGnu did not properly handle non-recursive make with 
> Automake-produced makefiles.  Beginning with 1.6.3, the testsuite is allowed 
> to be in ${srcdir}/testsuite instead of ${srcdir} exactly.  Enforcing the 
> long-documented (and mostly followed) requirement that there be a directory 
> named "testsuite" containing the testsuite allows DejaGnu to resolve the 
> ambiguity and determine if it has been invoked at package top-level or in the 
> testsuite/ directory directly.
>
> Even in 1.6.3, there was intent to continue to allow the broken cases to work 
> with a warning, but I made the conditional for that case too narrow (oops!) 
> and some of the Automake test cases fail as a result.  Fixing this now is 
> appropriate because no one is going to see the future deprecation warnings 
> due to the way Automake tests are run.
>
>
> -- Jacob