Re: [PATCH] Cwrapper should not eat -- arguments

[Ralf Wildenhues]

Hi Charles,

* Charles Wilson wrote on Mon, May 26, 2008 at 04:17:08PM CEST:
> Ralf Wildenhues wrote:
>
>>> Of course, if $build != $host, and $build does not have a suitable   
>>> emulation environment, everything in the whole test suite will 
>>> fail/skip  anyway, so that concern may not be an issue...
>>
>> Again, I don't understand.  
>
> 1. in a cross-compile case
> 2. where you do not have an emulation environment for $host
>
> no application that you link will be runnable.  So any test that depends  
> on running a compiled executable will fail unless it is skipped. But it  
> sure won't PASS.

Sure.  Thing is, we already have dozens of similar tests in the new
testsuite.  They mostly work as expected, skipping if a test cannot be
run at all.

>> Just because we don't use the cwrapper in
>> a test, doesn't mean that the test would now magically fail to pass "--"
>> on to the real program. 
>
> Okay, but now you have a wierd case. Suppose your test case relies on  
> *running* the target executable, not just "I successfully linked it".  
> There are several cases:
[...]
> This is a LOT of special casing.

You are thinking too complicated.  The test is to be as follows: we
build a program linked against an uninstalled library.  The program
echoes its arguments.  If we can execute the program at all, then we
require that its output contain the arguments we passed it (due to wine
debug blurb, we ignore extra output).  If the program cannot be
executed, we skip the test.  That is the only special case.

With testsuite tests, you have to think the differently: they are
supposed to exercise just the functionality that the user expects to
work.  Since we intend to deliver to the user an API that aims to be as
system-agnostic as possible (modulo mistakes done in designing the API,
and limitations in systems we do not cover up), our testsuite tests
should be possible to be written just as system-agnostic.

And if you would take a look at tests/*.at, especially helper macros
such as LT_AT_EXEC_CHECK in tests/testsuite.at, then you will see that
often this works quite well.

You cannot expect that our testsuite will uncover bugs in systems other
than the one for which the build tree is configured.  Let's not try to.

> (2) cross-to-mingw, $build has emulation environment -- this should work.
> Again, your target prog would act as above, and you could verify that '--'
> was passed on. [Aside: it could also report its env, so you could verify
> that the PATH was properly manipulated. But then the test would need to be
> different for different $host; in one case, it needs to verify
> LD_LIBRARY_PATH, on another, SHLIB_PATH, on another, PATH -- unless this
> particular testcase is going to be $host-specific]

Same idea again.  The testsuite shouldn't try to verify that FOO_PATH
was modified by the script.  Instead, it should try to execute the
uninstalled program, and thereby verify that the runtime linker can
actually *find* the uninstalled library.  Test the feature that is
supposed to work, not the way by which our actual implementation makes
it work.

Let me reformulate this more generally: whenever at all possible, do
not reimplement the libtool.m4 or ltmain logic within the testsuite.
If you absolutely have to differentiate, do not do it based on $host or
similar, but on libtool variables (those output by './libtool --config)
so that you at least reuse the libtool.m4 logic.

I am aware that we're deviating from this ideal quite a bit, but that
doesn't make it a less worthwhile goal.

Likewise, ltmain should ideally not use $host either, but I digress ...

Cheers,
Ralf