Re: Help with parallel test harness

[Dan Mahoney]

> On Jan 8, 2023, at 10:55 AM, Kip Warner <kip@thevertigo.com> wrote:
> 
> On Sun, 2023-01-08 at 04:31 -0800, Dan Mahoney wrote:
>> Hello,
> 
> Hey Dan,
> 
>> I’m one of the maintainers of OpenDKIM, dealing with an interesting
>> program where make -j is breaking some of our test programs. 
>> Specifically, we have tests that *must* run first to generate keys,
>> and *must* run last to do cleanup tasks.
>> 
>> Is there an easy way with automake to force this?
>> 
>> Our Makefile.am:
>> https://github.com/trusteddomainproject/OpenDKIM/blob/develop/libopendkim/tests/Makefile.am#L20
> 
> Indeed there is. What you are looking for, if I understand correctly,
> is what's called partial ordering constraints. This is done by
> explicitly declaring make prerequisites between the log files generated
> by each unit test. What you want to do is see § 15.2.3 of Automake:
> 
>   
> https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html
> 
> Scroll down and see the paragraph "In order to guarantee an ordering
> between tests...".
> 
> Hope that is helpful.

Somewhat.  I had come across it, but it has a scaling issue.

So if test2 needed to run after test1, you'd do:

test2.log : test1.log

That makes sense Here's the fun, I can't find an easy way to say:

(literally 159 tests).log : test-setup.log

test-cleanup.log : (literally all the tests above).log

Alternatively, if the test framework had a way to *define* setup and cleanup 
tasks, that would be useful.

The .WAIT primitive doesn’t work in TESTS either.  I’m loathe to just 
.NOTPARALLEL this but it would be an easy workaround.

-Dan