[bug#57050] [PATCH v4 08/14] gnu: racket: Update to 8.6.

[Philip McGrath]

On Sat, Aug 27, 2022, at 3:21 PM, Liliana Marie Prikler wrote:
> Am Samstag, dem 27.08.2022 um 14:55 -0400 schrieb Philip McGrath:
>> +       ((#:tests? _ #t)
>> +        ;; FIXME: There have been some flaky test failures. Some
>> have been
>> +        ;; fixed upstream post-release but have proven non-trivial
>> to
>> +        ;; backport; at least one issue remains. Re-enable tests
>> once
>> +        ;; https://github.com/racket/racket/issues/4359 is fixed.
>> +        #f)
> Rather than discarding all tests, just discard the flaky ones.

I've tried to explain this before, but, as far as I've been able to determine, 
the problem doesn't seem to be that a particular test or group of tests is 
flaky: the mysterious problems all seem to be related to setting up the test 
environment. There have been problems, for example, with files not being found, 
or the complex pile of patches used to construct variants on the expected error 
output not working properly. At least the unix, windows, ieee, date, oop, 
primvars, and misc test groups have been affected. Since it's not been easy to 
reproduce the problem (I've thought it was fixed several times), I have no 
reason to believe it wouldn't affect other test groups if we skipped those. An 
unfortunate consequence of the way the tests are organized, skipping particular 
test groups means that expected error output is never checked and requires 
manually choosing the configurations (combinations of compiler/interpreter 
settings) you want to test.

I do not see a viable path to fixing the tests for this release, and I don't 
know a way of running a subset of tests that would be both reliable and 
meaningful.

Other distributions are also skipping these tests.

In fact, Guix is currently skipping an unknown (probably large) portion of 
these tests because of the issue with "/bin/sh", which has lead to things 
silently not running.

I hope we will get this fixed upstream for the 8.7 release. In fact, I've not 
yet encountered the problem while building from Git HEAD, but it's not clear 
what change might be relevant. (Indeed, the problem might still be there, and I 
might just not have hit it yet.)

-Philip