Re: Running the full MariaDB test suite

[Ludovic Courtès]

Leo Famulari <address@hidden> skribis:

> On Tue, Jul 17, 2018 at 10:04:42PM +0200, Marius Bakke wrote:
>> Hello!
>> 
>> So far our MariaDB package has only run some very lightweight tests.
>> 10.1.34 flipped a switch that runs the entire "MTR" test suite.
>> 
>> After patching /bin/ls and /bin/sh, and fixing a couple of failures that
>> showed up after about 4 hours each, the most recent attempt took eight
>> hours during which my SSD was completely thrashed.  And then it failed a
>> test case related to the "disks" plugin.
>> 
>> The switch was flipped back shortly after the release:
>> 
>> https://github.com/MariaDB/server/commit/0a9d78f51d74be7708f2efd940311bf7b33108e9
>> 
>> Since 10.1.35 won't run these tests anymore, we could skip them for this
>> release and don't care about it.
>> 
>> Alternatively, we can replace the 'check' phase with something that
>> invokes "mtr" with sensible arguments, and passing our own list of tests
>> to skip.  That is what Debian does:
>> 
>> https://salsa.debian.org/mariadb-team/mariadb-10.1/blob/stretch/debian/rules#L96
>> 
>> Thoughts?
>> 
>> I'm leaning towards the latter approach, since I've already spent
>> considerable time tracking down related failures, and it already
>> identified a potential problem in the "disks" plugin.  But it means
>> building MariaDB will take many hours even on powerful machines.
>
> No strong opinion but the latter approach sounds good to me.

Same here.

Great that you spent energy tracking these issues!

Ludo’.