Re: Regression caught by replay_kernel.py:ReplayKernelNormal.test_aarch64_virt

[Peter Maydell]

On Tue, 27 Jul 2021 at 14:24, Cleber Rosa <crosa@redhat.com> wrote:
> Yes, I've spent quite some time with some flaky behavior while running
> the replay tests as well. But in the end, the test remained unchanged
> because we found the issues in the actual code under test (one time
> the recording of the replay file would sometimes be corrupted when
> using >=1 CPUs, but 100% of the time when using a single CPU).
>
> This time, it was failing 100% of the time in my experience, and now,
> after the fix in df3a2de51a07089a4a729fe1f792f658df9dade4, it's
> passing 100% of the time.  So I guess even tests with some observed
> flakiness can have their value.

To me they have very little value, because once I notice a test
is flaky I simply start to ignore whether it is passing or failing,
and then it might as well not be there at all.
(This is happening currently with the gitlab CI tests, which have
been failing for a week.)

-- PMM