Re: [PATCH v4 05/18] ci: explicitly skip I/O tests on alpine

[Daniel P . Berrangé]

On Wed, Nov 24, 2021 at 02:36:59PM +0100, Thomas Huth wrote:
> On 24/11/2021 14.01, Daniel P. Berrangé wrote:
> > The block I/O tests don't work on Alpine because their alternative libc
> > impl emits different strings for errnos, which breaks the expected
> > output matching. e.g.
> > 
> > === IO: pattern 102
> >   wrote 512/512 bytes at offset 512
> >   512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > -qemu-img: Error while reading offset 0 of 
> > blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
> > +qemu-img: Error while reading offset 0 of 
> > blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error
> >   4
> >   Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
> >   Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0
> > 
> > Currently the I/O tests are skipped as a side effect of the Alpine image
> > containing a minimal busybox 'sed' binary, rather than GNU Sed. This is
> > a fragile assumption that will be invalidated when the dockerfile is
> > changed to be autogenerated from a standardized package list that
> > includes GNU Sed.
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >   .gitlab-ci.d/buildtest.yml | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> > index 71d0f407ad..e1fe37e563 100644
> > --- a/.gitlab-ci.d/buildtest.yml
> > +++ b/.gitlab-ci.d/buildtest.yml
> > @@ -24,7 +24,7 @@ check-system-alpine:
> >         artifacts: true
> >     variables:
> >       IMAGE: alpine
> > -    MAKE_CHECK_ARGS: check
> > +    MAKE_CHECK_ARGS: check-unit check-qtest
> 
> Hmm, that's just a work-around ... what if some user wants to run "make
> check" on an Alpine installation that has the real GNU sed installed? ... I
> think this rather requires some fixing in the iotests instead - or maybe the
> related tests should simply not be in the "auto" group anymore?

Of course, the I/o tests should be fixed, but that's a big job that on
one has volunteered for.

Taking the tests in question out of the "auto" group would be very
detrimental for test coverage on other platforms. There are many
affected tests so that is not desirable.

As explained above, the CI job is already skipping the I/O tests today
as a side effect of only having Busybox sed. This change just makes
that explicit in the CI config so it is clear that we're missing this
coverage and won't accidentally break when GNU Sed appears in the
dockerfile.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|