Where is the testing? (was: Re: [PATCH] fs/xfs: Avoid unreadble filesystem if V4 superblock)

[Glenn Washburn]

On Thu, 2 Sep 2021 10:56:49 +0200
Carlos Maiolino <cmaiolino@redhat.com> wrote:

> On Wed, Sep 01, 2021 at 02:40:57PM +0200, Daniel Kiper wrote:
> > CC-ing Javier...
> > 
> > On Mon, Aug 30, 2021 at 01:48:50PM +0200, Carlos Maiolino wrote:
> > > Hi.
> > > On Mon, Aug 30, 2021 at 11:18:31AM +0200, Erwan Velu wrote:
> > > >    Good day list,
> > > >    Le jeu. 26 août 2021 à 15:26, Carlos Maiolino
> > > > <[1]cmaiolino@redhat.com> a écrit :
> > > >
> > > >      [..]
> > > >      Thanks for spotting this!
> > > >
> > > >    I'm adding the maintainers in CC. Carlos who commit the
> > > > patch I'm fixing, agreed on the content.
> > >
> > > I didn't test the patch itself yet, but I've reproduced the
> > > issue. I was quite sure I had tested this patch on a V4 fs, but
> > > looks like I miscalculated the sizing. Thanks again. I'll try to
> > > test the patch here asap.
> > 
> > Did you test this patch? If yes may I add your Tested-by to it?
> 
> Yup, patch works fine, just finished testing it, I was just trying to
> understand where/why I miscalculated the inode size on V4
> filesystems, and the reason was the same why Erwan split the
> last/first members of inode v2/v3 in two different unused structs.
> 
> Feel free to add to the patch:
> 
> Tested-by: Carlos Maiolino <cmaiolino@redhat.com>

It looks like the xfs_test test succeeds with tag grub-2.06-rc1a, fails
with tag grub-2.06, and succeeds with current master. Yes, as expected.
However, what this tells me is that no "make check" tests were done
before finalizing the 2.06 release. I was under the impression that that
was part of the release procedure. If its not, it would seem that we're
not using the tests at a time when they would have the most impact.

It is my understanding that we have travis-ci tests that get run (at
some point?), however they are only build tests and so would not have
caught this. It was precisely this scenario that I hoped to avoid by
doing more thorough continuous integration, which runs the extensive
array of "make check" tests, when I submitted the Gitlab-CI patch
series (which would've caught this automatically if it had been merged).
To me this scenario is the poster child for taking action on this
front. Can we make this a priority?

Glenn