Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip . lord]

On 2020-08-22 12:15, Eli Zaretskii wrote:
> > Date: Sat, 22 Aug 2020 11:53:54 +0100
> > From: phillip.lord@russet.org.uk
> > Cc: emacs-devel@gnu.org
> > I do not want to display all the output, I want to know what it
> > should be. And I don't care about the output from things that work
> > as expected only things that do not. All of this is what ert is
> > designed for. I might like to tweak the output a little, but as ert
> > doesn't have a pluggable interface, that's harder.
>
> I'm not talking about using ert or not, I'm talking about determining
> which of the features should check out and which shouldn't.  Are you
> going to edit w32-features.el by hand each time you add or remove an
> optional library from the build?  Or will the list of the features
> which should check out be constructed automagically somehow?


It would be manual. The list of features changes rarely enough that it's 
hard to automate this and know that it will work. The automation is as 
likely to break as a manual system, as a release is not routine enough 
(for instance, you slightly broke my build script by updating the 
version number on Emacs-27 before I had done the windows build -- not a 
complaint, but Emacs releases are not routine).

If I think back to the bugs with Emacs binary releases, this would have 
stopped only a few: harfbuzz and svg failure in this release. It would 
not have prevented missing dependencies (I have missed all of harfbuzz, 
lcms and libjansson till late in the day), nor the failed optimization 
or stripping. But, it would stop regressions of all of those, assuming I 
can work out how to test for them all.


> If the latter, I didn't see you describe how will that happen.  If the
> former, it means a file in the distribution will depend on the details
> of how you build the official binaries, which could be different from
> what end-users do on their systems (those who build their own Emacs),
> so the tests will fail for them, AFAIU.

That is true with all of our tests, I think. json parsing is tested by 
make check and it will not work on msys2 if the dependencies are not 
installed. In this case, I am not suggesting adding ert tests to make 
check, but a manually run test specifically for understanding the 
features of a binary build. I am guessing anyone running this would be 
understand what they are doing enough to know why. And I will document 
the entry function.

> It might also mean problems in merging from the release branch to 
> master,
> e.g. if we remove some optional library from master that is still being 
> used on the release
> branch.

That problem is there anyway in build-deps.py and it is worse there 
since it will break a release rather than fail to test it. And I think 
for configure.ac also no?


> I guess what I'm saying is that I don't yet see the overall picture of
> how this is supposed to work.  Apologies if I'm missing something.

I think it is worth the discussion -- releases are far apart and it 
takes a degree of insight to get this right.

Phil