[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Display test suite
Re: Display test suite
Thu, 7 Jul 2016 15:40:42 -0400
Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0
On 2016-07-07 12:29, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <address@hidden> Date: Thu, 7 Jul
>> 2016 11:47:54 -0400
>> Richard recently suggested giving Emacs the ability to take a
>> screenshot of itself. Could we imagine using this to run regression
>> tests of the display engine?
> Maybe I'm missing something, but I don't see how. A test suite
> needs to include the expected results for each test. Even if you
> include an image file for each test that shows what should be
> displayed, AFAIU the image will be specific to the display
> resolution, number of colors, dimensions, etc., so inappropriate for
> automatic comparison with the screenshot on the target machine. Am I
You're right; that's why I can only run the company-coq tests on my machine.
Still, one might imagine setting up Emacs in a particular way to make them
reproducible. Along these lines, I run the PNG through an optimizer and strip
the metadata to make sure that I get the same output every time.
> Also, comparing images that could have a small number of pixels
> different in some corner is error-prone.
There are very nice tools to do image diffs; they will highlight the offending
pixels in a different color.
> IMO, a test suite for the display engine needs a way to describe
> precisely what's on the screen, in some language, and then compare
> those descriptions.
Yes, that would be nice (we have something like that for font-lock, right?)
>> Of course, it could be hard to get screeenshots to looks exactly
>> the same for everyone. Instead, we could have a makefile target
>> that builds a large collection of these screenshots, and another
>> one that rebuilds it and compares against a previously generated
> But then each time I start using a new machine, my reference point
> is reset to a newer date, and I lose the ability for comparing
> against older versions.
Absolutely. I think such tests would serve as a nice way to check your own
commits (for before/after tests).
Also, maybe the variability is a good point: each people regularly run these
tests, then changes that affect only one configuration would be noticed. If we
have tests that produce the same output for every one, changes that affect only
one particular set of configurations would not be noticed.
In any case, I've found the graphical tests that I do for company-coq very
useful; whether they could generalize to all of Emacs is a non-trivial question.
Description: OpenPGP digital signature
Re: Dead or unused face handling code, Dmitry Antipov, 2016/07/07