freetype-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ft-devel] GSOC -Project Availability


From: armin
Subject: Re: [ft-devel] GSOC -Project Availability
Date: Mon, 8 Apr 2019 19:07:56 +0100

> A random thought on testing FreeType: I recently read about a small testing
> library for C code, here: 
> https://www.bassi.io/articles/2019/03/14/a-little-testing/.
> Might be interesting. Another option could be using the freetype-py wrapper
> and test with that (pytest is nice). One thing that would be welcome is a
> continuous integration ready approach, meaning that every commit would trigger
> test runs under different operating system images and compilers. This could
> help with uncovering differences due to compilers.

Wait Nikolaus, not so fast :D

The project described by Sadeep was the rendering test framework.  I don't 
think it's super possible to have a (reasonable) CI-ready test suite in 3 
months (due to many, many reasons).  As far as I see it (also Kushal's 
approach), you have to go for qualitative testing not for quantitative testing 
there.  Something along the lines of outputting `the old' result (with a 
reference version of FreeType) and `the new' result (with the version under 
test) and have a way of sorting through differences efficiently for human users 
(devs) with most important ones on top.  Like some senseful ranking of 
differences by some value, detecting "simple" shifts etc + having a good 
(clean) UI to inspect the diff (highlighting of differences one way or another).
The main problem (afaik) is that tiny differences can always occur and 
sometimes they're known and benign due to some change within some module.  But 
other times you have small changes between version that actually hint at grave 
issues;  finding a reasonable ranking algorithm seems difficult enough but 
finding one that is completely automatable within CI might not be possible 
easily.

That was at least the idea behind that project last year -- not saying that's 
the only way of approaching it :)




reply via email to

[Prev in Thread] Current Thread [Next in Thread]