[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
New image comparison method
From: |
Luca Fascione |
Subject: |
New image comparison method |
Date: |
Wed, 2 Oct 2024 12:49:57 +0200 |
Hi,
following up on Werner's query a while back where he was dissatisfied with
how the "image diff" metric we use was catching or not problems in test
artifacts, I put together a simple python script to address the same
problem.
Now that I have a script that does something that is of some use (see
below) I could use the help of someone that is familiar with the results of
the test suite to bounce with them a few iterations of this until we arrive
at a product that makes sense to show to everyone.
In my mind I could simply pass them the script when we have something to
look at and we could discuss next steps and iterate.
My problem is that the EPS filenames for the tests don't really tell me
much, so I am having some trouble assessing how well what I have is doing.
If you're curious about the details, and for the record:
The script compares two directories containing EPS artifacts,
meant to be the baseline directory and the test/check directory:
$build/input/regression/out-test and
$build/input/regression/out-test-baseline
It first weeds out all pairs of files with the same name and EPS contents
Then it converts the rest into PDF "catalogs" using ghostscript,
renders them with pymupdf (I'm particular about this happens exactly),
and then compares the pixels, weeding out a few more tests.
On what remains it uses a feature detector (at the moment ORB, SIFT and a
Harris corner detector are implemented) creating 2D pointclouds of features
which are then compared
for similarity, and it reports the average distance of the points that
don't have an exact match.
At the moment it doesn't have a scoring system for points only present in
one of the two images but these are already set aside for some upcoming use.
At the moment it produces this output whe comparing this
23b3a475aaa964ca9be3cf5af8bdc2390dce12e7 (baseline)
to this
23a3778f2ed6a12b2e41026d3e6646f7177f0a29 (test)
-------
Loaded 5446 EPS artifacts
2 artifacts are only in baseline
5433 artifacts have identical EPS sources
4 artifacts render to identical pixels
9 artifacts render to different pixels
Test 'beam-feather-1.eps' diff measure 2.32
Test 'beam-feather.eps' diff measure 2.32
Test 'beam-french-1.eps' diff measure 2.13
Test 'beam-french.eps' diff measure 2.13
Test 'tablature-tie-spanner-1.eps' diff measure 1.45
Test 'tablature-tie-spanner-2.eps' diff measure 1.17
Test 'tablature-tie-spanner.eps' diff measure 1.41
-------
Cheers,
L
--
Luca Fascione
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- New image comparison method,
Luca Fascione <=