freetype-devel
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: -warmup

From: Ahmet Göksu
Subject: Re: -warmup
Date: Fri, 18 Aug 2023 02:20:15 +0300
What happens if you use, say, `-c 100000', just running the
`Get_Char_Index` test? Are the percental timing differences then
still that large?
Actually Get_Char_Index, on the three pages I have sent in the prev. mail, is higher than 6% only 4 times out of 15 total. (which is seem on other tests as well).
Why do you think so? Please explain your reasoning. Just remember
that backup processes (like cleaning up the hard disk, running some
cron jobs, etc.) can pop up anytime, thus influencing the result.
Such spontaneous events have to be eliminated.
yes, right, didn't thought of the spontaneous events.

about outliers, i splitted every tests into chuncks that is sized 100. Made IQR calculations and calculated average time on valid chunks. you can find the result in the attachment also pushed to gitlab.

also, since statistics and benchmarking are a sciences their self, i am a bit struggling while approaching the problem as well as feels like out of the gsoc project scope. I would like to share this with your indulgence. yet, of course I will move in accordance with your instructions.

Best,
Goksu
goksu.in
On 18 Aug 2023 00:02 +0300, Werner LEMBERG <wl@gnu.org>, wrote:

I have added the total table that you suggested.

Thanks.

I think Get_Char_Index is not the problem, the results varies all
the time.

As far as I can see, there is a direct relationship between the total
cumulated time of a test and the timing variation: The smaller the
cumulated time, the larger the variation.

What happens if you use, say, `-c 100000', just running the
`Get_Char_Index` test? Are the percental timing differences then
still that large?

Should I proceed to detect outliers? Since we do not get the same
error rate consistently, I think we will not find the target we
expected by outliers.

Why do you think so? Please explain your reasoning. Just remember
that backup processes (like cleaning up the hard disk, running some
cron jobs, etc.) can pop up anytime, thus influencing the result.
Such spontaneous events have to be eliminated.

Have you actually tried something along the method I suggested?


Werner

Freetype Benchmark Results

Warning: Baseline and Benchmark have the same commit ID!

Info

InfoBaselineBenchmark
Parameters-c 550 -w 50-c 550 -w 50
Commit ID3553148135531481
Commit Date2023-08-18 02:04:38 +03002023-08-18 02:04:38 +0300
BranchGSoC-2023-AhmetGSoC-2023-Ahmet

* Average time for all iterations. Smaller values are better.

** N count in (x | y) format is for showing baseline and benchmark N counts seperately when they differs.

Total Results

TestNBaseline (µs) Benchmark (µs)Difference (%)
Load50000 402930 4021700.2
Load_Advances (Normal)50000 327190 357270-9.2
Load_Advances (Fast)50000 1388 1554-12.0
Load_Advances (Unscaled)50000 1534 13988.9
Render50000 199593 213029-6.7
Get_Glyph50000 85044 88477-4.0
Get_Char_Index47000 1316 1365-3.8
Iterate CMap500 1023 1039-1.6
New_Face500 25827 28613-10.8
Embolden50000 120237 125197-4.1
Stroke50000 1549165 1561868-0.8
Get_BBox50000 53948 54843-1.7
Get_CBox50000 38993 39678-1.8
New_Face & load glyph(s)50000 61266 63766-4.1
TOTAL598000 28694532940267 -2.5

Results for Roboto_subset.ttf

TestN * Baseline (µs) * Benchmark (µs) Difference (%)
Load12000 69078678471.8
Load_Advances (Normal)12000 5681764449-13.4
Load_Advances (Fast)12000 311358-15.4
Load_Advances (Unscaled)12000 290340-17.4
Render12000 4814651436-6.8
Get_Glyph12000 1976020782-5.2
Get_Char_Index9400 284306-7.9
Iterate CMap100 2402323.5
New_Face100 45146317-40.0
Embolden12000 2471328423-15.0
Stroke12000 3632683526572.9
Get_BBox12000 11871114883.2
Get_CBox12000 947688186.9
New_Face & load glyph(s)12000 11584110974.2
TOTAL283200 6203506245510.7

Results for Arial_subset.ttf

TestN * Baseline (µs) * Benchmark (µs) Difference (%)
Load9500 9213494812-2.9
Load_Advances (Normal)9500 7716390204-16.9
Load_Advances (Fast)9500 285370-29.9
Load_Advances (Unscaled)9500 33329112.8
Render9500 4099043614-6.4
Get_Glyph9500 1753117640-0.6
Get_Char_Index9400 2622494.8
Iterate CMap100 1991895.1
New_Face100 52615432-3.2
Embolden9500 22123217031.9
Stroke9500 294112295591-0.5
Get_BBox9500 1129711569-2.4
Get_CBox9500 76738519-11.0
New_Face & load glyph(s)9500 1312014388-9.7
TOTAL228200 5824856045723.8

Results for TimesNewRoman_subset.ttf

TestN * Baseline (µs) * Benchmark (µs) Difference (%)
Load9500 1120741080963.5
Load_Advances (Normal)9500 84173836900.6
Load_Advances (Fast)9500 258260-0.9
Load_Advances (Unscaled)9500 40924141.0
Render9500 3993940323-1.0
Get_Glyph9500 1535715654-1.9
Get_Char_Index9400 254259-2.1
Iterate CMap100 1831830.1
New_Face100 598254189.4
Embolden9500 2809328294-0.7
Stroke9500 3753183738810.4
Get_BBox9500 1194112127-1.6
Get_CBox9500 75057736-3.1
New_Face & load glyph(s)9500 14238139691.9
TOTAL228200 695723690131-0.8

Results for Tahoma_subset.ttf

TestN * Baseline (µs) * Benchmark (µs) Difference (%)
Load9500 6734667455-0.2
Load_Advances (Normal)9500 58555583160.4
Load_Advances (Fast)9500 255258-1.2
Load_Advances (Unscaled)9500 2442382.3
Render9500 35180344552.1
Get_Glyph9500 15612155120.6
Get_Char_Index9400 251252-0.3
Iterate CMap100 2001942.6
New_Face100 49385048-2.2
Embolden9500 22558220272.4
Stroke9500 2546962544660.1
Get_BBox9500 96219704-0.9
Get_CBox9500 722069523.7
New_Face & load glyph(s)9500 11067105614.6
TOTAL228200 487743485439-0.5

Results for Verdana_subset.ttf

TestN * Baseline (µs) * Benchmark (µs) Difference (%)
Load9500 6229963960-2.7
Load_Advances (Normal)9500 5048160612-20.1
Load_Advances (Fast)9500 279307-10.2
Load_Advances (Unscaled)9500 259288-11.2
Render9500 3533843200-22.2
Get_Glyph9500 1678418888-12.5
Get_Char_Index9400 265299-12.7
Iterate CMap100 200241-20.0
New_Face100 51336398-24.7
Embolden9500 2274924750-8.8
Stroke9500 261771285272-9.0
Get_BBox9500 92189954-8.0
Get_CBox9500 71197653-7.5
New_Face & load glyph(s)9500 1125813751-22.1
TOTAL228200 48315253557410.8

Freetype Benchmark
reply via email to
[Prev in Thread] Current Thread [Next in Thread]