[Discussion] Subpixel rendering issues with compressed video output formats

[VÍTOR RAMOS]

All,

I have been studying the subpixel rendering code and I could spot a
few improvement areas and I would like to hear your opinion about it.
In this message I discuss one of them.

There are a few issues people face with subpixel antialiasing that
yield artifacts on their setup. The most common culprit is gamma
mismatches, but quite a few other less obvious factors are involved,
such as geometry which is partially addressed.

One factor that is not addressed (or even mentioned in discussions at
all) is lossy color format and space transformations in display output
pipelines.

It has become increasingly common that display adapters may
autoconfigure the output to use compressed video streams under a lossy
(chroma-subsampled) YUV output color format. This is usually done in
the context of a trade-off to achieve higher resolutions, refresh
rates or bit depths. As an example, many high-DPI panels only offer
UHD at 60 Hz under a chroma-subsampled YUV 4:2:0 stream. As another
example, many high-refresh rate panels only offer their maximum rates
under a likewise compressed format. As yet another example, higher bit
depth may also only be available under a chroma-subsampled format. In
general, video throughput may be limited by panels, adapters and even
physical channels, and the display output may be automatically
configured to use a compressed format.

As is, the project implicitly assumes an RGB 4:4:4 uncompressed output
video format and performs excellently, but if the output has chroma
subsampling then subpixel antialiased rendering can exhibit color
fringe artifacts. Indeed, several complaints about fringing may very
well be caused by chroma subsampling rather than gamma issues.

Exposing a subpixel antialiasing filter that addresses cases where the
output format has subsampled chroma is quite a large undertaking but
is feasible. Though the simpler solution in this case is to switch to
grayscale antialiasing.

To my knowledge, no libraries commercial or otherwise currently
address this issue.

Vítor