Re: [PATCH v2] Arithmetic error in EDID generation fixed

[Stefan Weil]

Am 26.02.20 um 13:20 schrieb Anton V. Boyarshinov:
> To calculate screen size in centimeters we should calculate:
> pixels/dpi*2.54
> but not
> pixels*dpi/2540
>
> Using wrong formula we actually get 65 DPI and very small fonts.
>
> Signed-off-by: Anton V. Boyarshinov <address@hidden>
> ---
> Changes from v1: get rid of casts
>
>  hw/display/edid-generate.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/hw/display/edid-generate.c b/hw/display/edid-generate.c
> index 75c945a948..e58472fde5 100644
> --- a/hw/display/edid-generate.c
> +++ b/hw/display/edid-generate.c
> @@ -360,8 +360,8 @@ void qemu_edid_generate(uint8_t *edid, size_t size,
>      edid[20] = 0xa5;
>  
>      /* screen size: undefined */
Gerd, is this comment still correct?
> -    edid[21] = info->prefx * info->dpi / 2540;
> -    edid[22] = info->prefy * info->dpi / 2540;
> +    edid[21] = info->prefx * 254 / 100 / info->dpi;
> +    edid[22] = info->prefy * 254 / 100 / info->dpi;


The fix is good, but can be improved:

According to the standard, the values are "rounded to the nearest
centimeter". Currently they are not rounded, but truncated. So this
would be a better approximation:

edid[21] = (info->prefx * 254 / info->dpi + 50) / 100;

...


Regards,

Stefan