[bug#54302] [PATCH] nls: Implement translation thresholds.

[Ludovic Courtès]

Ping!  :-)

Ludovic Courtès <ludo@gnu.org> skribis:

> Hi!
>
> Julien Lepiller <julien@lepiller.eu> skribis:
>
>> manual and cookbook: only include new languages when they reach 10%
>> completion. Remove languages when they fall below 5%.
>
> SGTM (or even 15%/10%).
>
>> website (unrelated to this repo, but still important): only include new
>> languages when they reach 80% completion. Remove languages when they
>> fall below 60%. The reason for the higher threshold is that the website
>> acts as some sort of advertisement, so we want a higher quality than
>> half English half translated.
>
> SGTM.
>
>>>>From 37071410629a7d70c9b4e4f673f2c625d3ed4b47 Mon Sep 17 00:00:00 2001
>> Message-Id: 
>> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Tue, 8 Mar 2022 13:14:58 +0100
>> Subject: [PATCH 1/3] doc: Document inclusion requirements for new
>>  translations.
>>
>> * doc/contributing.texi (Translating Guix)[Conditions for Inclusion]:
>> New section.
>
> [...]
>
>> +There are no conditions for adding new translations of the guix and
>> +guix-packages components, other than they need at least one translated
>
> @code{guix} and @code{guix-packages}
>
>> +Given that the website is dedicated to new users, we want its translation
>
> “web site” (two words).
>
>> +target.  Everytime we synchronise translations, developpers need to
>
> “developers” and (if you feel overseas-inclined) “synchronize”.
>
>> +When a language is included, it may be removed in the future, if it stays
>
> Remove comma.
>
>>>>From 5cbb70ebcbf141cd05fa60bf0bfa806125a56381 Mon Sep 17 00:00:00 2001
>> Message-Id: 
>> <5cbb70ebcbf141cd05fa60bf0bfa806125a56381.1646763369.git.julien@lepiller.eu>
>> In-Reply-To: 
>> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
>> References: 
>> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Tue, 8 Mar 2022 19:11:38 +0100
>> Subject: [PATCH 2/3] maint: Implement translation thresholds.
>>
>> Do not download new translations for the cookbook and the manual when
>> they are below 10% completion, and remove existing translations when
>> they fall below 5%.
>>
>> * Makefile.am (download-po): Implement translation thresholds.
>> ---
>>  Makefile.am | 20 +++++++++++++++++---
>>  1 file changed, 17 insertions(+), 3 deletions(-)
>>
>> diff --git a/Makefile.am b/Makefile.am
>> index 8850c4562c..164804d96a 100644
>> --- a/Makefile.am
>> +++ b/Makefile.am
>> @@ -1066,21 +1066,35 @@ WEBLATE_REPO = 
>> https://framagit.org/tyreunom/guix-translations
>>  # form.
>>  download-po:
>>      dir=$$(mktemp -d); \
>> -    git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \
>> +    git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations" && \
>>      for domain in po/doc po/guix po/packages; do \
>>          for po in "$$dir/translations/$$domain"/*.po; do \
>>              translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 
>> | cut -f1 -d' '); \
>> +            untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 
>> 2>&1 | cut -f4 -d' '); \
>> +                    untranslated=$${untranslated:-0}; \
>> +                    total=$$(($$translated+$$untranslated)); \
>>              target=$$(basename "$$po"); \
>>              target="$$domain/$$target"; \
>> -            if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \
>> +            msgfmt -c "$$po"; \
>> +            if msgfmt -c "$$po" && [ "$$translated" != "0" ] && ([ 
>> "$$domain" != "po/doc" ] || [ "$$translated" -gt $$(($$total/10)) ] || [ -f 
>> $$target ]); then \
>>                  msgfilter --no-wrap -i "$$po" cat > "$$po".tmp; \
>>                  mv "$$po".tmp "$$target"; \
>>                  echo "copied $$target."; \
>>              else \
>> -                echo "WARN: $$target ($$translated translated messages) was 
>> not added/updated."; \
>> +                echo "WARN: $$target ($$translated translated messages 
>> ($$((translated/total*100))%)) was not added/updated."; \
>>              fi; \
>>          done; \
>>      done; \
>> +    for po in po/doc/*.po; do \
>> +        translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | 
>> cut -f1 -d' '); \
>> +        untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | 
>> cut -f4 -d' '); \
>> +            untranslated=$${untranslated:-0}; \
>> +            total=$$(($$translated + $$untranslated)); \
>> +            if [ "$$translated" -lt "$$(($$total/20))" ]; then \
>> +                echo "WARN: $$po was removed because it is below the 5% 
>> threshold: $$((translated/total*100))%"; \
>> +                    rm $$po; \
>> +            fi; \
>> +    done; \
>
> LGTM, but this is getting a bit hairy.  :-)
>
> No concrete suggestions, but it would be great if we could somehow split
> it and/or move it to a script in build-aux/ (that’d avoid double dollar
> signs) and/or write it in Scheme.  Future work…
>
>> From 726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8 Mon Sep 17 00:00:00 2001
>> Message-Id: 
>> <726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8.1646763369.git.julien@lepiller.eu>
>> In-Reply-To: 
>> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
>> References: 
>> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Tue, 8 Mar 2022 19:14:47 +0100
>> Subject: [PATCH 3/3] nls: Enforce translation thresholds.
>>
>> * po/doc/guix-cookbook.es.po: Remove file.
>> * po/doc/guix-cookbook.fa.po: Remove file.
>> * po/doc/guix-cookbook.fi.po: Remove file.
>> * po/doc/guix-cookbook.uk.po: Remove file.
>> * po/doc/local.mk: Remove them.
>> * doc/local.mk: Remove them.
>
> [...]
>
>>  # If adding a language, update the following variables, and info_TEXINFOS.
>>  MANUAL_LANGUAGES = de es fa fi fr it ko pt_BR ru sk zh_CN
>> -COOKBOOK_LANGUAGES = de es fa fi fr ko pt_BR ru sk uk zh_Hans
>> +COOKBOOK_LANGUAGES = de fr ko pt_BR ru sk zh_Hans
>
> Should we also remove fa, fi, it, ko, and sk from MANUAL_LANGUAGES and
> info_TEXINFOS?
>
>   https://translate.fedoraproject.org/projects/guix/documentation-manual/
>
> Thank you!
>
> Ludo’.