Re: libidn2 tr#46 support

[Simon Josefsson]

Tim Ruehsen <address@hidden> writes:

> On Thursday, November 3, 2016 9:44:45 PM CET Simon Josefsson wrote:
>> Tim Rühsen <address@hidden> writes:
>> > Hi Simon,
>> > 
>> > are you goind to support TR#46 for libidn2 ?
>> 
>> Hi Tim.  I would like to, but there is too little time.  I very much
>> like help with it.
>
> Simon, well known problem everywhere :-( Too less time and more and more 
> projects needs attention... huge lack of developers as it seems.

Right.  I am really looking for help with libidn and libidn2, so if
anyone reading this would like to step up: the sky is the limit :-)

>> > Today I rewrote IDN code for wget (libidn) to use libidn2 + libunistring.
>> > But that makes sense in the long term only if TR#46 is in sight.
>> 
>> What APIs in libunistring do you need?  NFC?  libidn2 uses libunistring
>> (from gnulib) internally, so if there is some simple API that would be
>> useful to export, to avoid having to link with libunistring as well, I'm
>> happy to add export it from libidn2.
>
> At least here the GNU linker is clever enough to link libunistring just one 
> time (checked with ldd).

There may be some code duplication though, since libidn2 imports some
libunistring source code via gnulib.

> What I need and already do via libunistring API is lowercasing and NFC/NFCK 
> (not 100% sure which one to use).
> I think adding two flags to the idn2 API should do it.

Adding this should be easy.  Maybe it is separate APIs rather than a
flag, I will have to think about that.

> Shouldn't TR#46 be mandatory ? If not, another flag would do it.

For IDNA2008 to be usable in practice, I believe TR#46 is necessary.
That is where the world appears to be going, judging by browser adoption
at least.  Patrik Fältström (one of the IDNA2008 people) suggested
NFC+IDNA2008, but there is no implementation experience with that and it
has the obvious security problems.

However TR46 is not a trivial algorithm to implement.  Last time I
looked, it appeared similar in effort as implementing IDNA2008 itself,
but I may have over-estimated the effort required.

>> The idea is that libidn2 should be a usable IDNA2008 library, and that
>> probably involves exposing NFC and TR46 APIs as well.
>
> Hiding libunistring details is IMO a good idea here.

Agreed.

> Ah yes, another API wish... decoding complete ACE (xn--) domain strings into 
> UTF-8. Currently not possible with idn2_register_u8(). It just works for 
> single labels with memory allocation for each label. So now each application 
> has to provide it's own loop with catching all corner cases... tedious and 
> unneeded code duplication.
>
> An additional flag would do it here as well (and also keep backward 
> compatibility).

The funny thing is... IDNA2008 does not specify how to go from xn--
format to Unicode format.  My plan was to implement something, and
document that as an IETF draft so that others can do the same.  The
basics are clear (punycode encode labels) but I recall some details
better be clarified.

However I agree such an API is needed for any realistic usage.  Libidn2
is currently really a low-level IDNA2008 tool, and there is some mileage
to go before it can be used in the real world in my opinion.  TR46 is
the major piece, exposing some additional APIs is another but simpler.

/Simon

signature.asc
Description: PGP signature