[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orgmode] Re: org-protocol: non-ASCII characters

From: Sebastian Rose
Subject: Re: [Orgmode] Re: org-protocol: non-ASCII characters
Date: Mon, 08 Feb 2010 12:30:09 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)

Jan Böcker <address@hidden> writes:
> On 06.02.2010 14:50, Jan Böcker wrote:
>> AFAIK, your current approach is correct.
> I was wrong. The attached patch fixes a bug in the encode_uri function.
> That fixes the non-ASCII characters problem in xournal for me.
> The gchar type is just typedef'd to char, which means it is signed. To
> get the byte value, it must be cast to unsigned int first.
> - Jan

Hi Jan and Daniel!

Sorry for answering with that long delay. I read Daniel's mail last
week, but I had to think about the answer.

I'll just describe, what the `org-protocol-unhex-string' functions do
here, and what they expect as arguments.

Basically, it is OK to url-encode each character who's binary
representation start with 1 (i.e., the value of the character is higher
than 127). The text to be url-encoded should be UTF-8 ideally.

If you use glib::ustring, it's easy to transform any iso-8859 string to
utf-8. Each character, whos binary representation start with a 1, has to
be url-encoded as well as the `%' character [1], but you could as
url-encode the entire utf-8 string.

The function that does the decoding is `org-protocol-unhex-string' which
in turn uses `org-protocol-unhex-compound'.

`man utf-8` shows, how org-protocol tries to decode characters.

The JavaScript-Funktion `encodeURIComponent()' returns exactly what we
need. It recodes a string to utf-8 and then encodes all characters,
except digits, ASCII letters and these punctuation characters: -_.!~*'()

See ECMA-262 Standard, Section 15.1.3
(http://bclary.com/2004/11/07/ecma-262.html#a-15.1.3 [2]):

   "The character is first transformed into a sequence of octets using
    the UTF-8 transformation..."

Again, note, that the decoding mechanism relies on the fact, that the
sequence to decode is url-encoded UTF-8.


  The url-encoded unicode representation of the German umlaut `ö' is
  `%C3%B6'. Thus

     (org-protocol-unhex-string "%C3%B6")

  gives you "ö".

  In iso-8859-1, the url-encoded representation of the same character `ö' was
  `%F6'. But

     (org-protocol-unhex-string "%F6")

  gives you "" - the empty string. There is no utf-8 character with this binary
  representation, since every byte starting with a 1 (i.e. is bigger than 127)
  starts a multibyte sequence (2 or more bytes).


     (org-protocol-unhex-string "%2F%3C")

  gives you, as expected,  "/<" which shows, that you could savely
  url-encode each and every character of a utf-8 encoded string.

==  Footnotes:

[1] The percent character `%' has to be encoded, if followed by
    [0-9A-Fa-f]{2}, because org-protocol will assume, that a sequence
    matching "\\(%[0-9a-f][0-9a-f]\\)+" is an encoded character. That
    said, a `%' has to be url-encoded, since one will hardly ever
    know for sure, that a `%' is never followed by "[0-9a-f][0-9a-f]".

[2] Get a PDF version of ECMA-262 third edition here:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]