From MAILER-DAEMON Wed Jul 01 08:15:11 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZAGv1-0001Yp-Dr for mharc-bug-libunistring@gnu.org; Wed, 01 Jul 2015 08:15:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAGuu-0001SY-CE for bug-libunistring@gnu.org; Wed, 01 Jul 2015 08:15:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZAGut-0005Jm-ES for bug-libunistring@gnu.org; Wed, 01 Jul 2015 08:15:04 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:43639) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAGuk-0004xD-Mt; Wed, 01 Jul 2015 08:14:54 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:36624 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZAGuk-0001Ep-2l; Wed, 01 Jul 2015 08:14:54 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Daiki Ueno References: X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 13 Messidor an 223 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Wed, 01 Jul 2015 14:14:49 +0200 In-Reply-To: (Daiki Ueno's message of "Thu, 25 Jun 2015 16:42:02 +0900") Message-ID: <87h9poulae.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.10 Cc: bug-libunistring@gnu.org Subject: Re: [bug-libunistring] libunistring 0.9.6 first release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2015 12:15:10 -0000 Hi, Tested on x86_64-linux-gnu with: guix build libunistring \ --with-source=3Dhttp://alpha.gnu.org/gnu/libunistring/libunistring-0.9.= 6-rc1.tar.xz All tests pass, installation succeeds. Thanks, Ludo=E2=80=99. From MAILER-DAEMON Thu Jul 02 01:59:51 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZAXXL-00074p-UV for mharc-bug-libunistring@gnu.org; Thu, 02 Jul 2015 01:59:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60896) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAXXK-00074j-47 for bug-libunistring@gnu.org; Thu, 02 Jul 2015 01:59:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZAXXF-0005lC-FD for bug-libunistring@gnu.org; Thu, 02 Jul 2015 01:59:50 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57256) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAXXF-0005l4-BR; Thu, 02 Jul 2015 01:59:45 -0400 Received: from du-a.org ([219.94.251.20]:48150 helo=dhcp-217-92.nrt.redhat.com) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZAXXE-00024E-Ma; Thu, 02 Jul 2015 01:59:45 -0400 Message-ID: From: Daiki Ueno To: Dagobert Michelsen In-Reply-To: <5598BD32-0C58-48F2-988E-3A6C7F61B0D1@opencsw.org> (Dagobert Michelsen's message of "Tue, 30 Jun 2015 10:21:02 +0200") References: <16075B6B-F403-4E43-A9AB-0D1B7026D14A@opencsw.org> <5519FBE1-DBC0-4CC4-9756-68951866DD0E@opencsw.org> <11397B51-5BA8-4C41-B790-2568171F5D71@opencsw.org> <07B5D5EE-4A12-41AA-8EC2-39050ACB9C45@opencsw.org> <5598BD32-0C58-48F2-988E-3A6C7F61B0D1@opencsw.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) Date: Thu, 02 Jul 2015 14:59:38 +0900 MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e Cc: bug-libunistring@gnu.org Subject: Re: [bug-libunistring] [platform-testers] libunistring 0.9.6 first release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 05:59:51 -0000 Dagobert Michelsen writes: >> (dbx) run >> Running: test-u16-strstr >> (process id 23637) >> Reading libc_psr.so.1 >> terminating signal 14 SIGALRM >> (dbx) where >> dbx: program is not active >> (dbx) Sorry, I somehow missed the fact that the test really uses alarm for timeout handling: http://git.savannah.gnu.org/cgit/gnulib.git/tree/tests/unistr/test-u16-strstr.c#n41 I did the build on your system (thanks for giving me the access), and the test can indeed take over 5 seconds (= the timeout value): $ time ./test-u16-strstr real 0m5.281s user 0m5.128s sys 0m0.162s I suppose the problem is intermittent and might be fixed by running it when the system is not so busy, but if it still constantly occurs, we could increase the timeout value (though it should be carefully chosen, based on the algorithm complexity). Regards, -- Daiki Ueno From MAILER-DAEMON Thu Jul 02 02:17:48 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZAXoi-0006M3-1U for mharc-bug-libunistring@gnu.org; Thu, 02 Jul 2015 02:17:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38793) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAXof-0006Ii-Gk for bug-libunistring@gnu.org; Thu, 02 Jul 2015 02:17:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZAXoc-0007td-5K for bug-libunistring@gnu.org; Thu, 02 Jul 2015 02:17:45 -0400 Received: from mail.opencsw.org ([82.220.5.51]:37785) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAXob-0007so-Kv; Thu, 02 Jul 2015 02:17:42 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.opencsw.org (Postfix) with ESMTP id 3mMTjw0JpCz1Q1; Thu, 2 Jul 2015 08:17:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=opencsw.org; h=to :references:message-id:content-transfer-encoding:date:date :in-reply-to:x-mailer:from:from:subject:subject:mime-version :content-type:content-type; s=csw02; t=1435817856; bh=rdgaIRJshl Nx+VxCkSXaXUIskEn9IFYGLIxOn5gqLjQ=; b=iiqtrj65nTiSD4OFqsvWsNQMcS neEs9RICcH2+EEIKF00X7UAQIctzetjFeosRDOIWdRdzSg+4Pxah9CeZXsDsFdLg LWOPxWXGt9SZbnh9/oxi/VDMjO2DSZpbsQdCocgTwOY7UR6DqM1IFhe6KlGJMGWe GWUbMbTrwc0tX/qkjvA1zMHwvIcP+uzxkBWibuawOWbOh6zHtrLa29VJlxgRP5Va SNXk2w3TkA9+M7jgyMAyRQj7rRmUr4zYqu1Tdq/yeSGq9b8VHYW3ypNoTXArx6a/ cjnJoUtyEYLlMrfhU6IXgaYHBn9VdJDj7RgEhz+ZvbrSWkUYzRT2xlAD3NDQ== X-Virus-Scanned: amavisd-new at opencsw.org Received: from mail.opencsw.org ([127.0.0.1]) by localhost (mail.opencsw.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id OBUlKPbccnQI; Thu, 2 Jul 2015 08:17:36 +0200 (CEST) Received: from [10.36.80.173] (tmo-111-160.customers.d1-online.com [80.187.111.160]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: dam) by mail.opencsw.org (Postfix) with ESMTPSA id 3mMTjr0svHz1Q0; Thu, 2 Jul 2015 08:17:36 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) From: Dagobert Michelsen X-Mailer: iPhone Mail (12F70) In-Reply-To: Date: Thu, 2 Jul 2015 08:17:33 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <52A17301-AB71-48AF-BEE0-4966BEEB6E4B@opencsw.org> References: <16075B6B-F403-4E43-A9AB-0D1B7026D14A@opencsw.org> <5519FBE1-DBC0-4CC4-9756-68951866DD0E@opencsw.org> <11397B51-5BA8-4C41-B790-2568171F5D71@opencsw.org> <07B5D5EE-4A12-41AA-8EC2-39050ACB9C45@opencsw.org> <5598BD32-0C58-48F2-988E-3A6C7F61B0D1@opencsw.org> To: Daiki Ueno X-detected-operating-system: by eggs.gnu.org: Solaris 10 X-Received-From: 82.220.5.51 Cc: "bug-libunistring@gnu.org" Subject: Re: [bug-libunistring] [platform-testers] libunistring 0.9.6 first release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 06:17:46 -0000 Hi Daiki, > Am 02.07.2015 um 07:59 schrieb Daiki Ueno : >=20 > Dagobert Michelsen writes: >=20 >>> (dbx) run = =20 >>> Running: test-u16-strstr=20 >>> (process id 23637) >>> Reading libc_psr.so.1 >>> terminating signal 14 SIGALRM >>> (dbx) where >>> dbx: program is not active >>> (dbx) >=20 > Sorry, I somehow missed the fact that the test really uses alarm for > timeout handling: > http://git.savannah.gnu.org/cgit/gnulib.git/tree/tests/unistr/test-u16-str= str.c#n41 >=20 > I did the build on your system (thanks for giving me the access), and > the test can indeed take over 5 seconds (=3D the timeout value): >=20 > $ time ./test-u16-strstr=20 > real 0m5.281s > user 0m5.128s > sys 0m0.162s >=20 > I suppose the problem is intermittent and might be fixed by running it > when the system is not so busy, but if it still constantly occurs, we > could increase the timeout value (though it should be carefully chosen, > based on the algorithm complexity). The machine is rather slow than overloaded, I guess more time (10?) would be= good. Best regards -- Dago= From MAILER-DAEMON Thu Jul 02 16:46:39 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZAlNX-0005rK-0u for mharc-bug-libunistring@gnu.org; Thu, 02 Jul 2015 16:46:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37525) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAkB0-0002Y7-3y for bug-libunistring@gnu.org; Thu, 02 Jul 2015 15:29:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZAkAw-0004gC-U4 for bug-libunistring@gnu.org; Thu, 02 Jul 2015 15:29:38 -0400 Received: from smtp01.uchicago.edu ([128.135.12.77]:21074) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAkAw-0004g8-Nh for bug-libunistring@gnu.org; Thu, 02 Jul 2015 15:29:34 -0400 Received: from xm-route.ad.uchicago.edu (xm-hub-01-prod.ad.uchicago.edu [10.50.194.11]) by smtp01.uchicago.edu (Postfix) with ESMTP id 614DC3DC45F for ; Thu, 2 Jul 2015 14:29:34 -0500 (CDT) Received: from XM-MBX-03-PROD.ad.uchicago.edu ([fe80::94b6:791b:ce18:6b55]) by xm-hub-01-prod.ad.uchicago.edu ([10.50.194.23]) with mapi id 14.03.0169.001; Thu, 2 Jul 2015 14:29:34 -0500 From: Seiya Kawashima To: "bug-libunistring@gnu.org" Thread-Topic: A bug in u-strtok.h and the fix in libunistring-0.9.5 Thread-Index: AdC0/SD+xDPkv9bBSAO0FidpF8PcaA== Date: Thu, 2 Jul 2015 19:29:34 +0000 Message-ID: <8CA79C4C24A06A48A11BA99F564B78ED37BDD9AA@XM-MBX-03-PROD.ad.uchicago.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.135.74.39] Content-Type: multipart/alternative; boundary="_000_8CA79C4C24A06A48A11BA99F564B78ED37BDD9AAXMMBX03PRODaduc_" MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 128.135.12.77 X-Mailman-Approved-At: Thu, 02 Jul 2015 16:46:37 -0400 Subject: [bug-libunistring] A bug in u-strtok.h and the fix in libunistring-0.9.5 X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 19:29:39 -0000 --_000_8CA79C4C24A06A48A11BA99F564B78ED37BDD9AAXMMBX03PRODaduc_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, First of all, thank you for the great Unicode library in C. Recently I've b= een using the library intensively. During my experiments with libunistring-0.9.5, I've found an error in u-str= tok.h as below. The lines starting with "////" are my changes. /* Move past the token. */ { UNIT *token_end =3D U_STRPBRK (str, delim); if (token_end) { /* NUL-terminate the token. */ *token_end =3D 0; *ptr =3D token_end + 1; //// These lines should be something like below. //// *ptr =3D token_end + (sizeof(uint8_t) * u8_strmblen(token_end)= ); //// *token_end =3D 0; } else *ptr =3D NULL; } So the original code tries to start the next search without checking how ma= ny bytes are actually taken by a matched delimiter but assuming 1 by "token= _end + 1". When the delimiter takes more than one UNIT such as a delimiter = in Japanese, this assumption fails and starts the next search from an inval= id location which is in the middle of a Unicode character. To solve the issue, one can define U_STRMBLEN with u8_strmblen,u16_strmblen= and u32_strmblen accordingly and call it like *ptr =3D token_end + (sizeof= (UNIT) * U_STRMBLEN(token_end)) instead of *ptr =3D token_end + 1. I've checked the source code and git log but could not find the relevant ch= anges. However, if I've missed the change or misunderstood the logic and if= it works as expected as it is, please discard this email. Thank you once again for your great work. Seiya Seiya Kawashima Intermediate Application Programmer | Department of Radiology The University of Chicago Biological Science 5841 S. Maryland Ave. | Rm. IB-012 | Chicago, IL 60637 Office: 773-834-1791 --_000_8CA79C4C24A06A48A11BA99F564B78ED37BDD9AAXMMBX03PRODaduc_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,

First of all, thank you for the great Unicode library in C. Recently I've b= een using the library intensively.

During my experiments with libunistring-0.9.5, I've found an error in u-str= tok.h as below. The lines starting with "////" are my changes.
  /* Move past the token. */
  {
    UNIT *token_end =3D U_STRPBRK (str, delim);
    if (token_end)
      {
        /* NUL-terminate the token. = ; */
        *token_end =3D 0;
        *ptr =3D token_end + 1;
        //// These lines should be somet= hing like below.
        //// *ptr =3D token_end + (sizeof= (uint8_t) * u8_strmblen(token_end));   
       ////  *token_end =3D 0;
      }
    else
      *ptr =3D NULL;
  }

So the original code tries to start the next search without checking how ma= ny bytes are actually taken by a matched delimiter but assuming 1 by "= token_end + 1". When the delimiter takes more than one UNIT such a= s a delimiter in Japanese, this assumption fails and starts the next search from an invalid location which is in the middle= of a Unicode character.

To solve the issue, one can define U_STRMBLEN with u8_strmblen,u16_strmblen= and u32_strmblen accordingly and call it like *ptr =3D token_end + (si= zeof(UNIT) * U_STRMBLEN(token_end)) instead of *ptr =3D token_end + 1.<= br>
I've checked the source code and git log but could not find the relevant ch= anges. However, if I've missed the change or misunderstood the logic and if= it works as expected as it is, please discard this email.

Thank you once again for your great work.
Seiya

Seiya Kawashima
Intermediate Application = Programmer | Department of Radiology

The University of Chicago Biological Science
5841
S. Maryland Ave. | Rm. IB-012 | Chicago, IL 60637
Office:
773-834-1791

--_000_8CA79C4C24A06A48A11BA99F564B78ED37BDD9AAXMMBX03PRODaduc_-- From MAILER-DAEMON Thu Jul 02 22:59:06 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZArBy-0008Uw-Ri for mharc-bug-libunistring@gnu.org; Thu, 02 Jul 2015 22:59:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50752) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZArBv-0008SB-30 for bug-libunistring@gnu.org; Thu, 02 Jul 2015 22:59:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZArBs-00005I-AX for bug-libunistring@gnu.org; Thu, 02 Jul 2015 22:59:03 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48787) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZArBl-0008Vv-Kd; Thu, 02 Jul 2015 22:58:53 -0400 Received: from du-a.org ([219.94.251.20]:51831 helo=dhcp-217-92.nrt.redhat.com) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZArBk-0004pz-Ka; Thu, 02 Jul 2015 22:58:53 -0400 Message-ID: From: Daiki Ueno To: Seiya Kawashima In-Reply-To: <8CA79C4C24A06A48A11BA99F564B78ED37BDD9AA@XM-MBX-03-PROD.ad.uchicago.edu> (Seiya Kawashima's message of "Thu, 2 Jul 2015 19:29:34 +0000") References: <8CA79C4C24A06A48A11BA99F564B78ED37BDD9AA@XM-MBX-03-PROD.ad.uchicago.edu> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) Date: Fri, 03 Jul 2015 11:58:45 +0900 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e Cc: "bug-libunistring@gnu.org" , bug-gnulib@gnu.org Subject: Re: [bug-libunistring] A bug in u-strtok.h and the fix in libunistring-0.9.5 X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jul 2015 02:59:05 -0000 --=-=-= Content-Type: text/plain Hello, Seiya Kawashima writes: > During my experiments with libunistring-0.9.5, I've found an error in u-strtok.h as below. The lines > starting with "////" are my changes. > > /* Move past the token. */ > { > UNIT *token_end = U_STRPBRK (str, delim); > if (token_end) > { > /* NUL-terminate the token. */ > *token_end = 0; > *ptr = token_end + 1; > //// These lines should be something like below. > //// *ptr = token_end + (sizeof(uint8_t) * u8_strmblen(token_end)); > //// *token_end = 0; > } > else > *ptr = NULL; > } > > So the original code tries to start the next search without checking how many bytes are actually > taken by a matched delimiter but assuming 1 by "token_end + 1". When the delimiter takes more > than one UNIT such as a delimiter in Japanese, this assumption fails and starts the next search from > an invalid location which is in the middle of a Unicode character. Indeed, thanks for the report. > To solve the issue, one can define U_STRMBLEN with u8_strmblen,u16_strmblen and u32_strmblen > accordingly and call it like *ptr = token_end + (sizeof(UNIT) * U_STRMBLEN(token_end)) instead of > *ptr = token_end + 1. Yes, I've pushed a slightly modified fix (attached) in your name, along with test cases. Regards, -- Daiki Ueno --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-unistr-uN-strtok-handle-multibyte-delimiters.patch >From 2c5543396b7c208b34fa714430801945aefb1fe8 Mon Sep 17 00:00:00 2001 From: Seiya Kawashima Date: Fri, 3 Jul 2015 11:42:43 +0900 Subject: [PATCH] unistr/uN-strtok: handle multibyte delimiters Previously, uN_strtok moved PTR to the next unit to the token end. When DELIM contained a multibyte character, the new position could be a middle of a multibyte character. * lib/unistr/u-strtok.h (FUNC): Place PTR at the next character after the token. * lib/unistr/u8-strtok.c (U_STRMBLEN): New macro. * lib/unistr/u16-strtok.c (U_STRMBLEN): New macro. * lib/unistr/u32-strtok.c (U_STRMBLEN): New macro. * modules/unistr/u8-strtok (Depends-on): Depend on unistr/u8-strmblen. * modules/unistr/u16-strtok (Depends-on): Depend on unistr/u16-strmblen. * modules/unistr/u32-strtok (Depends-on): Depend on unistr/u32-strmblen. * tests/unistr/test-u-strtok.h: New file. * tests/unistr/test-u8-strtok.c: New file. * tests/unistr/test-u16-strtok.c: New file. * tests/unistr/test-u32-strtok.c: New file. * modules/unistr/u8-strtok-tests: New file. * modules/unistr/u32-strtok-tests: New file. * modules/unistr/u16-strtok-tests: New file. Copyright-paperwork-exempt: yes Co-authored-by: Daiki Ueno --- ChangeLog | 26 +++++++++++++ lib/unistr/u-strtok.h | 2 +- lib/unistr/u16-strtok.c | 1 + lib/unistr/u32-strtok.c | 1 + lib/unistr/u8-strtok.c | 1 + modules/unistr/u16-strtok | 1 + modules/unistr/u16-strtok-tests | 14 +++++++ modules/unistr/u32-strtok | 1 + modules/unistr/u32-strtok-tests | 15 +++++++ modules/unistr/u8-strtok | 1 + modules/unistr/u8-strtok-tests | 14 +++++++ tests/unistr/test-u-strtok.h | 86 +++++++++++++++++++++++++++++++++++++++++ tests/unistr/test-u16-strtok.c | 37 ++++++++++++++++++ tests/unistr/test-u32-strtok.c | 37 ++++++++++++++++++ tests/unistr/test-u8-strtok.c | 37 ++++++++++++++++++ 15 files changed, 273 insertions(+), 1 deletion(-) create mode 100644 modules/unistr/u16-strtok-tests create mode 100644 modules/unistr/u32-strtok-tests create mode 100644 modules/unistr/u8-strtok-tests create mode 100644 tests/unistr/test-u-strtok.h create mode 100644 tests/unistr/test-u16-strtok.c create mode 100644 tests/unistr/test-u32-strtok.c create mode 100644 tests/unistr/test-u8-strtok.c diff --git a/ChangeLog b/ChangeLog index 8893c75..373583f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,29 @@ +2015-07-03 Seiya Kawashima (tiny-change) + and Daiki Ueno + + unistr/uN-strtok: handle multibyte delimiters + Previously, uN_strtok moved PTR to the next unit to the token end. + When DELIM contained a multibyte character, the new position could + be a middle of a multibyte character. + * lib/unistr/u-strtok.h (FUNC): Place PTR at the next character + after the token. + * lib/unistr/u8-strtok.c (U_STRMBLEN): New macro. + * lib/unistr/u16-strtok.c (U_STRMBLEN): New macro. + * lib/unistr/u32-strtok.c (U_STRMBLEN): New macro. + * modules/unistr/u8-strtok (Depends-on): Depend on + unistr/u8-strmblen. + * modules/unistr/u16-strtok (Depends-on): Depend on + unistr/u16-strmblen. + * modules/unistr/u32-strtok (Depends-on): Depend on + unistr/u32-strmblen. + * tests/unistr/test-u-strtok.h: New file. + * tests/unistr/test-u8-strtok.c: New file. + * tests/unistr/test-u16-strtok.c: New file. + * tests/unistr/test-u32-strtok.c: New file. + * modules/unistr/u8-strtok-tests: New file. + * modules/unistr/u32-strtok-tests: New file. + * modules/unistr/u16-strtok-tests: New file. + 2015-07-02 Friedrich Haubensak update-copyright: fix test failure with perl >= 5.22 (trivial) diff --git a/lib/unistr/u-strtok.h b/lib/unistr/u-strtok.h index 4cd6830..edafa1b 100644 --- a/lib/unistr/u-strtok.h +++ b/lib/unistr/u-strtok.h @@ -40,9 +40,9 @@ FUNC (UNIT *str, const UNIT *delim, UNIT **ptr) UNIT *token_end = U_STRPBRK (str, delim); if (token_end) { + *ptr = token_end + U_STRMBLEN (token_end); /* NUL-terminate the token. */ *token_end = 0; - *ptr = token_end + 1; } else *ptr = NULL; diff --git a/lib/unistr/u16-strtok.c b/lib/unistr/u16-strtok.c index 62e2f49..df36cf7 100644 --- a/lib/unistr/u16-strtok.c +++ b/lib/unistr/u16-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint16_t #define U_STRSPN u16_strspn #define U_STRPBRK u16_strpbrk +#define U_STRMBLEN u16_strmblen #include "u-strtok.h" diff --git a/lib/unistr/u32-strtok.c b/lib/unistr/u32-strtok.c index 0bb16ff..f8ef999 100644 --- a/lib/unistr/u32-strtok.c +++ b/lib/unistr/u32-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint32_t #define U_STRSPN u32_strspn #define U_STRPBRK u32_strpbrk +#define U_STRMBLEN u32_strmblen #include "u-strtok.h" diff --git a/lib/unistr/u8-strtok.c b/lib/unistr/u8-strtok.c index efb6a9a..1e4e6ef 100644 --- a/lib/unistr/u8-strtok.c +++ b/lib/unistr/u8-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint8_t #define U_STRSPN u8_strspn #define U_STRPBRK u8_strpbrk +#define U_STRMBLEN u8_strmblen #include "u-strtok.h" diff --git a/modules/unistr/u16-strtok b/modules/unistr/u16-strtok index 50773f5..98cfcf6 100644 --- a/modules/unistr/u16-strtok +++ b/modules/unistr/u16-strtok @@ -9,6 +9,7 @@ Depends-on: unistr/base unistr/u16-strspn unistr/u16-strpbrk +unistr/u16-strmblen configure.ac: gl_LIBUNISTRING_MODULE([0.9], [unistr/u16-strtok]) diff --git a/modules/unistr/u16-strtok-tests b/modules/unistr/u16-strtok-tests new file mode 100644 index 0000000..c6b71d8 --- /dev/null +++ b/modules/unistr/u16-strtok-tests @@ -0,0 +1,14 @@ +Files: +tests/unistr/test-u16-strtok.c +tests/unistr/test-u-strtok.h +tests/macros.h + +Depends-on: + +configure.ac: + +Makefile.am: +TESTS += test-u16-strtok +check_PROGRAMS += test-u16-strtok +test_u16_strtok_SOURCES = unistr/test-u16-strtok.c +test_u16_strtok_LDADD = $(LDADD) $(LIBUNISTRING) diff --git a/modules/unistr/u32-strtok b/modules/unistr/u32-strtok index 2e19c81..1206248 100644 --- a/modules/unistr/u32-strtok +++ b/modules/unistr/u32-strtok @@ -9,6 +9,7 @@ Depends-on: unistr/base unistr/u32-strspn unistr/u32-strpbrk +unistr/u32-strmblen configure.ac: gl_LIBUNISTRING_MODULE([0.9], [unistr/u32-strtok]) diff --git a/modules/unistr/u32-strtok-tests b/modules/unistr/u32-strtok-tests new file mode 100644 index 0000000..319e0a9 --- /dev/null +++ b/modules/unistr/u32-strtok-tests @@ -0,0 +1,15 @@ +Files: +tests/unistr/test-u32-strtok.c +tests/unistr/test-u-strtok.h +tests/macros.h + +Depends-on: +unistr/u32-uctomb + +configure.ac: + +Makefile.am: +TESTS += test-u32-strtok +check_PROGRAMS += test-u32-strtok +test_u32_strtok_SOURCES = unistr/test-u32-strtok.c +test_u32_strtok_LDADD = $(LDADD) $(LIBUNISTRING) diff --git a/modules/unistr/u8-strtok b/modules/unistr/u8-strtok index 28d7ed5..657b6fd 100644 --- a/modules/unistr/u8-strtok +++ b/modules/unistr/u8-strtok @@ -9,6 +9,7 @@ Depends-on: unistr/base unistr/u8-strspn unistr/u8-strpbrk +unistr/u8-strmblen configure.ac: gl_LIBUNISTRING_MODULE([0.9], [unistr/u8-strtok]) diff --git a/modules/unistr/u8-strtok-tests b/modules/unistr/u8-strtok-tests new file mode 100644 index 0000000..f6e2588 --- /dev/null +++ b/modules/unistr/u8-strtok-tests @@ -0,0 +1,14 @@ +Files: +tests/unistr/test-u8-strtok.c +tests/unistr/test-u-strtok.h +tests/macros.h + +Depends-on: + +configure.ac: + +Makefile.am: +TESTS += test-u8-strtok +check_PROGRAMS += test-u8-strtok +test_u8_strtok_SOURCES = unistr/test-u8-strtok.c +test_u8_strtok_LDADD = $(LDADD) $(LIBUNISTRING) diff --git a/tests/unistr/test-u-strtok.h b/tests/unistr/test-u-strtok.h new file mode 100644 index 0000000..735f033 --- /dev/null +++ b/tests/unistr/test-u-strtok.h @@ -0,0 +1,86 @@ +/* Test of uN_strtok() functions. + Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +static void +test_u_strtok (void) +{ + { + UNIT input[] = { 'f', 'o', 'o', 0 }; + const UNIT delim[] = { 0 }; + UNIT *state; + const UNIT *result = U_STRTOK (input, delim, &state); + ASSERT (result == input); + } + + { + UNIT input[] = + { 'A', 'B', 'C', ' ', 'A', 'B', 'C', 'D', 'A', 'B', ' ', '\t', + 'A', 'B', 'C', 'D', 'A', 'B', 'C', 'D', 'A', 'B', 'D', 'E', 0 + }; + const UNIT delim[] = { ' ', '\t', 0 }; + UNIT *state; + const UNIT *result; + result = U_STRTOK (input, delim, &state); + ASSERT (result == input); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == input + 4); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == input + 12); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == NULL); + } + + /* Check for multibyte delimiters. */ + { + ucs4_t u_input[] = + { 'A', 'B', 'C', 0x3000, 'A', 'B', 'C', 'D', 'A', 'B', 0x3000, 0x3001, + 'A', 'B', 'C', 'D', 'A', 'B', 'C', 'D', 'A', 'B', 'D', 'E', 0 + }; + ucs4_t u_delim[] = { 0x3000, 0x3001, 0 }; + size_t input_len = 6 * SIZEOF (u_input); + UNIT *input = (UNIT *) malloc (input_len); + size_t delim_len = 6 * SIZEOF (u_delim); + UNIT *delim = (UNIT *) malloc (delim_len); + UNIT *state; + const UNIT *result; + UNIT *ptr, *first_ptr, *second_ptr; + size_t i; + for (i = 0, ptr = input; i < SIZEOF (u_input) && u_input[i] != 0; i++) + { + int ret = U_UCTOMB (ptr, u_input[i], input_len - (ptr - input)); + if (i == 4) + first_ptr = ptr; + if (i == 12) + second_ptr = ptr; + ptr += ret; + } + *ptr = 0; + for (i = 0, ptr = delim; i < SIZEOF (u_delim) && u_delim[i] != 0; i++) + { + int ret = U_UCTOMB (ptr, u_delim[i], delim_len - (ptr - delim)); + ptr += ret; + } + *ptr = 0; + result = U_STRTOK (input, delim, &state); + ASSERT (result == input); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == first_ptr); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == second_ptr); + result = U_STRTOK (NULL, delim, &state); + ASSERT (result == NULL); + } +} diff --git a/tests/unistr/test-u16-strtok.c b/tests/unistr/test-u16-strtok.c new file mode 100644 index 0000000..cbd95ef --- /dev/null +++ b/tests/unistr/test-u16-strtok.c @@ -0,0 +1,37 @@ +/* Test of u16_strtok() function. + Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include "unistr.h" + +#include +#include + +#include "macros.h" + +#define UNIT uint16_t +#define U_STRTOK u16_strtok +#define U_UCTOMB u16_uctomb +#include "test-u-strtok.h" + +int +main (void) +{ + test_u_strtok (); + + return 0; +} diff --git a/tests/unistr/test-u32-strtok.c b/tests/unistr/test-u32-strtok.c new file mode 100644 index 0000000..04cded7 --- /dev/null +++ b/tests/unistr/test-u32-strtok.c @@ -0,0 +1,37 @@ +/* Test of u32_strtok() function. + Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include "unistr.h" + +#include +#include + +#include "macros.h" + +#define UNIT uint32_t +#define U_STRTOK u32_strtok +#define U_UCTOMB u32_uctomb +#include "test-u-strtok.h" + +int +main (void) +{ + test_u_strtok (); + + return 0; +} diff --git a/tests/unistr/test-u8-strtok.c b/tests/unistr/test-u8-strtok.c new file mode 100644 index 0000000..6745ea8 --- /dev/null +++ b/tests/unistr/test-u8-strtok.c @@ -0,0 +1,37 @@ +/* Test of u8_strtok() function. + Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include "unistr.h" + +#include +#include + +#include "macros.h" + +#define UNIT uint8_t +#define U_STRTOK u8_strtok +#define U_UCTOMB u8_uctomb +#include "test-u-strtok.h" + +int +main (void) +{ + test_u_strtok (); + + return 0; +} -- 2.4.2 --=-=-=-- From MAILER-DAEMON Thu Jul 02 23:46:35 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZArvv-00024i-9d for mharc-bug-libunistring@gnu.org; Thu, 02 Jul 2015 23:46:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34524) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZArvs-00020J-KS for bug-libunistring@gnu.org; Thu, 02 Jul 2015 23:46:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZArvr-00031C-N3 for bug-libunistring@gnu.org; Thu, 02 Jul 2015 23:46:32 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:49501) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZArvk-0002sS-OM; Thu, 02 Jul 2015 23:46:24 -0400 Received: from du-a.org ([219.94.251.20]:52142 helo=dhcp-217-92.nrt.redhat.com) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZArvk-0002MS-7x; Thu, 02 Jul 2015 23:46:24 -0400 Message-ID: From: Daiki Ueno To: bug-libunistring@gnu.org Date: Fri, 03 Jul 2015 12:46:20 +0900 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e Cc: platform-testers@gnu.org Subject: [bug-libunistring] libunistring 0.9.6 second release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jul 2015 03:46:33 -0000 Hello, The second release candidate of libunistring 0.9.6 is now available at: http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz.sig This incorporates only a couple of changes since the last time: the timeout adjustment for expensive tests and uN_strtok fixes for multibyte delimiters. I hope to release it as 0.9.6 in next week. Please send feedback to bug-libunistring@gnu.org (or to bug-gnulib@gnu.org for the Gnulib stuff). Thanks, -- Daiki Ueno From MAILER-DAEMON Fri Jul 03 00:59:33 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZAt4X-00073E-4z for mharc-bug-libunistring@gnu.org; Fri, 03 Jul 2015 00:59:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50049) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAt4U-000728-9R for bug-libunistring@gnu.org; Fri, 03 Jul 2015 00:59:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZAt4T-00007w-Cj for bug-libunistring@gnu.org; Fri, 03 Jul 2015 00:59:30 -0400 Received: from mail2.vodafone.ie ([213.233.128.44]:64341) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZAt4O-00006i-OD; Fri, 03 Jul 2015 00:59:24 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApYQADwVllVtTf4A/2dsb2JhbABbgxIfNV+BPb12hXMBAgKBU0wBAQEBAQGBC0EBBINeAQEEMgFGEAsNCwkWDwkDAgECATQRBgEMAQUCAQGILwEIvQWQOQEBAQEGAQEBAR6LSoUGB4QrAQSUEoRhiD9Eg1CDA494JoN7PTEBAQEBgkcBAQE Received: from unknown (HELO localhost.localdomain) ([109.77.254.0]) by mail2.vodafone.ie with ESMTP; 03 Jul 2015 05:59:23 +0100 Message-ID: <559616AA.9010106@draigBrady.com> Date: Fri, 03 Jul 2015 05:59:22 +0100 From: =?windows-1252?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Daiki Ueno , bug-libunistring@gnu.org References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 213.233.128.44 Cc: platform-testers@gnu.org Subject: Re: [bug-libunistring] libunistring 0.9.6 second release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jul 2015 04:59:31 -0000 On 03/07/15 04:46, Daiki Ueno wrote: > Hello, > > The second release candidate of libunistring 0.9.6 is now available at: > http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz > http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz.sig > > This incorporates only a couple of changes since the last time: the > timeout adjustment for expensive tests and uN_strtok fixes for multibyte > delimiters. I hope to release it as 0.9.6 in next week. > > Please send feedback to bug-libunistring@gnu.org (or to > bug-gnulib@gnu.org for the Gnulib stuff). Fedora 22 x86_64 # PASS: 485 # SKIP: 10 Fedora 18 ppc64 (power7) # PASS: 485 # SKIP: 10 OS X Yosomite x86_64 # PASS: 484 # SKIP: 11 FreeBSD 9. i386 # PASS: 484 # SKIP: 11 thanks! Pádraig. From MAILER-DAEMON Fri Jul 03 19:35:25 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZBAUP-0004pT-Do for mharc-bug-libunistring@gnu.org; Fri, 03 Jul 2015 19:35:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34866) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZBAUM-0004oZ-E3 for bug-libunistring@gnu.org; Fri, 03 Jul 2015 19:35:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZBAUJ-0003Yx-7K for bug-libunistring@gnu.org; Fri, 03 Jul 2015 19:35:22 -0400 Received: from mail-qk0-x230.google.com ([2607:f8b0:400d:c09::230]:35997) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZBAUJ-0003Yq-28 for bug-libunistring@gnu.org; Fri, 03 Jul 2015 19:35:19 -0400 Received: by qkei195 with SMTP id i195so81482574qke.3 for ; Fri, 03 Jul 2015 16:35:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=RsUdjcVD0ERCOELQypiHsyEp2fFtmDEE4SuYpC7vx9Y=; b=Pwlx9PGF3E7LCyynSUFrE+D/4+jz487bY8//il84Uz+8RxgSLlOLNbyR5TXL+CnXf1 KQTcg5Y2/Qbj8s/ChEilefuyKgZ6L37qSMai+MOo1a1GnQHbRvIFSHSbyvSdFNBmoibO DXy8HiDvNhvsGRsH4Y+cziavofEYfb+ljCoPnyGFYcQDqBL/dfdaNaba9Fv6OHhhqo4z YTJ2BQOKfFg4J/bh0jGE4XO1NJkZLWNVUXj7H1KkYob4GEdV4vh3iF4dRmoA+I720FqM TAmgWC55cBvDII2AbrA2dB5lBJ+vumdfCjsmurPZo+ALq5IRCBYff01MxufhuvHnY45b 4H4Q== X-Received: by 10.140.22.147 with SMTP id 19mr54131938qgn.52.1435966518631; Fri, 03 Jul 2015 16:35:18 -0700 (PDT) Received: from disco.erlich.nygenome.org ([162.212.111.2]) by mx.google.com with ESMTPSA id l79sm5189500qhl.24.2015.07.03.16.35.17 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jul 2015 16:35:18 -0700 (PDT) Message-ID: <55971C52.8060400@gmail.com> Date: Fri, 03 Jul 2015 19:35:46 -0400 From: Assaf Gordon User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: bug-libunistring@gnu.org References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:400d:c09::230 Subject: Re: [bug-libunistring] [platform-testers] libunistring 0.9.6 second release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jul 2015 23:35:23 -0000 Hello, On 07/02/2015 11:46 PM, Daiki Ueno wrote: > The second release candidate of libunistring 0.9.6 is now available at: > http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz > http://alpha.gnu.org/gnu/libunistring/libunistring-0.9.6-rc2.tar.xz.sig On FreeBSD 10.1 x86_64, 1 failure: FAIL: test-striconveh ===================== test-striconveh.c:519: assertion 'retval == -1 && errno == EILSEQ' failed FAIL test-striconveh (exit status: 134) with a printf I see errno was 22=EINVAL . No failures on (x86_64): CentOS 6.5, 7 Debian 8.1, 7.6, OpenSUSE 13.2 Ubuntu 14.04, 15.04, Fedora 21,20 OpenBSD 5.7, 5.6, 5.5 NetBSD 6.1.4 GNU Hurd 0.6, 0.5 (i686) No failures on OpenSolaris 5.11 intel+sparc. On OpenIndiana (Illumos/OpenSolaris-like), build fails with: --- /bin/sh ../libtool --tag=CC --preserve-dup-deps --mode=link gcc -g -O2 -o test-alloca-opt test-alloca-opt.o libtests.a ../lib/libunistring.la libtests.a libtool: link: gcc -g -O2 -o .libs/test-alloca-opt test-alloca-opt.o libtests.a ../lib/.libs/libunistring.so libtests.a -R/usr/local/lib Undefined first referenced symbol in file getlocalename_l ../lib/.libs/libunistring.so ld: fatal: symbol referencing errors. No output written to .libs/test-alloca-opt --- might be similar to this: http://lists.gnu.org/archive/html/bug-gnulib/2015-02/msg00042.html regards, - assaf From MAILER-DAEMON Mon Jul 06 00:24:10 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZBxww-0002Wg-G9 for mharc-bug-libunistring@gnu.org; Mon, 06 Jul 2015 00:24:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40515) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZBxwt-0002UU-5M for bug-libunistring@gnu.org; Mon, 06 Jul 2015 00:24:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZBxwo-00055o-Gi for bug-libunistring@gnu.org; Mon, 06 Jul 2015 00:24:07 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50517) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZBxwo-00055k-Dz; Mon, 06 Jul 2015 00:24:02 -0400 Received: from du-a.org ([219.94.251.20]:55581 helo=dhcp-217-92.nrt.redhat.com) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZBxwn-0005Ql-Sh; Mon, 06 Jul 2015 00:24:02 -0400 Message-ID: From: Daiki Ueno To: Assaf Gordon References: <55971C52.8060400@gmail.com> Date: Mon, 06 Jul 2015 13:23:54 +0900 In-Reply-To: <55971C52.8060400@gmail.com> (Assaf Gordon's message of "Fri, 03 Jul 2015 19:35:46 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e Cc: bug-libunistring@gnu.org Subject: Re: [bug-libunistring] [platform-testers] libunistring 0.9.6 second release candidate X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jul 2015 04:24:08 -0000 Hello Assaf, Thanks for testing and the report. Assaf Gordon writes: > On FreeBSD 10.1 x86_64, 1 failure: > > FAIL: test-striconveh > ===================== > test-striconveh.c:519: assertion 'retval == -1 && errno == EILSEQ' failed > FAIL test-striconveh (exit status: 134) > > with a printf I see errno was 22=EINVAL . I suppose this will be fixed if libiconv is installed. > On OpenIndiana (Illumos/OpenSolaris-like), > build fails with: > --- > /bin/sh ../libtool --tag=CC --preserve-dup-deps --mode=link gcc -g -O2 -o test-alloca-opt test-alloca-opt.o libtests.a ../lib/libunistring.la libtests.a > libtool: link: gcc -g -O2 -o .libs/test-alloca-opt test-alloca-opt.o libtests.a ../lib/.libs/libunistring.so libtests.a -R/usr/local/lib > Undefined first referenced > symbol in file > getlocalename_l ../lib/.libs/libunistring.so > ld: fatal: symbol referencing errors. No output written to .libs/test-alloca-opt > --- > might be similar to this: > http://lists.gnu.org/archive/html/bug-gnulib/2015-02/msg00042.html Yes, and it affects gettext build too, as Illumos people seem to be aware of it: http://permalink.gmane.org/gmane.os.illumos.devel/22249 I'm not sure what's the best way to fix it. Maybe we could have a configure time check for getlocalename_l? Regards, -- Daiki Ueno From MAILER-DAEMON Wed Jul 08 05:49:40 2015 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1ZClz2-00083S-E4 for mharc-bug-libunistring@gnu.org; Wed, 08 Jul 2015 05:49:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55769) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZClyz-0007x9-2D for bug-libunistring@gnu.org; Wed, 08 Jul 2015 05:49:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZClyv-0000Km-0X for bug-libunistring@gnu.org; Wed, 08 Jul 2015 05:49:36 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47321) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZClyu-0000Ki-Sj; Wed, 08 Jul 2015 05:49:32 -0400 Received: from du-a.org ([219.94.251.20]:38994 helo=dhcp-217-92.nrt.redhat.com) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ZClyu-0000z5-3S; Wed, 08 Jul 2015 05:49:32 -0400 Message-ID: From: Daiki Ueno To: info-gnu@gnu.org, bug-libunistring@gnu.org Date: Wed, 08 Jul 2015 18:49:25 +0900 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e Subject: [bug-libunistring] libunistring-0.9.6 released X-BeenThere: bug-libunistring@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports for GNU libunistring List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2015 09:49:38 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable GNU libunistring is a library that provides functions for manipulating Unicode strings and for manipulating C strings according to the Unicode standard. Here are the compressed sources: ftp://ftp.gnu.org/gnu/libunistring/libunistring-0.9.6.tar.gz (3.5MB) ftp://ftp.gnu.org/gnu/libunistring/libunistring-0.9.6.tar.xz (1.9MB) Here are the GPG detached signatures[*]: ftp://ftp.gnu.org/gnu/libunistring/libunistring-0.9.6.tar.gz.sig ftp://ftp.gnu.org/gnu/libunistring/libunistring-0.9.6.tar.xz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the MD5 and SHA1 checksums: dfae4ff5583649ed24d8f368f1d0543b libunistring-0.9.6.tar.gz cb09c398020c27edac10ca590e9e9ef3 libunistring-0.9.6.tar.xz 87cb4c18fbb1e0f1d8eb38311093835a924b4b12 libunistring-0.9.6.tar.gz d34dd5371c4b34863a880f2206e2d00532effdd6 libunistring-0.9.6.tar.xz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify libunistring-0.9.6.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys ueno@unixuser.org and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.15 Libtool 2.4.2 Gnulib v0.1-513-g9b6679d New in 0.9.6: * The data tables and line breaking algorithm have been updated to Unicode version 8.0.0. Regards, =2D-=20 Daiki Ueno --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJVnPIlAAoJENYFhI7X5phx0JUP/0rlHSX/NKt5wVRUj1aORelB 4/FwmtygW2cjP7tGan3jiSWcMLVvKLW0jkrs27Px9kQn0EA5tEvA/8ApuKTX07OD 8PXDRUfK3qRbcPWi35QSYdflgMgJR8I/JelNRVxnywnJ6QyTInHFN0epxzPNVXCw oMirCNV5N6xEBgDAla8xZl/9I8LnDUiPYH3hyAVW320UK8S6pBvqigaXX2YP0Dj9 wZpJaWEMjOay0v3L9zqbBplrZE/GgLBvl9/aEoJ/QA35Jr25ff6W/UJ/Kwjx0CGB cZahYi+Qg4Z9mhKkRblHEuBnkiSV6z9HTIWvqiNEP9qcx25b3aXMI9NlCBJoinzK f9q9pl08ppt7/nS3w9O4nRavnJldo9rohF6teAKAlwoXC9NggdtsQxKlKsbyHaha ZWYrIN+ORyi5FQiej379iQLTXlbN+8RjuU/tI2OcgZZ0F2iVPfDl6eAwYiuQQnWT uiFTIdT+HMdt04yEH4Ya2veNj6Beg+H8VjmhtwRkzMEHspVJavG7Swu9ygY/FBgQ VOzNSssRI6gfeAvqcUxAHf/QtzamtZfYGyJn/q21RUyaqcnGcEL7L+o1rsy49gjj uRMRotN/N721HsYTeV0KNZnosVkNVQSswWVT57LpIc8rhDkC8GbKEYGa3ScqQLYf GBavCAM4N6rWcXHR7OCD =wGkp -----END PGP SIGNATURE----- --=-=-=--