--- Begin Message ---
|
Subject: |
problems with string ports and unicode |
|
Date: |
Sat, 7 Apr 2012 22:07:01 +0200 (CEST) |
Hi,
;;;; a very very short example script to describe the problem:
;; open a string port with unicode characters >= 0x0100
(define p (open-input-string "čtyří"))
Put the line into a script and start guile. You will see the output:
=> Backtrace:
That's all, and guile will hang in an eternal loop.
If you enter the line interactively into the REPL, everything works
properly and you can read all characters with (read-char p).
;;;; another very short script, which is possibly the same problem:
;; open a string port and unread a unicode character >= 0x0100
(define p (open-input-string "ibenik"))
(unread-char #\Š p)
Running these two lines as a script generates an error message:
=> ERROR: In procedure unread-char:
=> ERROR: Throw to key `encoding-error' with args
`("scm_ungetc" "conversion to port encoding failed" 84 #f #\540)'.
If you enter the lines interactively into the REPL, everything works
properly and you can read all characters with (read-char p).
Cheers,
Klaus Stehle
----------------------------
guile --version
guile (GNU Guile) 2.0.5
uname -srm
Linux 2.6.32-5-amd64 x86_64
echo $LANG
de_DE.UTF-8
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#11197: problems with string ports and unicode |
|
Date: |
Wed, 20 Jun 2012 23:03:02 +0200 |
|
User-agent: |
Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.1 (gnu/linux) |
Hi,
address@hidden (Ludovic Courtès) skribis:
> Indeed, it’s stuck in a deadlock:
>
> (gdb) bt
> #0 0x00007ffff75e1204 in __lll_lock_wait () from
> /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #1 0x00007ffff75dc4d4 in _L_lock_999 () from
> /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #2 0x00007ffff75dc2ea in pthread_mutex_lock () from
> /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #3 0x00007ffff7b30499 in scm_dynwind_pthread_mutex_lock
> (mutex=0x7ffff7dd28c0) at threads.c:1962
> #4 0x00007ffff7b2bb0e in scm_mkstrport (pos=0x2, str=0x4, modes=327680,
> caller=<value optimized out>) at strports.c:287
> #5 0x00007ffff7aac20b in display_backtrace_body (a=0x7fffffffc1a0) at
> backtrace.c:487
> #6 0x00007ffff7b46c7b in vm_regular_engine (vm=0x6f61f0, program=0x7f5d50,
> argv=0x6fa3b0, nargs=-1) at vm-i-system.c:895
> #7 0x00007ffff7ac039e in scm_call_3 (proc=0x7f5d50, arg1=<value optimized
> out>, arg2=<value optimized out>, arg3=<value optimized out>) at eval.c:500
> #8 0x00007ffff7b32504 in scm_internal_catch (tag=<value optimized out>,
> body=<value optimized out>, body_data=<value optimized out>, handler=<value
> optimized out>, handler_data=<value optimized out>) at throw.c:222
> #9 0x00007ffff7aabbba in scm_display_backtrace_with_highlights (stack=<value
> optimized out>, port=<value optimized out>, first=<value optimized out>,
> depth=<value optimized out>, highlights=<value optimized out>)
> at backtrace.c:558
> #10 0x00007ffff7ab725e in print_exception_and_backtrace (error_port=0x6f6170,
> tag=0x66d4c0, args=0x8e6ea0) at continuations.c:490
> #11 pre_unwind_handler (error_port=0x6f6170, tag=0x66d4c0, args=0x8e6ea0) at
> continuations.c:534
> #12 0x00007ffff7b46c7b in vm_regular_engine (vm=0x6f61f0, program=0x7f3ce0,
> argv=0x6fa300, nargs=-1) at vm-i-system.c:895
> #13 0x00007ffff7b4846e in scm_call_with_vm (vm=0x6f61f0, proc=0x7f3ce0,
> args=<value optimized out>) at vm.c:878
> #14 0x00007ffff7b296db in scm_to_stringn (str=0x8dba80, lenp=0x7fffffffc4e8,
> encoding=<value optimized out>, handler=SCM_FAILED_CONVERSION_ERROR) at
> strings.c:2102
> #15 0x00007ffff7b2bb73 in scm_mkstrport (pos=0x2, str=0x8dba80, modes=196608,
> caller=<value optimized out>) at strports.c:312
>
> This could be fixed by calling ‘scm_new_port_table_entry’ after having
> prepared the backing buffer, but the problem is that ‘pt->encoding’ is
> needed before.
Fixed in 03fcf93bff9f02a3d12ab86be4e67b996310aad4 (not particularly
elegant, but I couldn’t think of a better way.) The test in that commit
captures the initial problem.
I’m marking this bug as “done”. If you would like to discuss string
port encodings, separate binary/textual ports, or any other significant
change, you’re welcome to do so on address@hidden, of course.
Thanks!
Ludo’.
--- End Message ---