Re: byte-order marks

[Neil Jerram]

Andy Wingo <address@hidden> writes:

> What do people think about this attached patch?
>
> Andy
>
>
>>From 831c3418941f2d643f91e3076ef9458f700a2c59 Mon Sep 17 00:00:00 2001
> From: Andy Wingo <address@hidden>
> Date: Mon, 28 Jan 2013 22:41:34 +0100
> Subject: [PATCH] detect and consume byte-order marks for textual ports

In case an example is of any help for this discussion, here's some code
that I wrote to consume a possible BOM on contacts data downloaded from
Google:

(define (read-csv file-name)
  (let ((s (utf16->string (get-bytevector-all (open-input-file file-name))
                          'little)))

    ;; Discard possible byte order mark.
    (if (and (>= (string-length s) 1)
             (char=? (string-ref s 0) #\xfeff))
        (set! s (substring s 1)))

    ...))

I wonder if I chose to use utf16->string because there wasn't - at that
time - a way of handling the BOM without slurping into a string first?
However it was a long time ago now so I really can't be sure what the
context was. 

Regards,
        Neil