--- Begin Message ---
|
Subject: |
gnutls guile bug receiving https data |
|
Date: |
Wed, 26 Feb 2020 16:20:09 +0100 |
|
User-agent: |
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 |
Hello,
I installed gnutls for guile and checked the gnutls module with this
simple code:
;;Guile version 3.0 and gnutls from git
,show version
GNU Guile 3.0.0.15-ff14b7
(gnutls-version)
$6 = "3.6.12"
;;Now, submitting this simple https request, you get an exception
(http-request "https://www.google.com";)
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum La
connessione TLS non è stata terminata in modo corretto.>
read_from_session_record_port)'.
;;instead, without https there are no errors
(http-request "http://www.google.com";) ;;works fine.
The error happens only on https://www.google.com and does not throw with
a lot of other https web sites.
Furthermore the error is not throw if the method is HEAD, so it is
related to the data part of the https answer.
I tried to enter in the internal implementation of the http web client
but after some tests I decided to do some simple tests at application level.
I rewrote the get-bytevector-all, with a loop that reads one byte per
time and the error was thrown anyway.
I catched the error and I've got the complete answer from the google web
server.
In the following there is my applicative solution, where I rewrote the
get-bytevector-all by adding the error checking and specifiyng to
http-request that the data has to be returned as a port (#:streaming? #t).
;;A macro to catch errors
(define-syntax my-noerr
(syntax-rules ()
((_ __error-return exp ...)
(let
((__st #f))
(catch #t
(lambda() exp ...)
(lambda (k . p) __error-return))))))
;;The rewriting of get-bytevector-all
(defun get-bytevector-all (port)
(u8-list->bytevector (let loop ((port port))
(let ((v (my-noerr (eof-object) (get-u8 port))))
(if (eof-object? v)
#nil
(cons v (loop port)))))))
;;the piece of code that now gives the correct result
(let-values (((a b)(http-request "https://www.google.com";
#:streaming? #t)))
(bytevector->string (get-bytevector-all b) "ISO-8859-1"))
As conclusion I can say that web modules read correctly the http answers
and, with some (one for me, the google web site) https sites there is a
misinterpretation of EOF in the layer between http and https.
Franco.
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#39800: gnutls guile bug receiving https data |
|
Date: |
Fri, 06 Mar 2020 23:46:35 +0100 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) |
Hi,
"address@hidden" <address@hidden> skribis:
> ;;Now, submitting this simple https request, you get an exception
> (http-request "https://www.google.com";)
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum La
> connessione TLS non è stata terminata in modo corretto.>
> read_from_session_record_port)'.
That happens when passing #:keep-alive? #f (the default), specifically
when reading the response body:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (http-request "https://www.google.com"; #:keep-alive? #f
#:streaming? #t)
$1 = #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
((date . #<date nanosecond: 0 second: 56 minute: 45 hour: 21 day: 6 month: 3
year: 2020 zone-offset: 0>) (expires . #<date nanosecond: 0 second: 0 minute: 0
hour: 0 day: 1 month: 1 year: 1970 zone-offset: 0>) (cache-control private
(max-age . 0)) (content-type text/html (charset . "ISO-8859-1")) (p3p .
"CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"") (server .
"gws") (x-xss-protection . "0") (x-frame-options . "SAMEORIGIN") (set-cookie .
"1P_JAR=2020-03-06-21; expires=Sun, 05-Apr-2020 21:45:56 GMT; path=/;
domain=.google.com; Secure") (set-cookie .
"NID=199=yXgE_KAGvxJbZAIGEXLt8CsEe3pre-RRLm1Jqap3b3iJRqZZq_PJ9wCT798mfDZ2TC5_3mKnM5KABSh8CguI64SsNoWHIc9EsW2osFltsIJnMswXhrtjQFDpfm_fUb6RDrWrqKHkOuvkG7Izp5im1Ys1TzGdztrFmOQV4FOraJk;
expires=Sat, 05-Sep-2020 21:45:56 GMT; path=/; domain=.google.com; HttpOnly")
(alt-svc . "quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\";
ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\";
ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000")
(accept-ranges none) (vary accept-encoding) (connection close)) port:
#<input-output: file 7f3ae7dd3540>>
$2 = #<input-output: file 7f3ae7dd3540>
scheme@(guile-user)> (define bv (get-bytevector-all $2))
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum The TLS connection
was non-properly terminated.> read_from_session_record_port)'.
--8<---------------cut here---------------end--------------->8---
The reason for this is that google.com closes the connection right away,
without sending a proper TLS “bye” message as is conventionally done.
Fixed in commit 076276c4f580368b4106316a77752d69c8f1494a, which will be
in 3.0.1.
Thanks,
Ludo’.
--- End Message ---