Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors

[Ludovic Courtès]

Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Tue, Apr 20, 2021 at 03:21:13AM +0200, pelzflorian (Florian Pelz) wrote:
>> > git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd
>
> It seems this is the bad commit.  Downloading the enlightenment
> substitute got stuck and after a few minutes displayed the usual TLS
> error.

Note that on master there have been changes in this area since this
commit, in particular 20c08a8a45d0f137ead7c05e720456b2aea44402.

I assume the error we’re after is still this one:

>>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 
>> 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': 
>> Resource temporarily unavailable, try again.

I believe the attached patch “addresses” this problem.

Now, I’m a bit skeptical: the error above is GNUTLS_E_AGAIN, which
happens if sendmsg(2) returns EAGAIN, which supposedly only happens on
datagram (UDP) sockets or on non-blocking sockets, neither of which
applies here.

This change should be safe, though since I’m not sure why we’re getting
EAGAIN in the first place, I wonder what could happen (infinite loop?).

Could you give it a try?

Can you reproduce the substitute issue in a simpler environment?
For instance, by running:

  rm -rf ~/.cache/guix/substitute/
  ./pre-inst-env guix weather $(guix package -A |head -2000 |cut -f1) 

This makes 2000 pipelined GETs to https://ci.guix.gnu.org, which is
probably similar to what you observe during system installation.

Thanks for the thorough debugging and bisecting, as always!

Ludo’.

diff --git a/guix/http-client.scm b/guix/http-client.scm
index a2e11a1b73..51bba250d5 100644
--- a/guix/http-client.scm
+++ b/guix/http-client.scm
@@ -38,7 +38,7 @@
   #:use-module (guix utils)
   #:use-module (guix base64)
   #:autoload   (gcrypt hash) (sha256)
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module ((guix build utils)
                 #:select (mkdir-p dump-port))
   #:use-module ((guix build download)
@@ -163,7 +163,8 @@ reusing stale cached connections."
       (if (or (and (eq? key 'system-error)
                    (= EPIPE (system-error-errno `(,key ,@args))))
               (and (eq? key 'gnutls-error)
-                   (eq? (first args) error/invalid-session))
+                   (memq (first args)
+                         (list error/again error/invalid-session)))
               (memq key
                     '(bad-response bad-header bad-header-component)))
           #f
diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 48309f9b3a..65940591a9 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -45,7 +45,7 @@
                 #:select (uri-abbreviation nar-uri-abbreviation
                           (open-connection-for-uri
                            . guix:open-connection-for-uri)))
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module (guix progress)
   #:use-module ((guix build syscalls)
                 #:select (set-thread-name))
@@ -417,7 +417,8 @@ server certificates."
         (if (or (and (eq? key 'system-error)
                      (= EPIPE (system-error-errno `(,key ,@args))))
                 (and (eq? key 'gnutls-error)
-                     (eq? (first args) error/invalid-session))
+                     (memq (first args)
+                           (list error/again error/invalid-session)))
                 (memq key '(bad-response bad-header bad-header-component)))
             (proc (open-connection-for-uri/cached uri
                                                   #:verify-certificate? #f