bug#19860: 25.0.50; One url-retrieve-synchronously call running concurrently with another can freeze it

[Dmitry Gutov]

This applies both to master and emacs-24 (with a caveat).

With sufficiently suitable server, calling long-ops-now or long-ops-idle
can lead to the "outer" request never returning:

--8<---------------cut here---------------start------------->8---
(defun long-ops-idle ()
  (interactive)
  (run-with-idle-timer 0.2 nil #'long-outer-op)
  (run-with-idle-timer 0.5 nil #'long-inner-op))

(defun long-ops-now ()
  (interactive)
  (run-with-timer 0.2 nil #'long-inner-op)
  (long-outer-op))

(defun long-outer-op ()
  (message "retrieving outer")
  (url-retrieve-synchronously "http://localhost:9292";)
  (message "outer retrieved"))

(defun long-inner-op ()
  (message "retrieving inner")
  (url-retrieve-synchronously "http://localhost:9292";)
  (message "inner retrieved"))
--8<---------------cut here---------------end--------------->8---

The window freezes, the last message in the echo area being "inner
retrieved" (but C-g breaks out of it, and if we're calling
`long-ops-idle', the "outer" request succeeds, too).

It doesn't happen with just any web server, though. I can reproduce it
using both master and emacs-24 using this Ruby server:

--8<---------------cut here---------------start------------->8---
run proc { |env|
  sleep 1
  [200, {'Content-Type' => 'text/plain'}, [""]]
}
--8<---------------cut here---------------end--------------->8---

Save it as test.ru and launch with 'rackup test.ru'.

But only with master using this Python server (an example for those who
don't have Ruby installed):

--8<---------------cut here---------------start------------->8---
from wsgiref.util import setup_testing_defaults
from wsgiref.simple_server import make_server
import time

def simple_app(environ, start_response):
    setup_testing_defaults(environ)

    status = '200 OK'
    headers = [('Content-type', 'text/plain')]

    time.sleep(1)

    start_response(status, headers)

    return ""

httpd = make_server('', 9292, simple_app)
print "Serving on port 9292..."
httpd.serve_forever()
--8<---------------cut here---------------end--------------->8---

Save it as server.py, and launch with 'python server.py'.

The key difference seems to be Keep-Alive (enabled by default by WEBrick
(Ruby); wsgiref doesn't support it at all). Adding "Connection: close"
to the Ruby example server makes emacs-24 behave as expected with it.

And someone who doesn't want to run the server can also try the
following URLs with master: http://xip.io/ and http://ya.ru/. When
calling xip.io, the outer request eventually returns, but much later
than expected; when calling ya.ru, it doesn't.

The present bug report is based on this long-standing issue, which I
haven't been able to reliably reproduce until now:
https://github.com/dgutov/robe/issues/36

In GNU Emacs 25.0.50.5 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2015-02-13 on axl
Repository revision: 69e38a5b1fdb5ac20440b6ce7ba1fc2cd575f4e6
Windowing system distributor `The X.Org Foundation', version 11.0.11601901
System Description:     Ubuntu 14.10