bug#22921: 25.1.50; Wishlist: There should be a new URL interface

[Lars Magne Ingebrigtsen]

The current interfaces to the URL library aren't very handy.  It's
`url-retrieve' and `url-retrieve-synchronously'.  The main problem is
that they're inconsistent and difficult to interact with.

For instance, most of the "inputs" to the functions are dynamically
bound specially variables like `url-request-method', and that sucks.
The buffers that are returned are inconsistent based on the URL
method -- HTTP buffers come with the headers first, while ftp buffers do
not.

So: `with-url-buffer'.

Typical usage would be

(with-url-buffer ("https://fsf.org";
                  :verbose t
                  :request-method "POST"
                  :timeout 10)
  (do-stuff-with-the-result-here))

That is, the callback is in the form.  There are no headers in the
buffer, and point is at point-min when the body is called.  Headers are
accessed with a new form like `(url-header "date")', and the headers are
available in a buffer-local variable called `url-headers'.

Upon completion of the body form, the buffer is killed, unless a
parameter :inhibit-kill-buffer is given.

Synchronous and asynchronous operation looks identical, but synchronous
operation happens when you say :synchronous t.

Implementation-wise, this happens with lexical bindings switched on, and
the body form is turned into a closure that's evaluated upon completion.

`with-url-buffer' will not support the exotic, never-actually-used
protocols like news:, irc:, imap:, etc, but only http:, https:, ftp:
and file:.



In GNU Emacs 25.1.50.23 (x86_64-unknown-linux-gnu, GTK+ Version 3.4.2)
 of 2016-03-06 built on stories
Repository revision: 20cc4182598c6c724f4556faf6bc322c9e58ad1d
Windowing system distributor 'The X.Org Foundation', version 11.0.11204000
System Description:     Debian GNU/Linux 7.9 (wheezy)

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GSETTINGS NOTIFY GNUTLS
LIBXML2 FREETYPE LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LANG: en_US
  locale-coding-system: iso-latin-1-unix

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  gnus-undo-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t

Recent messages:
Wrote /home/larsi/.newsrc
Saving /home/larsi/.newsrc.eld...
Saving file /home/larsi/.newsrc.eld...
Wrote /home/larsi/.newsrc.eld
Saving /home/larsi/.newsrc.eld...done
No more unseen articles
No more unread articles
next-line: End of buffer [2 times]
(No changes need to be saved)
Making completion list...

Load-path shadows:
/home/larsi/src/clock.el/clock hides /home/larsi/lisp/clock
/home/larsi/src/cddb.el/expect hides /home/larsi/lisp/expect
/home/larsi/src/pvr.el/pvr hides /home/larsi/lisp/pvr
~/pgnus/contrib/vcard hides /home/larsi/lisp/vcard
/home/larsi/src/cddb.el/captitle hides /home/larsi/lisp/captitle
~/lisp/zenirc-2.112/src/zenirc-example hides /home/larsi/lisp/zenirc-example
/home/larsi/lisp/dom hides /home/larsi/src/emacs/trunk/lisp/dom
~/pgnus/contrib/compface hides /home/larsi/src/emacs/trunk/lisp/image/compface

Features:
(shadow emacsbug sendmail binhex canlock ecomplete flow-fill eieio-opt
speedbar sb-image ezimage dframe find-func thingatpt view sgml-mode
misearch multi-isearch tabify imenu man vc vc-dispatcher copyright
vc-cvs eww shr-color color gnus-html url-queue help-fns url-cache
gnus-picon sort gnus-cite smiley ansi-color mm-archive gnus-async
gnus-dup qp gnus-ml gmane spam-gmane dns mm-url disp-table gnus-fun
gnus-mdrtn gnus-topic pop3 nndoc nnmbox nndraft utf-7 nnmh nnml nnfolder
network-stream nsm starttls nnir spam-report spam spam-stat gnus-uu yenc
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art
mm-uu mml2015 mm-view mml-smime smime dig nntp gnus-cache gnus-sum
gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source
utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range message
format-spec rfc822 mml mml-sec epa epg mailabbrev gmm-utils mailheader
gnus-win gnus nnheader gnus-util rmail rmail-loaddefs mail-utils movie
mkv shr svg imdb dom pvr debug debbugs-gnu easy-mmode derived debbugs
soap-client mm-decode mm-bodies mm-encode url-http tls gnutls url-auth
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-gw puny url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap warnings rng-xsd rng-dt rng-util xsd-regexp xml ido seq
flyspell ispell benchmark w3m browse-url doc-view subr-x dired
dired-loaddefs image-mode timezone w3m-hist w3m-fb w3m-ems wid-edit
w3m-ccl ccl w3m-favicon w3m-image w3m-proc w3m-util add-log mail-extr
mm-util mail-prsvr jka-compr cl finder-inf package epg-config
url-handlers url-parse auth-source cl-seq eieio byte-opt bytecomp
byte-compile cl-extra help-mode easymenu cconv eieio-core cl-macs gv
eieio-loaddefs cl-loaddefs pcase cl-lib password-cache url-vars
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev
obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 4019121 362371)
 (symbols 48 170790 312)
 (miscs 40 674 1974)
 (strings 32 317931 33454)
 (string-bytes 1 13992070)
 (vectors 16 43895)
 (vector-slots 8 1764946 243016)
 (floats 8 7065 3818)
 (intervals 56 929585 49118)
 (buffers 976 59)
 (heap 1024 212624 72649))

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no