with-url

[Lars Ingebrigtsen]

I've basically implemented the previously discussed `with-url' macro.
Doc string below.  I'll push it to a feature branch tomorrow.

But I have some style questions I'm not quite sure about what the right
approach is.

First of all, the basic form looks like this:

  (with-url (headers "http://fsf.org/";)
    (message "The size of the FSF front page is %s" (buffer-size)))

And you can cram all the parameters as keywords after the URL.  For
instance, if you put :method "POST" in there, it'll use POST instead of
GET, and if you put :wait t in there, everything will be done
synchronously instead of asynchronously.  (Which is often nice when
testing code.)

First question: Should the `headers' variable binding be explicit or
implicit?  In Common Lisp there's a culture for having them all be very
explicit, but in Emacs we have a culture for hiding variables in
buffer-local variables.  One is nice and readable, and the other is nice
and short.

  (with-url (headers "http://fsf.org/";)
    (message "Content type is %s" (url-header headers 'content-type)))

==> Content type is text/html;charset=utf-8

vs

  (with-url ("http://fsf.org/";)
    (message "Content type is %s" (url-header 'content-type)))

The latter looks nicer, no?  The problem with that is this:

  (with-url ("http://fsf.org/";)
    (with-url ("http://goole.org/";)
      (message "Content type is %s" (url-header 'content-type))))

But I wanted the first one!

  (with-url ("http://fsf.org/";)
    (let ((buffer (current-buffer)))
      (with-url ("http://goole.org/";)
        (message "Content type is %s" (with-current-buffer buffer
                                        (url-header 'content-type))))))

*waugh*

And, yes, working with URL-ey stuff, you're often nesting these calls,
because the first call gives you some information, and then you want to
do another request...  But do you often refer back to the headers of the
previous request?  I don't know?

Opinions?


The second question is about error handling.  URL requests will fail,
either because the page doesn't exist, or the host is down, or it times
out.  (Yes, there are timeouts in the new interface.)  So in real life,
these calls should always look like...

  (with-url (headers "http://fsf.org/";)
    (if (url-header-ok-p headers)
        (message "Content type is %s" (url-header headers 'content-type)
        (message "I guess I'll do something else"))))

Although looking at `url-retrieve' calls in the Emacs code base, many
callers don't bother because error handling is boring.  And, besides,
it'll indent the code so much.  :-)

An alternative would be to say something like...  er...

  (with-url (headers "http://fsf.org/";)
    (ok
      (message "Content type is %s" (url-header headers 'content-type)))
    (error
      (message "I guess I'll do something else")))

That is, make the with-url have two body forms -- one which is called
when things went OK, and one when it didn't.  (And marked by those two
symbols.)

If the body doesn't have those symbols, then it's all an `ok' form and
no error callbacks.  Or

  (with-url (headers "http://fsf.org/";)
    (error
      (message "I guess I'll do something else"))
    (message "Content type is %s" (url-header headers 'content-type)))

That is, you can put the error handling form at the start if the first
symbol is `error' in that form.  Well, it can't be `error', but perhaps
`url-error'.  Or...

Opinions?  :-)



----

with-url is a Lisp macro in ‘../../with-url/lisp/url/with-url.el’.

(with-url (HEADER-VARIABLE URL &key WAIT TIMEOUT READ-TIMEOUT SILENT
INHIBIT-COOKIES INHIBIT-CACHE HEADERS (METHOD "GET") DATA
(DATA-CHARSET 'utf-8) DATA-ENCODING) &body BODY)

Retrieve URL and execute BODY with point in a buffer with the response.

Example:

  (with-url (headers "http://fsf.org/";)
    (message "The size of the FSF front page is %s" (buffer-size)))

The buffer is killed after BODY has exited.

HEADER-VARIABLE is bound to a structure that contains the response
headers and status.  These can be accessed with ‘url-header’ like this:

  (url-header headers "Content-Type")

Case is not significant.

Additional keywords can be given to ‘with-url’ to alter its operation.

:wait t
Normal ‘with-url’ operation is asynchronous.  If this parameter is given,
the retrieval will be synchronous instead.

:timeout SECONDS
Give up after approximately SECONDS seconds and execute BODY.

:read-timeout SECONDS
If no data has been received for the last SECONDS seconds, give
up and execute BODY.

:silent t
Issue no messages during operation.

:inhibit-cookies t
Neither send nor store cookies.

:headers ALIST
Add ALIST to the headers sent over to the server.  This should typically
look like

  (("User-Agent" "Emacs"))

If the header name is the same as one of the automatically
generated headers, the value from this list will override the
automatically generated header.  To disable the header
completely, use nil as the value.

Additional elements in this alist are interpreted as the
charset (defaulting to utf-8) and the encoding method (defaulting
to url-encode).

:method GET/POST/etc
The method to use for retrieving an HTTP(S) resource.  This defaults
to GET, and other popular values are POST, UPDATE and PUT.

:data STRING
Data to include in the body of the HTTP(S) request when using
POST, UPDATE or PUT.

:data-charset CHARSET
What charset this data should be interpreted as.  This defaults
to UTF-8.

:data-encoding ENCODING
When using the posting methods, the data is usually encoded in
some fashion.  Popular encodings are ‘url-form’ and ‘base64’.

[back]

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no