bug#35969: 26.2, Excorporate

[Thomas Fitzsimmons]

OK, can you apply my patch, then run "make" again and confirm it works?

Thanks,
Thomas

Collin Day <dcday137@gmail.com> writes:

> Did a git pull and followed instructions above.
>
> There are some issues.  First, I need to run M-x package-install <RET>
> excorporate <RET>  four times because I see (each line after each
> invocation)
>
> package--with-response-buffer-1:
> https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not
> allowed
> package--with-response-buffer-1:
> https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed
> package--with-response-buffer-1:
> https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not
> allowed
>
> After the 4th call it compiles and becomes available.
>
> M-x customize-group <RET> excorporate, set up for no autoconfig right
> now, hit apply for current sessions, save not availiable....
>
> M-x excorporate <RET>
>
> enter uname and password
>
> Contacting host: outlook.office365.com:443
> error in process filter: exco--parse-xml-in-current-buffer: Server
> response is not an XML document
> error in process filter: Server response is not an XML documen
>
> *http outlook.office.365.com:443*

[...]

> So it appears not to work, at least on the current head as of today....
>
> Thanks!
>
> -C
>
> On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137@gmail.com> wrote:
>>
>> Sorry, I have not had the chance to.  A lot has been going on at my
>> place of employment.  It has crossed my mind, and as soon as I have
>> a chance, I will try what you suggested above.  Thanks, sorry for
>> the inconvenience.
>>
>> On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>>>
>>> Hi,
>>>
>>> Were you able to complete building Emacs?  I'd like to know if my patch
>>> is needed on top of Andreas's to make Excorporate work through your
>>> proxy.
>>>
>>> I'm not sure why the built version would interfere with the current
>>> installation.  If you were doing "make install" to a common prefix, that
>>> might explain it.  Instead you can try something like this:
>>>
>>> cd emacs-master [your github.com checkout]
>>> make
>>> mkdir test-home
>>> HOME=`pwd`/test-home ./src/emacs -Q
>>>
>>> That will ensure you're only running the built Emacs, and completely
>>> ignoring the packages installed in your home directory.
>>>
>>> Thomas
>>>
>>> "tenspd137 ." <dcday137@gmail.com> writes:
>>>
>>> > I haven't tried the first item on the list yet - ie reloading Andreas'
>>> > function.  i have tried to build emacs from
>>> > https://github.com/emacs-mirror/emacs.git because I can't reach the
>>> > official repo from my firewall for whatever reason.  I am sure it is
>>> > something here.  Anyway, I was having problems because my built
>>> > version (27.0.x) was getting seemingly mixed up with my current
>>> > installation.  I can try again later, but right now things are pretty
>>> > busy at work.
>>> >
>>> > Thanks.
>>> >
>>> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> 
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> "tenspd137 ." <dcday137@gmail.com> writes:
>>> >>
>>> >> > So - I am not sure if I did it correctly, but I copied this function
>>> >> > with Andreas' changes into a file:
>>> >> >
>>> >> > (defun url-https-proxy-after-change-function (_st _nd _length)
>>> >> >   (let* ((process-buffer (current-buffer))
>>> >> >          (proc (get-buffer-process process-buffer)))
>>> >> >     (goto-char (point-min))
>>> >> >     (when (re-search-forward "^\r?\n" nil t)
>>> >> >       (backward-char 1)
>>> >> >       ;; Saw the end of the headers
>>> >> >       (setq url-http-end-of-headers (set-marker (make-marker) (point)))
>>> >> >       (url-http-parse-response)
>>> >> >       (cond
>>> >> >        ((null url-http-response-status)
>>> >> >         ;; We got back a headerless malformed response from the
>>> >> >         ;; server.
>>> >> >         (url-http-activate-callback)
>>> >> >         (error "Malformed response from proxy, fail!"))
>>> >> >        ((= url-http-response-status 200)
>>> >> >         (if (gnutls-available-p)
>>> >> >             (condition-case e
>>> >> >                 (let ((tls-connection (gnutls-negotiate
>>> >> >                                        :process proc
>>> >> >                                        :hostname (url-host 
>>> >> > url-current-object)
>>> >> >                                        :verify-error nil)))
>>> >> >                   ;; check certificate validity
>>> >> >                   (setq tls-connection
>>> >> >                         (nsm-verify-connection tls-connection
>>> >> >                                                (url-host 
>>> >> > url-current-object)
>>> >> >                                                (url-port 
>>> >> > url-current-object)))
>>> >> >                   (with-current-buffer process-buffer (erase-buffer))
>>> >> >                   (set-process-buffer tls-connection process-buffer)
>>> >> >                   (setq url-http-after-change-function
>>> >> >                         'url-http-wait-for-headers-change-function)
>>> >> >                   (set-process-filter tls-connection 
>>> >> > 'url-http-generic-filter)
>>> >> >                   (process-send-string tls-connection
>>> >> >                                        ;; Use the non-proxy form of 
>>> >> > the request
>>> >> >                                        (let (url-http-proxy)
>>> >> >                                          (url-http-create-request))))
>>> >> >               (gnutls-error
>>> >> >                (url-http-activate-callback)
>>> >> >                (error "gnutls-error: %s" e))
>>> >> >               (error
>>> >> >                (url-http-activate-callback)
>>> >> >                (error "error: %s" e)))
>>> >> >           (error "error: gnutls support needed!")))
>>> >> >        (t
>>> >> >         (url-http-debug "error response: %d" url-http-response-status)
>>> >> >         (url-http-activate-callback))))))
>>> >> >
>>> >> > and then loaded it before running excorporate.  After that, I did M-x
>>> >> > excorporate, and the minibuffer returns:  error in process filter:
>>> >> > Server response is not an XML document
>>> >>
>>> >> In this scenario, if you immediately (without restarting Emacs/reloading
>>> >> anything) re-run M-x excorporate does it still fail?  I just want to
>>> >> make sure that's not a transient failure.  If it does fail the second
>>> >> time, can you post the HTTP response from the server?
>>> >>
>>> >> > When I do the similar test by loading the url-http-create-request with
>>> >> > Thomas's changes, I can get a connection and grab my schedule
>>> >> > through the proxy.
>>> >>
>>> >> OK.
>>> >>
>>> >> > Let me know if I need to try something different.
>>> >>
>>> >> Are you in a position to build Emacs master tip and retry the experiment
>>> >> without patching anything?
>>> >>
>>> >> Thanks,
>>> >> Thomas
>>> >>
>>> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons 
>>> >> > <fitzsim@fitzsim.org> wrote:
>>> >> >>
>>> >> >> Hi,
>>> >> >>
>>> >> >> Good to hear that the patch I posted worked!
>>> >> >>
>>> >> >> Yes, that's the patch that Andreas's commit
>>> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied.  From what I can 
>>> >> >> tell,
>>> >> >> you will not have that in Emacs 26.2.
>>> >> >>
>>> >> >> Can you try reverting my patch and applying Andreas's patch, and see 
>>> >> >> if
>>> >> >> Excorporate still works through the proxy?
>>> >> >>
>>> >> >> Thanks,
>>> >> >> Thomas
>>> >> >>
>>> >> >> "tenspd137 ." <dcday137@gmail.com> writes:
>>> >> >>
>>> >> >> > The patch Thomas seems to work from behind the proxy.  My current
>>> >> >> > emacs version is 26.2, so I would think it would include the commit
>>> >> >> > Andreas is talking about....  I went and looked it up - is this the
>>> >> >> > correct commit?
>>> >> >> >
>>> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
>>> >> >> > index 53798f7..817c5ce 100644
>>> >> >> > --- a/lisp/url/url-http.el
>>> >> >> > +++ b/lisp/url/url-http.el
>>> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the
>>> >> >> > retrieval buffer."
>>> >> >> > 'url-http-wait-for-headers-change-function)
>>> >> >> > (set-process-filter tls-connection 'url-http-generic-filter)
>>> >> >> > (process-send-string tls-connection
>>> >> >> > - (url-http-create-request)))
>>> >> >> > + ;; Use the non-proxy form of the request
>>> >> >> > + (let (url-http-proxy)
>>> >> >> > + (url-http-create-request))))
>>> >> >> > (gnutls-error
>>> >> >> > (url-http-activate-callback)
>>> >> >> > (error "gnutls-error: %s" e))
>>> >> >> >
>>> >> >> > Thanks!
>>> >> >> >
>>> >> >> > -C
>>> >> >> >
>>> >> >> >
>>> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab 
>>> >> >> > <schwab@linux-m68k.org> wrote:
>>> >> >> >>
>>> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>>> >> >> >>
>>> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
>>> >> >> >> > index 00803a103a..723d111d58 100644
>>> >> >> >> > --- a/lisp/url/url-http.el
>>> >> >> >> > +++ b/lisp/url/url-http.el
>>> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request
>>> >> >> >> >               ;; The request
>>> >> >> >> >               (or url-http-method "GET") " "
>>> >> >> >> >               (url-http--encode-string
>>> >> >> >> > -              (if using-proxy (url-recreate-url 
>>> >> >> >> > url-http-target-url) real-fname))
>>> >> >> >> > +              (if (and using-proxy
>>> >> >> >> > +                       (not (equal "https" (url-type 
>>> >> >> >> > url-http-target-url))))
>>> >> >> >> > +                  (url-recreate-url url-http-target-url)
>>> >> >> >> > +                real-fname))
>>> >> >> >>
>>> >> >> >> That should already be handled by commit 84613dae5c.
>>> >> >> >>
>>> >> >> >> Andreas.
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Andreas Schwab, schwab@linux-m68k.org
>>> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 
>>> >> >> >> A9DA AEC1
>>> >> >> >> "And now for something completely different."