Re: `Compile' hang on remote files, not sure if it is a concurrent race issue

[Michael Albinus]

Fan Yang <Fan_Yang@sjtu.edu.cn> writes:

> Hi all,

Hi,

[sorry for the delay, I have been busy with other tasks last week]

> I’m look for any help for a hang issue when compiling remote files using 
> “compile.el”.
>
> Here is how I re-produce it:
>   - emacs -Q
>   - C-x C-f "/scp:vm01:pbd/virtio_balloon.c” (I’ve injected some typos into 
> this file to make the compilation error).
>   - M-x compile
>   - The “*compilation*” buffer shows with the following content:
>     -*- mode: compilation; default-directory: "/scp:vm01:/home/fan/pbd/" -*-
>     Compilation started at Sat May 23 23:18:30
>     
>     make
>     make -C /lib/modules/5.3.7-301.fc31.x86_64/build M=$PWD
>     make[1]: Entering directory '/usr/src/kernels/5.3.7-301.fc31.x86_64'
>       CC [M]  /home/fan/pbd/virtio_balloon.o
>   - Then Emacs hangs with no CPU activity.
>   - I start a terminal and run "pkill -SIGUSR2 Emacs”
>   - The Emacs becomes alive, with the “*compilation*” buffer displaying the 
> full content of error, and a “*Backtrace*”:
>     Debugger entered--entering a function:
>     * tramp-signal-hook-function(quit nil)
>       accept-process-output(#<process *tramp/scp vm01*> nil nil t)
>       tramp-accept-process-output(#<process *tramp/scp vm01*>)
>       tramp-wait-for-regexp(#<process *tramp/scp vm01*> nil 
> "\\(^\\|\0\\)[^#$\n]*///4e71ffd477719118f2d80d0a44acf5e3#\\$\\(\33[[0-9]+n\\)?\15?$")
>       tramp-wait-for-output(#<process *tramp/scp vm01*>)
>       tramp-send-command((tramp-file-name "scp" nil nil "vm01" nil 
> "/usr/src/kernels/5.3.7-301.fc31.x86_64" nil) "\\readlink 
> --canonicalize-missing /usr/src/kernels/5.3.7-301.fc31.x86_64 2>/dev/null; 
> echo tramp_exit_status $?")

What happens, if you open a shell on your remote vm01 host, and raise
the command

# \readlink --canonicalize-missing /usr/src/kernels/5.3.7-301.fc31.x86_64

> This happens only to two of my remote projects, and only when I inject such 
> typos that producing many lines of compilation errors and warnings.  Once I 
> run into this issue (for a specific remote project, and specific typos), I 
> can re-produce again and again.  I tried Emacs on mac (--with-ns) and Emacs 
> on Ubuntu 18.04, they both have this issue.
>
> I once wondered whether it is due to a race to the same TRAMP connection 
> since the “compilation--parse-region” seems can run concurrently with the 
> “compile” process.  I boldly hacked “compilation-filter” to make it deferred 
> until the compilation completes:
>
>   (defun compilation-filter (proc string)
>     "Process filter for compilation buffers.
>   Just inserts the text,
>    handles carriage motion (see `comint-inhibit-carriage-motion'),
>    and runs `compilation-filter-hook'."
>      (when (buffer-live-p (process-buffer proc))
>   +    (while (memq proc compilation-in-progress) (sleep-for 1))
>        (with-current-buffer (process-buffer proc)
>          (let ((inhibit-read-only t)
>
> Then the hang issue disappears.  Though I’m totally not sure if the issue is 
> really due to the race.  And I know this is only a temporary workaround.
>
> Do you know can TRAMP handle concurrent access to the same connection 
> correctly, especially in this case?  And do you have any idea/advise how to 
> fix this hang issue?

I haven't seen such race conflicts so far. In general, Tramp shall be
able to handle concurrent (asynchronous) processes. But if there is much
output from compilation, it could happen.

You could try to set tramp-verbose to 6, and rerun your test. Maybe we
see something in the traces, but it would be hard if there are many data.

> Thanks
> Fan

Best regards, Michael.