[Orgmode] Re: with-parsed-tramp-file-name

[Dan Davison]

Hi Austin,

"Austin F. Frank" <address@hidden> writes:

> On Thu, Sep 30 2010, Dan Davison wrote:
>
>> Just btw, I use Org every day at the moment (with emacs24) to execute
>> code (mostly R) on remote machines. The method I use is to store all
>> Org files locally, and use the :dir header argument to specify the
>> machine on which to run R, e.g.
>
> At some point I was (naively?) worried about the portability of this
> kind of path.

What portability concerns did/do you have?

>  Additionally, I do sometimes want to run things locally,
> like if I'm offline.

With :dir that is easy: either alter the :dir header arg so that it
explicitly points to a local path, or remove it entirely and allow it to
default to default-directory. Like other babel parameters, :dir can be
set at various levels: as a header arg, in an Org property, in
a #+babel: line, in org-babel-default-header-args:LANG, or in
org-babel-default-header-args.

In fact, since babel header args can be lisp forms which are eval'd,
then a good solution may be to use an elisp variable as a flag with
which we can switch between local and remote execution (please pull, as
there was a bug preventing lisp forms being used like this in
properties):

--8<---------------cut here---------------start------------->8---
* local/remote switching
  :PROPERTIES:
  :dir: (if dan/local-p "." "/address@hidden:")
  :results: output
  :END:

src_emacs-lisp{(setq dan/local-p t)}

#+begin_src sh
hostname
pwd
#+end_src

#+results:
: Luscinia
: /home/dan

src_emacs-lisp{(setq dan/local-p nil)}

#+begin_src sh
hostname
pwd
#+end_src

#+results:
: oak
: /home/davison
--8<---------------cut here---------------end--------------->8---


Otoh aIui your workflow involves maintaining Org source files on a
remote machine, therefore it is only possible to work offline if you
have a local copy. I find it works well to keep the Org source files
locally, and only use the remote machine when necessary for execution
and access to remote data files.

>  Because a file that is opened over TRAMP usually
> transparently starts processes over TRAMP, and because it's less typing,
> and because of my untested portability concern, I have stuck to using
> the edit-and-execute over TRAMP method, with all paths defined relative
> to the edited file.

The two methods are very closely related however, and I certainly want
Babel to work correctly using both methods. The :dir method basically
does

(let ((default-directory <dir-header-arg-value>))
   ... evaluate org babel code as normal ...)

which is very similar to the effect of working with an Org file opened
via tramp on a remote machine.

>
>> #+babel:   :dir /address@hidden:~/bench
>> [...]
>> #+begin_src R
>> system("hostname", intern=TRUE)
>> #+end_src
>>
>> #+results:
>> : oak
>>
>> Do you get the same error evaluating R remotely using this method?
>
> I do, when using the previously-reported config file.

Have you made any progress understanding what was going on with the
error below? I just found myself suffering from a similar error (invalid
function `with-parsed-tramp-file-name') and it turned out to be because
I had compiled the Org files yesterday. After running `make clean' it
went away. Not that that is a satisfactory solution of course.

Question: we have

(declare-function with-parsed-tramp-file-name "tramp" (filename var &rest body))

but is it correct to use declare-function on a macro? That might be the
problem?

>
>> What's your emacs version?
>
> GNU Emacs 24.0.50.7 (powerpc-apple-darwin8.11.0, NS apple-appkit-824.48)
> of 2010-09-30

I'm also using emacs24 but my git HEAD is at 55590b86 (May 18 2010)
because I find the X clipboard interaction to be broken under
ubuntu/gnome in recent emacs24 (haven't tested for a few weeks though).

Dan

>
>> I don't see tramp-sh in emacs23; I do see it in emacs24.
>
> Yup, I use emacs24 built from a checkout from the git mirror of the bzr
> sources.
>
>> How about we test this simple source block:
>>
>> #+begin_src R
>> system("hostname", intern=TRUE)
>> #+end_src
>>
>> in a file on the remote machine opened using C-x C-f with the
>> /address@hidden:path/to/file syntax, with this minimal config:
>>
>> (add-to-list 'load-path "/PATH/TO/ess/lisp")
>> (require 'ess-site)
>> (add-to-list 'load-path "/PATH/TO/org-mode/lisp")
>> (require 'org-install)
>> (setq debug-on-error t)
>> (org-babel-do-load-languages
>>  'org-babel-load-languages
>>  '((R . t)))
>>
>> I suspect part of the problem is to do with those (require tramp...)
>> lines in your config. Could you try without those lines, and without
>>
>> (add-to-list 'tramp-remote-path 'tramp-own-remote-path)
>
> I can try this for debugging, but I do need that configuration option
> eventually.  I build R and install R packages within my home directory
> on the server in question.  Without 'tramp-own-remote-path my R packages
> aren't found by the R process.  Also, this
>
>> Instead, could you try
>>
>> (setq org-babel-R-command "/PATH/TO/R/EXECUTABLE --slave --no-save")
>
> is not a portable solution, as on my local machine I use
> /usr/local/bin/R and on the remote machine I use ~/bin/R.  I suppose I
> could setup a conditional assignment based on host name, but since I
> already know 'tramp-own-remote-path works, that's my preferred option.
> Also, all of the results below are unchanged when I explicitly set the
> org-babel-R-command.
>
> Results:
>
> 1.  This file, opened locally:
> #+begin_src emacs-lisp
> (add-to-list 'load-path "/usr/local/share/emacs/site-lisp")
> (require 'ess-site)
> (add-to-list 'load-path (expand-file-name "~/src.remote/org-mode/lisp"))
> (require 'org-install)
> (setq debug-on-error t)
> (org-babel-do-load-languages
>  'org-babel-load-languages
>  '((R . t)))
> #+end_src
>
> #+results:
> | (R . t) |
>
> #+begin_src R :dir /bcs:~/
> system("hostname", intern=TRUE)
> #+end_src
>     
> Gives this backtrace:
> #v+
> Debugger entered--Lisp error: (void-function org-babel-temp-file)
>   org-babel-temp-file("R-")
>   org-babel-R-evaluate-external-process("system(\"hostname\", intern=TRUE)" 
> value nil nil)
>   org-babel-R-evaluate(nil "system(\"hostname\", intern=TRUE)" value nil nil)
>   org-babel-execute:R("system(\"hostname\", intern=TRUE)\n" ((:cache . "no") 
> (:comments . "") (:dir . "/bcs:~/") (:exports . "code") (:hlines . "no") 
> (:noweb . "no") (:results . "replace") (:session . "none") (:shebang . "") 
> (:tangle . "no")))
>   org-babel-execute-src-block(nil ("R" "system(\"hostname\", intern=TRUE)\n" 
> ((:cache . "no") (:comments . "") (:dir . "/bcs:~/") (:exports . "code") 
> (:hlines . "no") (:noweb . "no") (:results . "replace") (:session . "none") 
> (:shebang . "") (:tangle . "no")) "" nil nil 0))
>   org-babel-execute-src-block-maybe()
>   run-hook-with-args-until-success(org-babel-execute-src-block-maybe)
>   org-ctrl-c-ctrl-c(nil)
>   call-interactively(org-ctrl-c-ctrl-c nil nil)
> #v-
> This is also true if I explicitly set `org-babel-R-command' in the
> emacs-lisp source block, as well as if I explicitly (require 'ob).  This
> feels to me like it isn't even getting to the point where the TRAMP
> errors could arise.
>
>>> If I manually load ob.el by M-x load-file with Location as
>>> /path/to/ob.el then C-c C-c works. But (require 'ob) is not
>>> working. Am I missing something?
>
> With the above minimal config file edited locally with a remote
> execution directory, I get the same as Noruul.  The above error
> disappears if I (load-file "~/src.remote/org-mode/lisp/ob.el").  I also
> no longer get the TRAMP error I originally reported.
>
>> Noruul, what does (featurep 'ob) evaluate to before you manually attempt
>> the (require 'ob) ? And what goes wrong when you do the require?
>
> (featurep 'ob) reports `t' for me, before a (require 'ob).
> (fboundp #'org-babel-temp-file) and
> (fboundp #'with-parsed-tramp-file-name) both return nil.  After I
> explicitly load, org-babel-tempfile is defined (but this does NOT work
> with just (require 'ob), as reported by Noruul).  After I (require
> 'tramp), (fboundp #'with-parsed-tramp-file-name) is `t'.  After I
> (require 'tramp-sh), (boundp 'tramp-remote-path) is `t'.
>
> It looks to me like there are two issues affecting my original report.
>
> The first is that with (require 'org-install) and even with (require
> 'ob), certain babel-related functions aren't being exported (don't know
> if that's the right term).
>
> The second is that I _need_ to be able to add 'tramp-own-remote-path to
> 'tramp-remote-path, which at least on my system requires doing (require
> 'tramp) and (require 'tramp-sh).  That is, presumably, a TRAMP bug.
>
> Some interaction with explicitly requiring the tramp packages and then
> trying to use org-babel's remote functions seems to be messing things
> up.
>
>> If (part of) this is an issue of tramp differences between emacs23 and
>> 24 then the tramp author Michael Albinus has already been extremely
>> helpful on several occasions with tramp issues and I am confident he
>> will be able to advise.
>
> Thanks for all of your help already.  I'm sure we'll chase this down.
>
> /au