[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Orgmode] [PATCH] sha1 hash of latex fragments to avoid regeneration
From: |
Eric Schulte |
Subject: |
Re: [Orgmode] [PATCH] sha1 hash of latex fragments to avoid regeneration |
Date: |
Sat, 05 Dec 2009 09:35:35 -0700 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/23.1.50 (gnu/linux) |
Hi Francesco,
You raise good points below, and I am not sure how best to respond to
them. My initial reaction is that you should not be checking
automatically generated files (e.g. the results of ditaa exports) into a
version control repository, however I understand that there are times
when such measures are required.
Would it be possible to switch from using org-exp-blocks to using
org-babel? If so then you could use org-babel's caching which does
*not* affect the exported file name, but rather saves a sha1 key as
(mostly) hidden text in the org-mode buffer.
so for example
#+BEGIN_ditaa procedure.png
+-----------------+
| Example |
| |
| +---------+ |
| | | |
| | | |
| +---------+ |
+-----------------+
#+END_ditaa
would be written as
#+begin_src ditaa :file procedure.png :cache :exports results
+-----------------+
| Example |
| |
| +---------+ |
| | | |
| | | |
| +---------+ |
+-----------------+
#+end_src
and pressing C-cC-c on the block (or exporting) would result in the
insertion of a link to the resulting image into the org-mode buffer
behind a results line as follows -- only in org most of the hash is
hidden.
#+results[bdffac60833c9f925a52bd6617dace39832b7dda]:
[[file:procedure.png]]
My problem with your proposed solution is that it entails storing
information *outside* of the org-mode buffer -- which granted we are
already doing although in a less dramatic way. The creation of external
files whose sole purpose is to save a hash in their filename strikes me
as wasteful/messy.
Best -- Eric
Francesco Pizzolante <address@hidden> writes:
> Hi,
>
>> I just pulled, reloaded, and re-ran my simple tests and the patch
>> appears to have been applied successfully.
>
> This idea of caching images is really great and works well.
>
> Nevertheless, I have a remark about the way it is implemented.
>
> When I write the following code:
>
> #+BEGIN_ditaa procedure.png
>
> I expect that:
>
> - the system generates a `procedure.png' file in order for me to check it in
> the repository;
>
> - that the `procedure.png' file is referenced everywhere (i.e. in the exported
> HTML code or LaTeX code).
>
> The current way it works is:
>
> 1. The `procedure_XXX.png' is committed to the repository;
>
> 2. I alter my ditaa code, and export the new document;
>
> 3. The `procedure_XXX.png' file has been removed from my working copy and a
> new file `procedure_YYY.png' has been added in my working copy.
>
> Thus, in order to commit the new files as they are, I need to:
>
> 1. revert the `procedure_XXX.png' file, in order to delete it afterward;
>
> 2. add the `procedure_YYY.png' file to the repository.
>
> A nice solution would have been that all ditaa exports are done under the
> filename `procedure.png', and that empty files such as
> `procedure.png.ZZZ.cache' are created to "store" the fact that they have
> already been generated.
>
> That way, we even can ignore all the `*.cache' files from our working copy,
> when checking for updates.
>
> What do you think?
>
> Francesco
>
>>
>> Thanks -- Eric
>>
>> Carsten Dominik <address@hidden> writes:
>>
>>> Hi Eric,
>>>
>>> I had a problem while pushing, please verify that the patch got in
>>> correctly. Thanks!
>>>
>>> - Carsten
>>>
>>> On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote:
>>>
>>>> Carsten Dominik <address@hidden> writes:
>>>>
>>>>> Wow, this is fantastic!
>>>>>
>>>>> Do you think it is ready to be included (because you say first
>>>>> pass...)
>>>>>
>>>>
>>>> Yes,
>>>>
>>>> I said first pass because I had only done minimal testing. However
>>>> all
>>>> indications are that it works, and there are no further changes I
>>>> would
>>>> like to make, so if it looks good to you I would say "yes", please
>>>> apply
>>>> it.
>>>>
>>>> Thanks -- Eric
>>>>
>>>>>
>>>>> - Carsten
>>>>>
>>>>> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote:
>>>>>
>>>>>> "Eric Schulte" <address@hidden> writes:
>>>>>>
>>>>>>> Hi Carsten,
>>>>>>>
>>>>>>> Thanks for the feedback, I have comments inline below
>>>>>>>
>>>>>>> Carsten Dominik <address@hidden> writes:
>>>>>>
>>>>>> [...]
>>>>>>
>>>>>>>> Now, I am sure that you are already planning to do the same
>>>>>>>> for ditaa images etc?
>>>>>>>
>>>>>>> of course :)
>>>>>>
>>>>>> A first pass at a patch implementing caching of ditaa and dot images
>>>>>> generated by org-exp-blocks is attached. It seems to work in all
>>>>>> initial tests, and it will only remove files which match the
>>>>>> following
>>>>>> pattern
>>>>>>
>>>>>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension"
>>>>>>
>>>>>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40
>>>>>> characters.
>>>>>>
>>>>>> Best -- Eric
>>>>>>
>>>>>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00
>>>>>> 2001
>>>>>> From: Eric Schulte <address@hidden>
>>>>>> Date: Mon, 16 Nov 2009 19:33:11 -0700
>>>>>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images
>>>>>> when exporting through org-exp-blocks
>>>>>>
>>>>>> ---
>>>>>> lisp/ChangeLog | 6 ++++
>>>>>> lisp/org-exp-blocks.el | 60 +++++++++++++++++++++++++++++++++++++
>>>>>> +---------
>>>>>> 2 files changed, 54 insertions(+), 12 deletions(-)
>>>>>>
>>>>>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog
>>>>>> index 5f83aaa..c2d44fa 100755
>>>>>> --- a/lisp/ChangeLog
>>>>>> +++ b/lisp/ChangeLog
>>>>>> @@ -1,3 +1,9 @@
>>>>>> +2009-11-17 Eric Schulte <address@hidden>
>>>>>> +
>>>>>> + * org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1
>>>>>> + hash keys to cache and re-use images generated by the
>>>>>> + org-exp-blocks interface to ditaa and dot.
>>>>>> +
>>>>>> 2009-11-16 Carsten Dominik <address@hidden>
>>>>>>
>>>>>> * org-html.el (org-export-html-home/up-format): Add an ID to the
>>>>>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el
>>>>>> index 71e1608..72fe6c4 100644
>>>>>> --- a/lisp/org-exp-blocks.el
>>>>>> +++ b/lisp/org-exp-blocks.el
>>>>>> @@ -217,9 +217,15 @@ Specify the path at which the image should be
>>>>>> saved as the first
>>>>>> element of headers, any additional elements of headers will be
>>>>>> passed to the ditaa utility as command line arguments."
>>>>>> (message "ditaa-formatting...")
>>>>>> - (let ((out-file (if headers (car headers)))
>>>>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) "
>>>>>> ")))
>>>>>> - (data-file (make-temp-file "org-ditaa")))
>>>>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>>>>> " ")))
>>>>>> + (data-file (make-temp-file "org-ditaa"))
>>>>>> + (hash (sha1 (prin1-to-string (list body args))))
>>>>>> + (raw-out-file (if headers (car headers)))
>>>>>> + (out-file-parts (if (string-match
>>>>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>>>>> $" raw-out-file)
>>>>>> + (cons (match-string 1 raw-out-file)
>>>>>> + (match-string 2 raw-out-file))
>>>>>> + (cons raw-out-file "png")))
>>>>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>>>>> out-file-parts))))
>>>>>> (unless (file-exists-p org-ditaa-jar-path)
>>>>>> (error (format "Could not find ditaa.jar at %s" org-ditaa-jar-
>>>>>> path)))
>>>>>> (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body)
>>>>>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line
>>>>>> arguments."
>>>>>> "\n")))
>>>>>> (cond
>>>>>> ((or htmlp latexp docbookp)
>>>>>> - (with-temp-file data-file (insert body))
>>>>>> - (message (concat "java -jar " org-ditaa-jar-path " " args " "
>>>>>> data-file " " out-file))
>>>>>> - (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>>>>> args " " data-file " " out-file))
>>>>>> + (unless (file-exists-p out-file)
>>>>>> + (mapc ;; remove old hashed versions of this file
>>>>>> + (lambda (file)
>>>>>> + (when (and (string-match (concat (regexp-quote (car out-
>>>>>> file-parts))
>>>>>> + "_\\([[:alnum:]]+\\)\
>>>>>> \."
>>>>>> + (regexp-quote (cdr out-
>>>>>> file-parts)))
>>>>>> + file)
>>>>>> + (= (length (match-string 1 out-file)) 40))
>>>>>> + (delete-file (expand-file-name file
>>>>>> + (file-name-directory
>>>>>> out-file)))))
>>>>>> + (directory-files (or (file-name-directory out-file)
>>>>>> + default-directory)))
>>>>>> + (with-temp-file data-file (insert body))
>>>>>> + (message (concat "java -jar " org-ditaa-jar-path " " args "
>>>>>> " data-file " " out-file))
>>>>>> + (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>>>>> args " " data-file " " out-file)))
>>>>>> (format "\n[[file:%s]]\n" out-file))
>>>>>> (t (concat
>>>>>> "\n#+BEGIN_EXAMPLE\n"
>>>>>> @@ -259,14 +277,32 @@ digraph data_relationships {
>>>>>> }
>>>>>> #+end_dot"
>>>>>> (message "dot-formatting...")
>>>>>> - (let ((out-file (if headers (car headers)))
>>>>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) "
>>>>>> ")))
>>>>>> - (data-file (make-temp-file "org-ditaa")))
>>>>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>>>>> " ")))
>>>>>> + (data-file (make-temp-file "org-ditaa"))
>>>>>> + (hash (sha1 (prin1-to-string (list body args))))
>>>>>> + (raw-out-file (if headers (car headers)))
>>>>>> + (out-file-parts (if (string-match
>>>>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>>>>> $" raw-out-file)
>>>>>> + (cons (match-string 1 raw-out-file)
>>>>>> + (match-string 2 raw-out-file))
>>>>>> + (cons raw-out-file "png")))
>>>>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>>>>> out-file-parts))))
>>>>>> (cond
>>>>>> ((or htmlp latexp docbookp)
>>>>>> - (with-temp-file data-file (insert body))
>>>>>> - (message (concat "dot " data-file " " args " -o " out-file))
>>>>>> - (shell-command (concat "dot " data-file " " args " -o " out-
>>>>>> file))
>>>>>> + (unless (file-exists-p out-file)
>>>>>> + (mapc ;; remove old hashed versions of this file
>>>>>> + (lambda (file)
>>>>>> + (when (and (string-match (concat (regexp-quote (car out-
>>>>>> file-parts))
>>>>>> + "_\\([[:alnum:]]+\\)\
>>>>>> \."
>>>>>> + (regexp-quote (cdr out-
>>>>>> file-parts)))
>>>>>> + file)
>>>>>> + (= (length (match-string 1 out-file)) 40))
>>>>>> + (delete-file (expand-file-name file
>>>>>> + (file-name-directory
>>>>>> out-file)))))
>>>>>> + (directory-files (or (file-name-directory out-file)
>>>>>> + default-directory)))
>>>>>> + (with-temp-file data-file (insert body))
>>>>>> + (message (concat "dot " data-file " " args " -o " out-
>>>>>> file))
>>>>>> + (shell-command (concat "dot " data-file " " args " -o "
>>>>>> out-
>>>>>> file)))
>>>>>> (format "\n[[file:%s]]\n" out-file))
>>>>>> (t (concat
>>>>>> "\n#+BEGIN_EXAMPLE\n"
>>>>>> --
>>>>>> 1.6.4.73.gc144
>>>>>>
>>>>>
>>>>> - Carsten
>>>
>>> - Carsten
>>
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Remember: use `Reply All' to send replies to the list.
>> address@hidden
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode