Re: official orgmode parser

[Tim Cross]

Przemysław Kamiński <pk@intrepidus.pl> writes:

> Hello,
>
> I oftentimes find myself needing to parse org files with some external 
> tools (to generate reports for customers or sum up clock times for given 
> month, etc). Looking through the list
>
> https://orgmode.org/worg/org-tools/
>
> and having tested some of these, I must say they are lacking. The 
> Haskell ones seem to be done best, but then the compile overhead of 
> Haskell and difficulty in embedding this into other languages is a drawback.
>
> I think it might benefit the community when such an official parser 
> would exist (and maybe could be hooked into org mode directly).
>
> I was thinking picking some scheme like chicken or guile, which could be 
> later easily embedded into C or whatever. Then use that parser in org 
> mode itself. This way some important part of org mode would be outside 
> of the small world of elisp.
>
> This is just an idea, what do you think? :)
>

The problem with this idea is maintenance. It is also partly why
external tools are not terribly reliable/good. Org mode is constantly
being enhanced and improved. It is very hard for external tools to keep
pace with org-mode development, so they soon get out of date or stop
working correctly. 

Org mode IS an elsip application. This is the main goal. The reason it
works so well is because elisp is largely a DSL that focuses on text
manipulation and is therefore ideally suited for a text based organiser. 

This means if you want to implement parsing of org files in any
other language, there is a lot of fundamental functionality which willl
need to be implemented that is not necessary when using elisp as it is
already built-in. Not only that, it is also 'battle hardened' and well
tested. The other problem would be in selecting another language which
behaves consistently across all the platforms Emacs and org-mode is
supported on. As org-mode is a stnadard part of Emacs, it also needs to
be implemented in something which is also available on all the platforms
emacs is on without needing the user to install additional software. 

The other issue is that you would need another skill in order to
maintain/extend org-mode. In addition to elisp, you will also need to
know whatever the parser implementation language is.

A third negative is that if the parser was in a different language to
elisp, the interface between the rest of org mode (in elisp) and the
parser would become an issue. At the moment, there are far fewer
barriers as it is all elisp. However, if part of the system is in
another language, you are now restricted to whatever defined interface
exists. This would likely also have performance issues and overheads
associated with translating from one format to another etc.

So, in short, the chances of org mode using a parser written in
something other than elisp is pretty close to 0. This leaves you with 2
options -

1. Implement another external tool which can parse org-files. As
metnioned above, this is a non-trivial task and will likely be difficult
to maintain. Probably not the best first choice.

2. Provide some details about your workflow where you believe you need
to use external tools to process the org-files. It is very likely there
are alternative approaches to give you the result you want, but without
the need to do external parsing of org-files. There isn't sufficient
details in the examples you mention to provide any specific details.
However, I have used org-mode for reporting, invoicing, time tracking,
documentation, issue/request tracking, project planning and project
management and never needed to parse my org files with an external tool.
I have exported the data in different formats which have then been
processed by other tools and I have tweaked my setup to support various
enterprise/corporate standards or requirements (logos, corporate
colours, report formats, etc). Sometimes these tweaks are trivial and
others require more extensive effort. Often, others have had to do
something the same or similar and have working examples etc.

So my recommendation is post some messages to this list with details on
what you need to try and do and see what others can suggest. I would
keep each post to a single item rather than one long post with multiple
requests. From watching this list, I've often see someone post a "How
can I ..." question only to get the answer "Oh, that is already
built-in, just do .....". Org is a large application with lots of
sophisticated power that isn't always obvious from just reading the
manual.