Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

[Ihor Radchenko]

Dear Fellow Orgers,

The recent spike of discussions following Karl's presentation in
Emacsconf 2021 revealed a lot of controversy among Org and Emacs
enthusiasts. Yet, Karl named a number of very real problems surrounding
Org mode usage outside Emacs.

>From the narrow perspective of this mailing list, I would like to list
some of the problems and possible solutions to them on our (Org dev)
side.

1. Org mode is almost impossible to separate from Emacs in its full strength
   - Yet, a number of people seems to be interested in using Org mode
     outside Emacs
     + Most notably, mobile users
     + A number of websites, like Github/Gitlab
   - The existing interest gave a rise to a number of third-party
     Org syntax parsers
     + None of the parsers support all the Org features, and even not
      all the grammar!
     + The parsers often do not even try to support all the features.
       They are merely looking at Org as a lightweight markup format.
       
2. Despite user interest, we lack a clear definition of Org grammar with
   examples and concrete guidelines for third-party parser developers

3. Many elements of the grammar are excessive for simple cases not
   involving document export, babel, and other powerful Org mode
   features

4. "Org mode" is an ambiguous word combination for search engines and
   people may not be able to find relevant information.
   - This one is not 100% true from my quick search. Try the following
     links:
     + https://duckduckgo.com/?q=org-mode&ia=web
     + https://duckduckgo.com/?q=org+mode&ia=web
     + https://duckduckgo.com/?q=org+mode+syntax&ia=web
     + https://duckduckgo.com/?q=org+mode+markup&ia=web
     The results are extremely relevant to Org, though orgmode.org
     search result looks slightly confusing (more below).
          
------------------------------------------------------
| My suggestions how we can address the above points |
------------------------------------------------------

1. Despite webengines delivering fairly good results for "Org mode"
   search term, I am a bit concerned about the first search hit, which
   is our flagship "https://orgmode.org"; website.

   The website title is "Org mode for Emacs", repelling users who _do
   not want_ to use Org inside Emacs. Maybe we can do better? Something
   with less accent on Emacs like "Org mode: your life in plain text"

   The "abstract" in the search result is also not fully relevant:
   > Org and Org-mode have so many use cases that it is simply not
   > possible to easily document them, let alone show them all off on a
   > single page. As a result, Worg serves as a community wiki and
   > provides a place to document and share information about all aspects
   > of using and working with Org. For example, Worg contains:
   Again, we can make a simple change revealing the paragraph shown the
   at our front page:
   > Org is a highly flexible structured plain text file format,
   > composed of a few simple, yet versatile, structures — constructed
   > to be both simple enough for the novice and powerful enough for the
   > expert. 
   > 
   > Org mode is also a GNU Emacs major mode for keeping notes,
   > authoring documents, computational notebooks, literate programming,
   > maintaining to-do lists, planning projects, and more — in a fast
   > and effective plain text system.

2. Our front pages gives an impression that user must install Org
   I refer to the big image links "Features Install Quickstart Contribute"

   Maybe we can add "Try in browser" linking to our own instance of
   https://organice.200ok.ch/sample

3. We can provide a "source of truth" for Org syntax for third-party
   parser developers. Something easily reachable from the front page:
   "Org-Mode Logo Org Mode
   Features
   Releases
   ...
   --> Add Org support in third-party apps"

   The page should give a nice summary of existing third-party
   libraries, official _technical_ Org syntax, and tools for developers.

   3.1. In particular, I suggest to link
        https://orgmode.org/worg/dev/org-syntax.html (it will be ready
        eventually)
   
   3.2. Also, we may add a simplified Org syntax, as Karl suggested
        (similar to Basic and Extended syntax in
        https://www.markdownguide.org/, but more technical)
   
   3.3. I strongly suggest to add a community test set with example Org
        files. The files should be a source of tests for Org parsers
        with the true parsed representations in sexp format (possibly
        also converted to json).

        The example files can live in a separate repo for easy
        contributions (possibly with Github/Gitlab mirrors is someone is
        willing to maintain those).

        The example files will be used by Org mode itself in our test
        suite and will serve as a benchmark for external parser quality.

   3.4. Finally, we can have a separate page listing recommended
        features for editors interacting with Org files. Something like
        "implementation roadmap" (citing Timothy) for external devs.
        Again, unlike our existing feature page, this should be more
        technical and target developers.

        The features may include (we can add them as needed):
        - Folding / structural editing
        - Table editing / alignment
        - Source block execution
        - Babel
        - Export / publish
        - Setting TODO keywords
        - Agenda / searching in Org files
        - Clocking data
        - Capture
        - ...

WDYT?

Best,
Ihor