Re: [PATCH] feat: add markdown-ts-mode

[Rahul Martim Juliato]

Philip Kaludercic <philipk@posteo.net> writes:

> Rahul Martim Juliato <rahuljuliato@gmail.com> writes:
>
>> Thanks Eli and Jostein!
>>
>> No problem criticising :)
>>
>>
>> Please find a new patch attached. Things I've made:
>>
>> - changed the file to the textmodes folder
>>
>> - made it inherit from `text-mode'
>>
>> - changed the news with +++
>>
>> - changed `emacs.texi' (is it here?) to add Markdown
>>
>> - scratched my head until 2:30AM thinking why it would not work for
>>   Jostein test.
>>
>>
>> Well Jostein, I've been played by the tree-sitter github repository
>> maintainer, lol.
>>
>> First, a little note. There's actually no major changes to the version
>> on github. I only changed documenting strings and made some small name
>> changes to more "alike" already existing modes.
>>
>> It happens that on tree-sitter repository, unlike any other of their
>> repositories (that I've seen), "main" branch is NOT the current branch,
>> but the "we moved from here bye" one.
>>
>> The current default branch is "split_parser" where
>> tree-sitter-markdown/src/ resides, hence the confusion.
>>
>> So, I wrote the mode with the "main" one in mind.
>>
>> I already checked the "new" (split_parser) one, and it seems it is
>> possible to convert the current work to use this one instead. I'll try
>> to do this in the middle of the week or sooner :)
>>
>> In the mean time, I realized more people would like to test this patch
>> and tree-sitter setup is really trick, so I made this little "guide" on
>> how to apply it until the end result (this might work now, pointing to
>> the "main" branch with the current patch).
>>
>> If you could please try the path (B) on the guide and tell me if this
>> works for you, It would be nice, Jostein.
>>
>>
>> Thanks!
>>
>> Rahul
>>
>>
>>
>> --- beggining of guide on how to apply this patch and test it
>>
>> Git apply this patch.
>>
>> Run the autogen script:
>>
>> ./autogen.sh
>>
>> Make sure configure uses tree-sitter:
>>
>> ./configure --with-tree-sitter
>>
>> Compile:
>>
>> make bootstrap
>>
>> Check the build:
>>
>> ./src/emacs -Q --version
>>
>> Open emacs
>>
>> ./src/emacs -Q --init-dir=~/tmp_emacs_dir/
>>
>>
>> A) The Hard Way
>>
>> Visit a .md file
>>
>> C-x f TEST.md
>>
>> Note the mode will not automatically load.
>>
>> This behavior is the same as typescript-ts-mode or other
>> treesitter modes I've been using.
>>
>> M-x markdown-ts-mode
>>
>> It will fail since you have no tree sitter grammar installed.
>> and will suggest installing it with `treesit-install-language-grammar'.
>>
>> This behaviour is also standard for the tree-sitter modes I
>> currently use (typescript, tsx, rust).
>>
>> Issuing "M-x treesit-install-language-grammar RET".
>>
>> It asks for language, complete with "markdown RET".
>>
>> It says there's no recipie for it, if you want to build it
>> interactivelly. Anwser "yes".
>>
>> It asks for the URL where the grammar is hosted, enter:
>> "https://github.com/tree-sitter-grammars/tree-sitter-markdown RET"
>>
>> It asks for the tag or branch, setting the default, enter "main RET".
>>
>> It asks for the subfolder, leave the default (src), enter "RET".
>>
>> It asks for the C compiler to use (default: auto-detect), just "RET".
>>
>> It asks for the C++ compiler to use (default: auto-detect), just "RET".
>>
>> Install to (default "~/tmp_emacs_dir/treesitter"), just "RET".
>>
>> It will clone the repository, compile the library and tell in the
>> minibuffer the library is installed to your folder.
>>
>>
>> B) The Easier Way.
>>
>> Copy-paste and eval this use-package definition.
>>
>> (use-package markdown-ts-mode
>>   :mode ("\\.md\\'" . markdown-ts-mode)
>>   :defer 't
>>   :config
>>   (add-to-list 'treesit-language-source-alist '(markdown 
>> "https://github.com/tree-sitter-grammars/tree-sitter-markdown"; "main" 
>> "src")))
>>
>>
>> Visit your markdown test file.
>>
>> It will probably fail due to the missing grammar.
>>
>> Issue "M-x treesit-install-language-grammar RET".
>>
>> Now with TAB it should complete "markdown", if not, type it.
>>
>> As we already have the source now defined, just hit "RET" to install
>> to the default treemacs folder.
>>
>> It will clone the repository and say it installed on your folder.
>>
>> Just reload the mode with "M-x markdown-ts-mode".
>>
>> From now on (with the use-package definition on `init.el') you should
>> just open .md and have the highlight and stuff.
>>
>> --- end of the guide
>>
>> From 45796df36129ec77c532b3fa21cdd0b8033c9777 Mon Sep 17 00:00:00 2001
>> From: Rahul Martim Juliato <rahul.juliato@gmail.com>
>> Date: Fri, 19 Apr 2024 23:21:20 -0300
>> Subject: [PATCH] feat: add markdown-ts-mode
>>
>>  * lisp/textmodes/markdown-ts-mode.el: New file.
>>  * doc/emacs/emacs.texi: Add Markdown to the manual
>>  * etc/NEWS: Announce markdown-ts-mode
>> ---
>>  doc/emacs/emacs.texi               |   1 +
>>  etc/NEWS                           |   5 ++
>>  lisp/textmodes/markdown-ts-mode.el | 106 +++++++++++++++++++++++++++++
>>  3 files changed, 112 insertions(+)
>>  create mode 100644 lisp/textmodes/markdown-ts-mode.el
>>
>> diff --git a/doc/emacs/emacs.texi b/doc/emacs/emacs.texi
>> index 7d77f13ab21..244c822ee04 100644
>> --- a/doc/emacs/emacs.texi
>> +++ b/doc/emacs/emacs.texi
>> @@ -618,6 +618,7 @@ Top
>>  * Enriched Text::       Editing text enriched with fonts, colors, etc.
>>  * Text Based Tables::   Commands for editing text-based tables.
>>  * Two-Column::          Splitting text columns into separate windows.
>> +* Markdown::            Major mode for editing Markdown files.
>>  
>>  Filling Text
>>  
>> diff --git a/etc/NEWS b/etc/NEWS
>> index 8ad1e78ca60..06fbaa03b55 100644
>> --- a/etc/NEWS
>> +++ b/etc/NEWS
>> @@ -1688,6 +1688,11 @@ A major mode based on the tree-sitter library for 
>> editing Elixir files.
>>  *** New major mode 'lua-ts-mode'.
>>  A major mode based on the tree-sitter library for editing Lua files.
>>  
>> ++++
>> +*** New major mode 'markdown-ts-mode'.
>> +A major mode based on the tree-sitter library for editing Markdown files.
>> +
>> +
>>  ** Minibuffer and Completions
>>  
>>  +++
>> diff --git a/lisp/textmodes/markdown-ts-mode.el 
>> b/lisp/textmodes/markdown-ts-mode.el
>> new file mode 100644
>> index 00000000000..063780a772f
>> --- /dev/null
>> +++ b/lisp/textmodes/markdown-ts-mode.el
>> @@ -0,0 +1,106 @@
>> +;;; markdown-ts-mode.el --- tree sitter support for Markdown  -*- 
>> lexical-binding: t; -*-
>> +
>> +;; Copyright (C) 2024 Free Software Foundation, Inc.
>> +
>> +;; Author     : Rahul Martim Juliato <rahul.juliato@gmail.com>
>> +;; Maintainer : Rahul Martim Juliato <rahul.juliato@gmail.com>
>> +;; Created    : April 2024
>> +;; Keywords   : markdown md languages tree-sitter
>> +
>> +;; This file is part of GNU Emacs.
>> +
>> +;; GNU Emacs is free software: you can redistribute it and/or modify
>> +;; it under the terms of the GNU General Public License as published by
>> +;; the Free Software Foundation, either version 3 of the License, or
>> +;; (at your option) any later version.
>> +
>> +;; GNU Emacs is distributed in the hope that it will be useful,
>> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +;; GNU General Public License for more details.
>> +
>> +;; You should have received a copy of the GNU General Public License
>> +;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
>> +
>> +;;; Commentary:
>> +;;
>
> You should probably briefly explain the package here.
>
>> +;;; Code:
>> +
>> +(require 'treesit)
>> +(require 'subr-x)
>> +
>> +(declare-function treesit-node-parent "treesit.c")
>> +(declare-function treesit-node-type "treesit.c")
>> +(declare-function treesit-parser-create "treesit.c")
>> +
>> +(defvar markdown-ts--treesit-settings
>> +  (treesit-font-lock-rules
>> +   :language 'markdown
>> +   :override t
>> +   :feature 'delimiter
>> +   '([ "[" "]" "(" ")" ] @shadow)
>> +
>> +   :language 'markdown
>> +   :feature 'paragraph
>> +   '([((atx_heading) @font-lock-keyword-face)
>> +      ((block_quote_marker) @font-lock-string-face)
>> +      ((code_span) @font-lock-string-face)
>> +      ((emphasis) @underline)
>> +      ((image_description) @link)
>> +      ((indented_code_block) @font-lock-string-face)
>> +      ((link_destination) @font-lock-string-face)
>> +      ((setext_heading) @font-lock-keyword-face)
>> +      ((strong_emphasis) @bold)
>> +      ((thematic_break) @shadow)
>> +      (block_quote (block_quote_marker) @font-lock-string-face)
>> +      (block_quote (paragraph) @font-lock-string-face)
>> +      (fenced_code_block (code_fence_content) @font-lock-string-face)
>> +      (fenced_code_block (fenced_code_block_delimiter) @font-lock-doc-face)
>> +      (inline_link (link_destination) @font-lock-string-face)
>> +      (inline_link (link_text) @link)
>> +      (list_item (list_marker_dot) @font-lock-keyword-face)
>> +      (list_item (list_marker_minus) @font-lock-keyword-face)
>> +      (list_item (list_marker_plus) @font-lock-keyword-face)
>> +      (list_item (list_marker_star) @font-lock-keyword-face)
>> +      (shortcut_link (link_text) @link)
>> +      ])))
>> +
>> +(defun markdown-ts-imenu-node-p (node)
>> +  "Check if NODE is a valid entry to imenu."
>> +  (equal (treesit-node-type (treesit-node-parent node))
>> +         "atx_heading"))
>> +
>> +(defun markdown-ts-imenu-name-function (node)
>> +  "Return an imenu entry if NODE is a valid header."
>> +  (let ((name (treesit-node-text node)))
>> +    (if (markdown-ts-imenu-node-p node)
>> +    (thread-first (treesit-node-parent node)(treesit-node-text))
>> +      name)))
>> +
>> +(defun markdown-ts-setup ()
>> +  "Setup treesit for `markdown-ts-mode'."
>> +  (setq-local treesit-font-lock-settings markdown-ts--treesit-settings)
>> +  (treesit-major-mode-setup))
>> +
>> +;;;###autoload
>> +(define-derived-mode markdown-ts-mode text-mode "Markdown"
>> +  "Major mode for editing Markdown using tree-sitter grammar."
>> +  (setq-local font-lock-defaults nil
>> +          treesit-font-lock-feature-list '((delimiter)
>> +                                           (paragraph)))
>> +
>> +  (setq-local treesit-simple-imenu-settings
>> +              `(("Headings" markdown-ts-imenu-node-p nil 
>> markdown-ts-imenu-name-function)))
>> +
>> +  (when (treesit-ready-p 'markdown)
>> +    (treesit-parser-create 'markdown)
>> +    (markdown-ts-setup)))
>> +
>> +(derived-mode-add-parents 'markdown-ts-mode '(markdown-mode))
>> +
>> +(if (treesit-ready-p 'markdown)
>> +    (add-to-list 'auto-mode-alist '("\\.md\\'" . markdown-ts-mode)))
>> +
>> +(provide 'markdown-ts-mode)
>> +;;; markdown-ts-mode.el ends here
>
> So if I understand correctly, there is no keymap or any convenience
> features that markdown-mode provides, because this is just the
> beginning, right?  A large part of major modes for me is not just the
> syntax highlighting and integration into Emacs systems, but also
> bindings that in the case of Markdown would insert links or add/remove
> emphasis.  I think it would be nice, if we could add these features in
> the future, and re-use bindings from a package like AucTeX to build on
> existing intuition (org-mode would be an alternative, but I am not a fan
> or their choice of bindings).  Some commands to "compile" and preview a
> document would also be nice.


Hello there again!

Please find attached the current version of this patch.

It now works with the "official" tree-sitter grammar for markdown:
https://github.com/tree-sitter-grammars/tree-sitter-markdown

Please note (as explained above) the `main branch' is not the default
anymore, but `split_parser' is.

They splitted the parser into two, one for the body of the document and
another to inline.

So, in order to test this patch, configure it with:

(use-package markdown-ts-mode
    :ensure nil
    :defer t  
    :mode ("\\.md\\'" . markdown-ts-mode)
    :config
    (add-to-list 'treesit-language-source-alist '(markdown 
"https://github.com/tree-sitter-grammars/tree-sitter-markdown"; "split_parser" 
"tree-sitter-markdown/src"))
    (add-to-list 'treesit-language-source-alist '(markdown-inline 
"https://github.com/tree-sitter-grammars/tree-sitter-markdown"; "split_parser" 
"tree-sitter-markdown-inline/src")))

If you visit a markdown file treesit issues an error, as always, asking
for the grammars.

Then, install BOTH grammars:

M-x treesit-install-language-grammar RET markdown RET
M-x treesit-install-language-grammar RET markdown-inline RET

Reload the `markdown-ts-mode'.

And everything should be working fine :)


@Philip, thanks for bringing that up! I completely agree with you;
aiming for a fully-featured mode is our long-term objective.

The idea here is to establish a foundation for supporting markdown files
within Emacs, without relying on external packages.

>From there, we can delve into discussions about editing and exporting
features. For instance, we could consider mimicking bindings and prompts
from org-mode, as well as exploring integrations if users opt to also
utilize third-party markdown-mode.

However, simply having the mode built-in reduces friction significantly
for new ideas and collaborations :)

0001-feat-add-markdown-ts-mode.patch
Description: Text Data