[bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.

[Andrew Tropin]

On 2023-02-09 18:04, Andrew Tropin wrote:

> On 2023-02-09 13:39, zimoun wrote:
>
>> Hi,
>>
>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>
>>> I applied tree-sitter and tree-sitter-cli patches,
>>
>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>> right?
>>
>>
>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>
>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>
>> Leaving out all the others, right?
>
> Merged first 5 patches from 01 to 05, also added one more commit, which
> addresses some things from reviews and one commit, which adds html
> grammar.
>
> The html grammar is added for the testing purposes.  It relies on
> generated parser.c and scanner.c and we will need to repackage it using
> grammar.js instead.  I'm not sure if a separate build system is needed
> for this, I guess we can just rewrite tree-sitter-grammar function,
> which generates packages as in example with tree-sitter-grammar-html:
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>
> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
> read it carefully, evaluate and either introduce new build system or
> just move all needed parts to tree-sitter-grammar function.  WDYT?
> After we done with it we can package all other grammars.

Ok, I realized that the proper build process for tree-sitter grammars is
a little harder than I expected, tree-sitter-build system make sense.  I
reviewed it, made a small change:

--8<---------------cut here---------------start------------->8---
@@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
 ;; Commentary:
 ;;
 ;; Build procedures for tree-sitter grammar packages.  This is the
-;; builder-side code, which builds on top fo the node build-system.
+;; builder-side code, which builds on top of the node build-system.
 ;;
 ;; Tree-sitter grammars are written in JavaScript and compiled to a native
 ;; shared object.  The `tree-sitter generate' command invokes `node' in order
@@ -114,7 +114,7 @@ (define (compile-language dir)
                    "-fno-exceptions"
                    "-O2"
                    "-g"
-                   "-o" ,(string-append lib "/" lang ".so")
+                   "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
                    ;; An additional `scanner.{c,cc}' file is sometimes
                    ;; provided.
                    ,@(cond
--8<---------------cut here---------------end--------------->8---

rewrote html grammar to use this build system and made it work with
built-in treesit package.  Also, tried examples of c and cpp grammars
from patches in this thread.

If you ok with it, I'll push the build system to master and update the
html grammar accordingly.

The final result will look like this:

--8<---------------cut here---------------start------------->8---
(define tree-sitter-delete-generated-files
  #~(begin
      (delete-file "binding.gyp")
      (delete-file-recursively "bindings")
      (delete-file "src/grammar.json")
      (delete-file "src/node-types.json")
      (delete-file "src/parser.c")
      (delete-file-recursively "src/tree_sitter")))

(define* (tree-sitter-grammar
          language language-for-synopsis version commit hash
          #:key
          (repository-url
           (format #f "https://github.com/tree-sitter/tree-sitter-~a"; language))
          (inputs '()))
  (let ((synopsis (string-append language-for-synopsis
                                 " grammar for tree-sitter"))
        (name (string-append "tree-sitter-grammar-" language)))
    (package
      (name name)
      (version version)
      (home-page repository-url)
      (source (origin
                (method git-fetch)
                (uri (git-reference
                      (url repository-url)
                      (commit commit)))
                (file-name (git-file-name name version))
                (sha256 (base32 hash))
                (modules '((guix build utils)))
                (snippet tree-sitter-delete-generated-files)))
      (build-system tree-sitter-build-system)
      (inputs inputs)
      (synopsis synopsis)
      (description (string-append synopsis "."))
      (license license:expat))))

(define-public tree-sitter-grammar-html
  (tree-sitter-grammar
   "html" "HTML"
   "0.19.0" "v0.19.0"
   "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
--8<---------------cut here---------------end--------------->8---

After that we can bring the rest of the grammars.

>
> According to emacs integration: I already have a working prototype on
> my local setup, which uses builtin treesit package, it's similiar to
> what Luis mentioned with search-paths.  We just need to figure out a
> few minor details and I'll add it.

I've sent a patch for emacs+tree-sitter integration in a different reply
in this thread.

Thank you very much for all the work, it's really valuable!

-- 
Best regards,
Andrew Tropin

signature.asc
Description: PGP signature