bug#61303: 29.0.60; rust-ts-mode does not consistently fontify nested macro-invocations

[Jostein Kjønigsen]

Hey there.

Thanks for another tree-sitter based mode to hack at :D

I think I've found an error/lacking in rust-ts-mode where we may have to 
move upstream to fix it.

Consider the following contrived (and yes, not very idiomatic) rust-code:

let result = format!("{}{}",
    "Well yes",
    format!("Or {} no?", "possibly"));

With rust-ts-mode first format! invocation will get highlighted as a 
macro-invocation, while the second will not.

Inspecting this using treesit-explore-mode, I get the following tree:

(let_declaration let pattern: (identifier) =
  value:
   (macro_invocation macro: (identifier) !
    (token_tree (
     (string_literal " ")
     (string_literal " ")
     (identifier)
     (token_tree (
      (string_literal " ")
      (string_literal " ")
      ))
     )))
  ;)

Notice the lack of a second/nested "macro_invocation"-node. Instead the 
second macro is just denoted as a "identifier".

To fix this, we'll probably need to upstream some bug-reports and PRs?

--
Jostein

In GNU Emacs 29.0.60 (build 7, x86_64-pc-linux-gnu, GTK+ Version
 3.24.36, cairo version 1.17.6) of 2023-02-05 built on thinkpad-t14s
Repository revision: 088425538f2122d88a4f4e132dbb2f1139648531
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12201007
System Description: Arch Linux

Configured using:
 'configure --with-json --with-tree-sitter'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Rust

Minor modes in effect:
  treesit-explore-mode: t
  electric-pair-mode: t
  highlight-symbol-mode: t
  flycheck-mode: t
  editorconfig-mode: t
  company-mode: t
  which-function-mode: t
  helm-mode: t
  helm-minibuffer-history-mode: t
  shell-dirtrack-mode: t
  helm--remap-mouse-mode: t
  async-bytecomp-package-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-nlinum-mode: t
  nlinum-mode: t
  ido-yes-or-no-mode: t
  override-global-mode: t
  server-mode: t
  global-hl-line-mode: t
  pixel-scroll-precision-mode: t
  doom-modeline-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  hs-minor-mode: t

Load-path shadows:
/home/jostein/.emacs.d/elpa/transient-20230107.1528/transient hides 
/home/jostein/build/emacs/lisp/transient

Features:
(shadow sort mail-extr emacsbug message yank-media rfc822 mml mml-sec
epa derived epg rfc6068 epg-config mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums bug-reference face-remap cus-start rust-ts-mode
c-ts-common treesit helm-command helm-elisp helm-eval edebug helm-info
vc-hg vc-git diff-mode vc-bzr vc-dispatcher disp-table cargo
cargo-process markdown-mode color elec-pair rust-utils rust-mode
rust-rustfmt rust-playpen rust-compile rust-cargo vc-svn dired-aux
flyspell ispell help-fns radix-tree cl-print tramp-sh image-file
image-converter winner helm-external helm-net tramp-archive tramp-gvfs
tramp-cache time-stamp zeroconf dbus ffap add-log ido-completing-read+
memoize minibuf-eldef elisp-slime-nav paredit highlight-symbol flycheck
editorconfig editorconfig-core editorconfig-core-handle
editorconfig-fnmatch company-oddmuse company-keywords company-etags
etags fileloop generator company-gtags company-dabbrev-code
company-dabbrev company-files company-clang company-capf company-cmake
company-semantic company-template company-bbdb company eglot
external-completion array jsonrpc ert ewoc debug backtrace flymake-proc
flymake warnings which-func hideshow eww url-queue thingatpt shr
pixel-fill kinsoku url-file svg xml dom puny mm-url gnus nnheader
gnus-util mail-utils range mm-util mail-prsvr helm-imenu helm-mode
helm-misc helm-files image-dired image-dired-tags image-dired-external
image-dired-util xdg image-mode dired dired-loaddefs exif tramp
tramp-loaddefs trampver tramp-integration cus-edit pp cus-load wid-edit
files-x tramp-compat shell parse-time iso8601 ls-lisp helm-buffers
helm-occur helm-tags helm-locate helm-grep helm-regexp helm-utils
helm-help helm-types helm helm-global-bindings helm-easymenu helm-core
async-bytecomp helm-source helm-multi-match helm-lib async pcase imenu
ob-plantuml org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete pcomplete org-list org-footnote
org-faces org-entities time-date noutline outline icons ob-emacs-lisp
ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys
oc org-loaddefs find-func cal-menu calendar cal-loaddefs org-version
org-compat org-macs format-spec delsel autorevert filenotify yasnippet
nlinum linum ido-yes-or-no advice ido edmacro kmacro
use-package-bind-key bind-key easy-mmode xref project server hl-line
pixel-scroll cua-base compile-eslint compile text-property-search comint
ansi-osc ansi-color ring doom-modeline doom-modeline-segments
doom-modeline-env doom-modeline-core all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons shrink-path rx f f-shortdoc s dash compat
dracula-theme cl-extra help-mode use-package-ensure use-package-core
finder-inf yasnippet-autoloads ido-yes-or-no-autoloads
elisp-slime-nav-autoloads cmake-mode-autoloads flycheck-autoloads
pkg-info-autoloads magit-autoloads all-the-icons-autoloads
crontab-mode-autoloads powershell-autoloads doom-modeline-autoloads
undo-tree-autoloads rust-mode-autoloads magit-section-autoloads
paredit-autoloads dracula-theme-autoloads cargo-autoloads
yaml-mode-autoloads helm-autoloads popup-autoloads queue-autoloads
nlinum-autoloads bmx-mode-autoloads company-autoloads
git-commit-autoloads multiple-cursors-autoloads dap-mode-autoloads
lsp-treemacs-autoloads treemacs-autoloads cfrs-autoloads
posframe-autoloads hydra-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads bui-autoloads transient-autoloads
ido-completing-read+-autoloads memoize-autoloads with-editor-autoloads
compat-autoloads epl-autoloads lsp-docker-autoloads yaml-autoloads
highlight-symbol-autoloads expand-region-autoloads lsp-mode-autoloads
lv-autoloads markdown-mode-autoloads spinner-autoloads ht-autoloads
shrink-path-autoloads f-autoloads dash-autoloads s-autoloads info
editorconfig-autoloads helm-core-autoloads async-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache
json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs
cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 595433 43945)
 (symbols 48 41127 1)
 (strings 32 186487 7325)
 (string-bytes 1 5326600)
 (vectors 16 70866)
 (vector-slots 8 906939 66598)
 (floats 8 868 339)
 (intervals 56 7370 541)
 (buffers 984 35))