[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: On elisp running native

From: Gong-Yi Liao 廖宮毅
Subject: Re: On elisp running native
Date: Thu, 5 Mar 2020 06:43:49 -0600

That's pretty good on (I assume) x86_64, but native-comp branch seems
not working on ARM64 Debian Buster at this moment:
Not sure if it's a GCCJIT issue or the native-comp branch issue:

Any comments on this compilation log are appreciated.


make -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
make[2]: Entering directory '/home/pi/Downloads/emacs/master/lisp'
 ELC+ELN   emacs-lisp/comp.elc
 ELC+ELN   emacs-lisp/bytecomp.elc
 ELC+ELN   emacs-lisp/autoload.elc
/bin/bash: line 1:   626 Segmentation fault      EMACSLOADPATH=
'../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval
'(setq load-prefer-newer t)' -l comp -f
batch-byte-native-compile-for-bootstrap emacs-lisp/autoload.el
make[2]: *** [Makefile:312: emacs-lisp/autoload.elc] Error 139
make[2]: *** Waiting for unfinished jobs....

Fatal error 11: Segmentation fault
/bin/bash: line 1:   625 Segmentation fault      EMACSLOADPATH=
'../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval
'(setq load-prefer-newer t)' -l comp -f
batch-byte-native-compile-for-bootstrap emacs-lisp/bytecomp.el
make[2]: *** [Makefile:312: emacs-lisp/bytecomp.elc] Error 139
/bin/bash: line 1:   624 Segmentation fault      EMACSLOADPATH=
'../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval
'(setq load-prefer-newer t)' -l comp -f
batch-byte-native-compile-for-bootstrap emacs-lisp/comp.el
make[2]: *** [Makefile:312: emacs-lisp/comp.elc] Error 139
make[2]: Leaving directory '/home/pi/Downloads/emacs/master/lisp'
make[1]: *** [Makefile:827: bootstrap-emacs.pdmp] Error 2
make[1]: Leaving directory '/home/pi/Downloads/emacs/master/src'
make: *** [Makefile:424: src] Error 2


On Thu, Mar 5, 2020 at 4:55 AM Adam Porter <address@hidden> wrote:
> Hi Andrea,
> Andrea Corallo <address@hidden> writes:
> > - Finally I've set up a docker image for people willing to test it
> >   without having to configure and compile Emacs and libgccjit.  Now that
> >   I've learned how it works I plan to use it to setup a small CI to keep
> >   an eye that the latest GCC does not break with our build.
> Thank you for setting up the Docker image.  I had tried to build the
> native-comp branch on my system but finally was unable to overcome a
> weird error related to libgccjit (probably because I need to upgrade my
> system).  I even tried using Guix to build it, but libgccjit isn't
> packaged in it (and I'm completely new to Guix packaging, so I wasn't
> able to figure out how to adjust the GCC build for that in a reasonable
> amount of time).
> I managed to get the Docker image working, and I did a little testing
> with one of my packages, org-ql, which is for searching Org files.  I
> installed it and its dependencies, then used native-comp-async to
> compile all of ~/.emacs.d/elpa.  Then I restarted Emacs to ensure that
> the native libraries were loaded, and finally I ran the following code
> and observed a more than 2x improvement!
> #+BEGIN_SRC elisp
>   ;; Reset the cache to force search to run again.
>   (setf org-ql-cache (make-hash-table :test #'equal))
>   ;; Set my personal groups (omitted) and enable org-super-agenda-mode
>   ;; to display in groups.
>   (setf org-super-agenda-groups my-groups)
>   (org-super-agenda-mode)
>   ;; Load all my Org files and search them.
>   (let* ((enable-local-variables nil)
>          (files (directory-files org-directory 'full (rx ".org" eos))))
>     (dolist (file files)
>       (find-file-noselect file 'nowarn))
>     (benchmark-run-compiled
>         (org-ql-search files
>           ;; The (tags) selector tends to be slow compared to other
>           ;; predicates because of searching the hierarchy repeatedly
>           ;; and tag inheritance, so it makes for an interesting test.
>           '(and (todo) (tags "Emacs"))
>           :super-groups org-super-agenda-groups)))
> | 5.819892666 | 1 | 0.6101534569999956 |  In Emacs 26.3
> | 2.317804428 | 3 | 0.6223487619999997 |  In Emacs 28 with native-comp
> Then I tested this more complex query and observed an even more
> impressive speedup of about 3.4x!
>   '(and (not (done))
>         (or (habit)
>             (deadline auto)
>             (scheduled :to today)
>             (ts-active :on today)))
> | 8.377843199 | 1 | 0.6277314330000081 |  In Emacs 26.3
> | 2.436048375 | 2 | 0.4278436059999997 |  In Emacs 28 with native-comp
> This is very exciting!  The difference between 8.4 seconds and 2.4
> seconds is a huge usability improvement.
> And everything seems to work fine!  No bugs, errors, or crashes of any
> kind.
> I did encounter a minor issue related to macro-expansion and loading
> that may be my fault.  When I tried to (require 'org-ql) or call a
> function that is autoloaded from it, I got an error saying that the .eln
> file did not provide the org-ql feature.  I surmised that meant that the
> file hadn't been compiled properly, so I looked at the async compilation
> log and saw an error saying that the variable org-ql-predicates was not
> defined.  At this URL you can see a macro definition,
> org-ql--def-plain-query-fn, and the call to it that's wrapped in
> cl-eval-when to ensure it works properly with the byte-compiler:
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L1671
> The macro uses the value of org-ql-predicates, which is defined here:
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L126
> But when the org-ql--defpred macro:
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L140
> ...is called, it adds to that variable.  So all of that works with the
> byte-compiler, but apparently not with the native compilation.
> To work around it, I wrapped (defvar org-ql-predicates...) in
> eval-and-compile, and then wrapped all of the calls to org-ql--defpred
> in eval-and-compile as well.  Then I deleted the org-ql.eln file and
> recompiled it, restarted Emacs, and then everything worked fine.
> I'm guessing that this issue is "native" to the native-compilation and
> might need to be documented in some way, but I wouldn't expect it to
> affect many packages, only ones that do tricky things with variables and
> macro-expansion like this.
> So, a few ideas for fixes or improvements:
> 1.  I guess the .eln file should not have been produced or left behind
>     in the directory, since there was a compilation error.
> 2.  The compilation error should probably be shown as a warning or
>     something, because it's easily lost in the async compilation log
>     buffer.
> 3.  It would probably be good if the loader fell back to .elc and .el
>     files if the .eln file doesn't seem to work, and showed a warning
>     when doing so.  The library would remain usable in its byte-compiled
>     form, and in the case of a package like this, the user could then
>     report the native-compilation error to the developer.
> Other than that minor issue, everything works and feels very fast!  This
> is great!
> >   ~eln~ files are compiled in specific sub-folders taking in account
> >   host architecture and current Emacs configuration to disambiguate
> >   possible incompatibilities.
> >  As example a file ~.../foo.el~ will compile into something like
> >  ~.../x86_64-pc-linux-gnu-12383a81d4f33a87/foo.eln~.
> FYI, I did not see the .eln files being put in subdirectories like that
> in the Docker image.  Maybe it didn't receive that update yet.
> This is all very exciting.  I'm imagining an update to package.el that
> would byte-compile packages immediately, as usual, then start
> native-compile-async'ing them in the background, loading each file's
> native version as it completes, with the packages being immediately
> usable and seeming to get faster as each file is compiled and loaded.
> > I'm generally very satisfied (surprised) about the stability of the
> > toy. I'm looking forward going into compile time mitigation.  I guess
> > I'll start this weekend with deferred compilation and GCC profiling.
> I think it isn't a toy anymore!  :)  Thanks very much for your work on
> this.  Exciting times for Emacs, as this feature, in one form or
> another, has been eagerly awaited for years.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]