Re: Some thoughts about Emacs performance

[Robert Boyer]

Dear Simon,

You note: 

On the other hand when sorting it
again, i.e. when the vector is already fully sorter, Emacs is quite a
bit faster than SBCL—maybe Emacs chose to optimize for partly-sorted
vectors at the expense of a bit of performance for random input.

You are so right.  'msa' kills 'stable sort' on a few examples, namely an array

that is all 0's and an array that is 1, 2, 3, ...

Below  is some info on sorting arrays that are 21 million long.

But I should mention that I regard what my 'msa' does to catch the easy cases may be nothing in comparison to what sorting pros can do.

Best,

Bob

Running (test-msa-0 21,000,000).
Timing (MSA TEST-MSA-AR1 '< :KEY 'CAR).
WARNING: redefining COMMON-LISP-USER::KEY in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-EQUAL in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-PREDICATE in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-1 in DEFUN
Evaluation took:
  3.334 seconds of real time
  3.267129 seconds of total run time (2.323391 user, 0.943738 system)
  [ Run times consist of 0.154 seconds GC time, and 3.114 seconds non-GC time. ]
  97.99% CPU
  31 forms interpreted
  115 lambdas converted
  5,990,145,243 processor cycles
  174,860,640 bytes consed
 

Timing (STABLE-SORT TEST-MSA-AR2 '< :KEY 'CAR).
Evaluation took:
  12.729 seconds of real time
  12.562949 seconds of total run time (11.935063 user, 0.627886 system)
  [ Run times consist of 0.232 seconds GC time, and 12.331 seconds non-GC time. ]
  98.70% CPU
  22,870,125,432 processor cycles
  168,006,352 bytes consed
 

Running (test-msa-ascending 21,000,000).
Timing (MSA TEST-MSA-AR1 '< :KEY 'CAR).
WARNING: redefining COMMON-LISP-USER::KEY in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-EQUAL in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-PREDICATE in DEFMACRO
WARNING: redefining COMMON-LISP-USER::MSA-1 in DEFUN
Evaluation took:
  2.525 seconds of real time
  2.514507 seconds of total run time (2.471494 user, 0.043013 system)
  99.60% CPU
  31 forms interpreted
  115 lambdas converted
  4,536,551,428 processor cycles
  6,823,488 bytes consed
 

Timing (STABLE-SORT TEST-MSA-AR2 '< :KEY 'CAR).
Evaluation took:
  12.914 seconds of real time
  12.823063 seconds of total run time (12.407732 user, 0.415331 system)
  [ Run times consist of 0.227 seconds GC time, and 12.597 seconds non-GC time. ]
  99.30% CPU
  23,203,467,086 processor cycles
  168,006,128 bytes consed
  

On Fri, Feb 16, 2024 at 8:08 AM Simon Leinen <simon.leinen@gmail.com> wrote:

Bob,

welcome to the emacs-devel list, and thanks a lot for *your* wonderful
contributions (theorem prover, string search, and Lisp benchmarking -
I remember boyer.lisp from RPG's reference work[1]).

On Thu, Feb 8, 2024 at 8:15 AM Robert Boyer
<robertstephenboyer@gmail.com> wrote:
>
> Emacs 27.1 has a 'sort' function that takes longer than stable-sort of SBCL. Maybe
> by a factor of 2. See also my attached file 'ms.lisp'.
>
> There may be a lot that can be improved in Emacs'
> handling of cl-loop, setf, elt, or cl-random.

In this case, cl-random seems to be the main culprit for the slow
initialization—replacing that with plain "random" speeds it up by
about a factor of ten.  There was some discussion on the list recently
about cl-random vs. random. The main functional difference is that
cl-random supports a defined state. But the performance difference may
be due more to the fact that random is written in C, and cl-random in
Lisp.

As for the sorting itself, both Emacs and SBCL seem to use mergesort
in their implementations of (stable-)sort.  Emacs's implementation is
written in C, SBCL's in Lisp. Performance is quite similar—on my
system (Apple Macbook Air M2) Emacs takes about 35% longer to sort a
million random numbers than SBCL.  (On the other hand when sorting it
again, i.e. when the vector is already fully sorter, Emacs is quite a
bit faster than SBCL—maybe Emacs chose to optimize for partly-sorted
vectors at the expense of a bit of performance for random input.)

In general, the Emacs Lisp runtime system and compiler(s) aren't as
optimized as SBCL for general Lisp use.  But it gets quite close!

On the other hand, Emacs has editor-specific code (e.g. redisplay) and
data structures (e.g. buffers) which are highly optimized and partly
written in C.  But it doesn't try to be a standalone platform for
performance-oriented Lisp developers.  Of course Emacs is very
suitable as a Software Development Environment for systems such as
SBCL, and there are many good integration options—personally I use the
SLIME package these days.

Best regards, and enjoy Lisping in Emacs!
--
Simon.

> ;; First some Emacs, with times on my $100 Chromebook.
>
> (setq n 6)
> (defun make-random-array (n)
>   (let ((a (make-vector n 0)))
>     (cl-loop for i below n do
>              (setf (elt a i) (cl-random 1000000)))
>     a))
> (byte-compile 'make-random-array)
> (benchmark '(setq foo (make-random-array (expt 10 n))) 1) -- 2.3 seconds
> (benchmark '(sort foo '<) 1) -- 1 second
>
> ;; Second some Common Lisp, with times for SBCL on my $100 Chromebook.
>
> (defparameter n 6)
> (defun make-random-array (n)
>   (declare (fixnum n))
>   (let ((a (make-array n)))
>     (declare (type array a))
>     (loop for i fixnum below n do
>           (setf (aref a i) (random most-positive-fixnum)))
>     a))
> (time (defparameter foo (make-random-array (expt 10 n))))  -- .041 seconds
> (time (progn (stable-sort foo '<) nil)) -- .45 seconds
>
> Thanks so much for Emacs, which is so great that I cannot put it
> into words.
>
> Bob


[1] https://dreamsongs.com/Files/Timrep.pdf