Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 06, 2024 at 16:46:52 -0800, Stefan Kangas wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > On Wed, Nov 06, 2024 at 21:48:05 +0100, Joost Kremers wrote:
> >> (info "(elisp) Lexical Binding")

> >> ,----
> >> | Lexical binding was introduced to Emacs, as an optional feature, in
> >> | version 24.1.  We expect its importance to increase with time.  Lexical
> >> | binding opens up many more opportunities for optimization, so programs
> >> | using it are likely to run faster in future Emacs versions.

> > This was several major Emacs versions ago.  Has anybody actually done
> > any measurements to demonstrate how effective this optimisation has
> > been?  When I tried comparing dynamic vs. lexical .elc versions of CC
> > Mode scrolling through xdisp.c, dynamic took 8.3247s, lexical took
> > 8.3285s.  That's the same, within measurement accuracy.

> Let me first point out that, at least to my mind, the most important
> benefit of lexbind is that it's easier to use and reason about: the
> byte-compiler can catch more mistakes, and it eliminates an entire class
> of bugs (with dynbind, even presumed "local" variables can be
> manipulated by any function you call).

Lexical binding makes debugging more difficult.  For example, the
compilation process discards the names of parameters to functions.

> Consider a simple example like this:

>     (defun another-function ()
>       (setq x 3))

>     (defun foo ()
>       (let ((x 1) (y 2))
>         (another-function)
>         (+ x y)))

> This code is clearly bad, but regardless of that we get with dynbind,

>     (foo) => 5

> whereas with lexbind,

>     (foo) => 3

Yes.  Both results are clearly correct.

> Now, it's also true that theory tells us that lexbind should indeed
> often lead to better performance, since the compiler can do
> optimizations on such code that are hard or impractical with dynbind.

But the overhead of having to construct closures for internal functions
tends to counter other optimisations.

> For example, if you compile and then disassemble the above function
> `foo` using something like

>     (byte-compile #'foo)
>     (disassemble #'foo (current-buffer))

> you will see that, with dynbind, this comes out to 11 byte-code
> instructions, many of which are about checking for new values of `x` or
> `y`, basically in case they have gotten new values after calling
> `another-function`.  With lexbind, the byte-compiler can constant fold
> `x` and `y`, and it comes out to 4 instructions instead.

As a counter, try compiling the following under both styles of bindings
and printing out the disassemblies (M-: (hack-local-variables) is your
friend when changing the lexical-binding setting):

;; -*- lexical-binding: nil -*-
(defvar external-function nil)

(defun foo (a)
  (funcall external-function
           (lambda (b)
             (+ a b))))

Under dynamic binding this is 8 byte-code instructions.  With lexical
binding, it's 11, including closure creation instructions.  Should foo
get called in a tight loop, these instructions will be slow.  There are
no such problems with the dynamic version.

> Such optimizations will of course not matter for all Emacs Lisp code.
> I'm also not sure that we have exhausted all optimization opportunities
> that lexbind opens up in the current byte-compiler, but Mattias
> Engdegård would know more about this.  (BTW, if that's true, it should
> perhaps not come as a surprise, given that it started out as a compiler
> for dynbind.)

> The optimizations will also not improve performance in every single use
> case.  The scrolling benchmark you did with CC Mode might be an example
> of this.  I would guess that things like displaying lots of images,
> which mostly takes place in C, are also not much faster.

Most of the time in that scrolling is taken by font locking, which is
primarily compiled Lisp.

-- 
Alan Mackenzie (Nuremberg, Germany).