Surrounding Lexical Variable Reference in the Body of defun

[Deokhwan Kim]

Hi there,

I'm having trouble understanding lexical binding in Emacs 24. I came across the 
following sentence in the Emacs Lisp manual 
<http://www.gnu.org/software/emacs/manual/html_node/elisp/Lexical-Binding.html>:

> the code in the body of a defun or defmacro cannot refer to surrounding 
> lexical variables.

It was a great shock to me because it sounded quite awkward and Common Lisp 
does not have such restriction AFAIK. Rather, I suspected that I might 
misunderstand what the sentence really meant. So I decided to make some 
experiments with the following code stored in foo.el:

  ;;; -*- lexical-binding: t -*-
  (let ((x 0))
    (defun counter ()
      (setq x (1+ x))))

  (message "%d" (counter))
  (message "%d" (counter))

Surprisingly, when I ran it in the form of source code, it worked:

  $ emacs -Q -batch -l foo.el
  1
  2

On the other hand, when I tried to byte-compile it, I got the following warning 
messages:

  $ emacs -Q -batch -f batch-byte-compile foo.el
  In toplevel form:
  foo.el:2:1:Warning: Function counter will ignore its context (x)
  foo.el:2:1:Warning: Unused lexical variable `x'
  foo.el:4:11:Warning: reference to free variable `x'
  foo.el:4:17:Warning: assignment to free variable `x'

  In end of data:
  foo.el:8:1:Warning: the function `counter' is not known to be defined.
  Wrote foo.elc

When I ran the resulting byte-compiled code, I got an error as the manual 
claims:

  $ emacs -Q -batch -l foo.elc
  Symbol's value as variable is void: x

Now I'm so confused. Here are my two questions:

  1. Why does this restriction exists? Is it inevitable because of some design 
decision of Emacs? Or is it temporary and removed in a (near) future release?
  2. Why does the original source code behave differently from its compiled 
code?

Best regards,
Deokhwan Kim