Emacs 21.1: jit-lock hangs emacs in certain circumstances

[Alan Mackenzie]

In GNU Emacs 21.1.4 (i686-pc-linux-gnulibc1, X toolkit, Xaw3d scroll bars)
 of 2001-11-03 on acm
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: en_GB
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  locale-coding-system: iso-latin-1
  default-enable-multibyte-characters: t

My machine:  Cyrix 6x86 166 Mhz with 64 Mb ram running GNU/linux.

Start Emacs running, get a cup of coffee:  Come back some while later to
find that Emacs has hung:  :-(.

Extensive investigation reveals the following:

*************************************************************************

Have "top" running, displaying the CPU usage of all running processes.
Start emacs with desktop enabled - desktop loads (too) many buffers.

font-lock-support-mode is jit-lock-mode.  (Though its cousin lazy-lock
fares somewhat better, it sometimes hangs too.  Even when it doesn't, it
spends a _long_ time (~10 minutes?) consuming 15%-50% of the CPU time,
even though apparently idle.  With fast-lock or no support mode, the
problem doesn't happen.)

Just after emacs starts, jit-lock-mode starts fontifying all the buffers
simultaneously.  Emacs's CPU load goes immediately to around 15% and
never drops back to zero (unless a key is pressed).

Emacs's CPU load, as reported by "top" swings wildly around, and several
minutes later hits 100% (less top's usage), at which point Emacs is hung.
At this point, the only key sequence I could get Emacs to respond to was
C-g C-g.  Emacs's store occupancy doesn't mushroom, though it _may_ be
creeping up _very_ gradually.  (8 hours later, having slept and woken up
to find it wasn't just a bad nightmare, Emacs was still hung, consuming
100% CPU).

How do I know it's jit-lock that's doing it?  (i) A keypress at any time
before Emacs hangs reduces Emacs CPU usage back to zero.  (ii) Setting
global-font-lock-mode to nil completely eliminated Emacs's CPU usage
whilst idle.  (iii) Setting jit-lock-stealth-time to 15 makes Emacs go to
zero CPU usage for these 15 seconds.

Even though I stumbled over this problem using a development version of
cc-mode, the bug still shows itself using cc-mode-5.28 and the awk-mode
which were released with Emacs 21.1. 

This phenomenon does not trouble XEmacs.

*************************************************************************

Here is a typical sequence of Emacs's CPU usage in percent, as reported
by "top", noted every 15 seconds (, a key having been pressed at -15
seconds):
             +0 seconds   +15 seconds  +30 seconds  +45 seconds 
0 minutes          0         16.3         22.9         17.6
1 minute        17.6         28.6         21.8         16.6
2 minutes       18.6         21.2         15.7         13.1
3 minutes       22.0         24.7         18.4         20.6

4 minutes       41.0         17.8         17.2         21.4
5 minutes       32.4         17.4         17.2         21.2
6 minutes       29.2         18.2         14.5         98.4
7 minutes       98.6         and stays at this level forever.

Here is a copy of my buffer list as it was for this run:
.   mkcfg.c         21246  C
  % jit-lock.el     17856  Emacs-Lisp
  % *info*          46823  Info
  % font-lock.el   127034  Emacs-Lisp
    cc-awk.el       47855  Emacs-Lisp 
    cc-mode.README  67379  Fundamental
  % .emacs           7432  Emacs-Lisp
 *  mkcfg.awk       21003  AWK
 *  input.awk        2019  AWK
 *  test.awk         1251  AWK
 *  spellcheck.awk   9136  AWK
    cc-mode.el      33855  Emacs-Lisp
    cc-cmds.el<2>  118827  Emacs-Lisp
    cc-defs.el<2>   47464  Emacs-Lisp
    cc-awk.200902.el  37900 Emacs-Lisp
    cc-defs.el      13553  Emacs-Lisp
    cc-styles.el    21848  Emacs-Lisp
    cc-engine.el<2>  192966 Emacs-Lisp
  % 181002.diff     20932  Diff
    cc-cmds.el     116877  Emacs-Lisp
    cc-engine.el   145927  Emacs-Lisp
 *  bracesemi.awk     191  AWK
  % syntax.c        91468  C
  % lisp.el         14608  Emacs-Lisp
    test.c           1147  C
    cc-langs.el     29989  Emacs-Lisp
    cc-align.el     31238  Emacs-Lisp
 *  statement-15.awk  1296 AWK
    texi.README       166  Fundamental
  % elp.el          24765  Emacs-Lisp
    bracesemi.dat      60  Fundamental
    edebug.el<2>   154276  Emacs-Lisp
    cc-vars.el      55196  Emacs-Lisp
    statement-15.res  2611 Fundamental
    statement-15.c   1080  C
  % zwang-awk-mode.el  15665 Emacs-Lisp
    cc-backup          93  Shell-script
    cc-menus.el     15049  Emacs-Lisp
    *scratch*         191  Lisp Interaction
 *  *Messages*       1338  Fundamental

Conjecture:  I think that jit-lock is using an idle-timer for _each_
buffer, about 37 in all, and that these timers are somehow thrashing
eachother to bits.

PS:  a cc of followups would be most appreciated.

-- 
Alan Mackenzie (Munich, Germany)