bug#33536: Package state inconsistent on package install after abnormal Emacs startup [was Re: bug#33536: 26.1.90; package wants to remove bbdb]

[N. Jackson]

retitle 33536 Package state inconsistent on package install after abnormal 
Emacs startup
quit

Hello Glenn,

At 16:06 -0500 on Wednesday 2018-12-05, Glenn Morris wrote:
>
> Can you reproduce this from a clean state, eg
> rm -rf /tmp/foo
> mkdir /tmp/foo
> HOME=/tmp/foo emacs

Yes, I was able to reproduce it from a clean slate, after some
fiddling which shed some more light on the matter.

I observe the problem in Emacs 26.1, 26.1.90 (tarball), and master
of a few days ago (e02d375cb6670e2306b9c67d7f6fd2dd1d1b2711).

The problem is not specific to bbdb.

The invalid state that Package gets into is that the problematic
package is missing from `package-selected-packages', it is listed by
Package as a "dependency" instead of "installed", and Package will
offer to remove it after the next package operation.

One way to get into that invalid state is to install a package after
Emacs encounters an init file error at startup. (See tediously
detailed recipe below.)

This is arguably a user error: An error in one's init files should
be resolved before using Emacs. Note however, that no message is
written to *Messages* [1] and in my normal configuration (with
multiple frames being restored by Desktop), the warning window is in
a buried frame that I never see, so it's quite easy to install a
package without realising that there was an init file error.

In any case, user error or not, I think that Package should avoid
putting itself into an inconsistent state.


DETAILED RECIPE:

0.  Create a clean home:
$ rm -rf /tmp/foo; mkdir /tmp/foo

1.  Install a package (chess, for example) and exit Emacs:
$ HOME=/tmp/foo src/emacs &
M-x list-packages RET
M-<                             ; Navigate to top of list.
C-s chess C-s RET               ; Navigate to the `chess' package.
i                               ; Mark it for install.
x                               ; Install it.
=> Packages to install: 1 (chess-2.0.4).  Proceed? (y or n)
y
Operation [ Install 1 ] finished
C-x C-c                         ; Exit Emacs.

2.  Place some invalid Elisp in init file:
$ HOME=/tmp/foo src/emacs &
C-x C-f /tmp/foo/.emacs RET     ; Open init file.
M->                             ; Navigate to end of buffer.
This is invalid Elisp.          ; Type some invalid Elisp.
C-x C-s                         ; Save init file.
C-x C-c                         ; Exit Emacs.

3.  Fail to notice the warning at start up. Install another package (e.g. 
csv-mode)
$ HOME=/tmp/foo src/emacs &
;; A warning of an initialization error is displayed in a window
;; but we fail to notice it.
M-x list-packages RET
M-<                             ; Navigate to top of list.
C-s csv-mode C-s RET            ; Navigate to the `csv-mode' package.
i                               ; Mark it for install.
x                               ; Install it.
=> Packages to install: 1 (csv-mode-1.7).  Proceed? (y or n)
y
=> Operation [ Install 1 ] finished
C-x C-c                         ; Exit Emacs.

4.  Fix the invalid Elisp in init file.
$ HOME=/tmp/foo src/emacs &
;; A warning of the initialization error is displayed in a window.
C-x C-f /tmp/foo/.emacs RET     ; Open init file.
M->                             ; Navigate to end of buffer.
C-S-backspace                   ; Kill the line of invalid Elisp.
C-x C-s                         ; Save init file.
C-x C-c                         ; Exit Emacs.

5.  Observe that Package is in an inconsistent state.
$ HOME=/tmp/foo src/emacs &
;; Now Package is in an inconsistent state:
;; 1. csv-mode is missing from 'package-selected-packages'.
;; 2. csv-mode is listed as a "dependency" instead of "installed".
;; 3. Package will offer to remove csv-mode after the next package
;; operation.


[1] I filed a bug about lack of feedback in *Messages* about init
file errors: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33654.