bug#40651: 26.3; eshell using (fn param) syntax breaks lookups

[Phil]

Start with clean emacs install.
Install company-mode using package installer.
Start eshell.
Enable company-mode.

Satisfy yourself that lookups work on paths, eg:
cd /home/
Will give a list of completions of directories under home.

Satisfy yourself that lookups work on regular eshell command syntax, eg:
find-
Will give a list of completions including find-file, etc.

Now try the same thing but use lisp syntax as supported by eshell, eg:
(find-

This fails with:
Completion function pcomplete-completions-at-point uses a deprecated
calling convention

Error was original reported to company maintains who have advised it's
an emacs bug not a company bug:
https://github.com/company-mode/company-mode/issues/974

I've debugged a bit further, it looks like:
completion--capf-wrapper(pcomplete-completions-at-point optimist)

Ultimately calls:
completion--capf-wrapper(lisp-completion-at-point all)

And ultimately that is returning to the pcomplete-completions-at-point a
result which fails parsing of result in the outer
completion--capf-wrapper.

More details from edebug backtrace below from inside the inner call of
completion--capf-wrapper.

The problem isn't just that the lisp bracket style of function calling
fails lookups.  It means that all further lookups using the bracket
style or even the eshell style of function call that had previously
worked now fail.

I presume this is because pcomplete completions of both styles are
disregarded by adding the function to completion--capf-misbehave-funs.

The only practical way to resolve is to restart emacs.

It would be great to get a workaround that allowed look-ups of both
styles to work.

If not possible a workaround that prevented the disabling of all
pcomplete completions would mean look-ups would still be partially
usable.

Thanks for any advice,
Phil.

Backtrace below:

  (consp (edebug-after 0 21 res))
  (and (edebug-after (edebug-before 20) 22 (consp (edebug-after 0 21 res))) 
(edebug-after (edebug-before 23) 27 (not (edebug-after (edebug-before 24) 26 
(functionp (edebug-after 0 25 res))))))
  (cond ((edebug-after (edebug-before 19) 28 (and (edebug-after (edebug-before 
20) 22 (consp (edebug-after 0 21 res))) (edebug-after (edebug-before 23) 27 
(not (edebug-after (edebug-before 24) 26 (functionp ...)))))) (edebug-after 
(edebug-before 29) 38 (if (edebug-after (edebug-before 30) 33 (member 
(edebug-after 0 31 fun) (edebug-after 0 32 completion--capf-safe-funs))) nil 
(edebug-after (edebug-before 34) 37 (let* ((v ...)) (setq 
completion--capf-safe-funs (cons v ...)))))) (edebug-after (edebug-before 39) 
68 (and (edebug-after (edebug-before 40) 46 (eq (quote no) (edebug-after 
(edebug-before 41) 45 (plist-get ... :exclusive)))) (edebug-after 
(edebug-before 47) 65 (null (edebug-after (edebug-before 48) 64 (try-completion 
... ... ...)))) (edebug-after (edebug-before 66) 67 (setq res nil))))) 
((edebug-after (edebug-before 69) 78 (not (edebug-after (edebug-before 70) 77 
(or (edebug-after (edebug-before 71) 73 (listp ...)) (edebug-after 
(edebug-before 74) 76 (functionp ...)))))) (edebug-after (edebug-before 79) 91 
(if (edebug-after (edebug-before 80) 83 (member (edebug-after 0 81 fun) 
(edebug-after 0 82 completion--capf-misbehave-funs))) nil (edebug-after 
(edebug-before 84) 86 (message "Completion function %S uses a deprecated 
calling convention" (edebug-after 0 85 fun))) (edebug-after (edebug-before 87) 
90 (let* ((v ...)) (setq completion--capf-misbehave-funs (cons v ...))))))))
  (let ((res (edebug-after (edebug-before 15) 17 (funcall (edebug-after 0 16 
fun))))) (edebug-after (edebug-before 18) 92 (cond ((edebug-after 
(edebug-before 19) 28 (and (edebug-after (edebug-before 20) 22 (consp ...)) 
(edebug-after (edebug-before 23) 27 (not ...)))) (edebug-after (edebug-before 
29) 38 (if (edebug-after (edebug-before 30) 33 (member ... ...)) nil 
(edebug-after (edebug-before 34) 37 (let* ... ...)))) (edebug-after 
(edebug-before 39) 68 (and (edebug-after (edebug-before 40) 46 (eq ... ...)) 
(edebug-after (edebug-before 47) 65 (null ...)) (edebug-after (edebug-before 
66) 67 (setq res nil))))) ((edebug-after (edebug-before 69) 78 (not 
(edebug-after (edebug-before 70) 77 (or ... ...)))) (edebug-after 
(edebug-before 79) 91 (if (edebug-after (edebug-before 80) 83 (member ... ...)) 
nil (edebug-after (edebug-before 84) 86 (message "Completion function %S uses a 
deprecated calling convention" ...)) (edebug-after (edebug-before 87) 90 (let* 
... ...))))))) (edebug-after (edebug-before 93) 99 (if (edebug-after 0 94 res) 
(edebug-after (edebug-before 95) 98 (cons (edebug-after 0 96 fun) (edebug-after 
0 97 res))))))
  (if (edebug-after (edebug-before 1) 13 (let* ((val (edebug-after 0 2 which))) 
(cond ((eq val (quote all)) t) ((eq val (quote safe)) (edebug-after 
(edebug-before 3) 6 (member (edebug-after 0 4 fun) (edebug-after 0 5 
completion--capf-safe-funs)))) ((eq val (quote optimist)) (edebug-after 
(edebug-before 7) 12 (not (edebug-after ... 11 ...)))) (t nil)))) (edebug-after 
(edebug-before 14) 100 (let ((res (edebug-after (edebug-before 15) 17 (funcall 
(edebug-after 0 16 fun))))) (edebug-after (edebug-before 18) 92 (cond 
((edebug-after (edebug-before 19) 28 (and ... ...)) (edebug-after 
(edebug-before 29) 38 (if ... nil ...)) (edebug-after (edebug-before 39) 68 
(and ... ... ...))) ((edebug-after (edebug-before 69) 78 (not ...)) 
(edebug-after (edebug-before 79) 91 (if ... nil ... ...))))) (edebug-after 
(edebug-before 93) 99 (if (edebug-after 0 94 res) (edebug-after (edebug-before 
95) 98 (cons (edebug-after 0 96 fun) (edebug-after 0 97 res))))))))

  completion--capf-wrapper(lisp-completion-at-point all)

  run-hook-wrapped(completion--capf-wrapper lisp-completion-at-point all)
  completion-at-point()

  eshell-complete-lisp-symbol()
  eshell-complete-parse-arguments()
  pcomplete-parse-arguments(nil)
  pcomplete-completions()
  pcomplete-completions-at-point()
  funcall(pcomplete-completions-at-point)
  (let ((res (edebug-after (edebug-before 15) 17 (funcall (edebug-after 0 16 
fun))))) (edebug-after (edebug-before 18) 92 (cond ((edebug-after 
(edebug-before 19) 28 (and (edebug-after (edebug-before 20) 22 (consp ...)) 
(edebug-after (edebug-before 23) 27 (not ...)))) (edebug-after (edebug-before 
29) 38 (if (edebug-after (edebug-before 30) 33 (member ... ...)) nil 
(edebug-after (edebug-before 34) 37 (let* ... ...)))) (edebug-after 
(edebug-before 39) 68 (and (edebug-after (edebug-before 40) 46 (eq ... ...)) 
(edebug-after (edebug-before 47) 65 (null ...)) (edebug-after (edebug-before 
66) 67 (setq res nil))))) ((edebug-after (edebug-before 69) 78 (not 
(edebug-after (edebug-before 70) 77 (or ... ...)))) (edebug-after 
(edebug-before 79) 91 (if (edebug-after (edebug-before 80) 83 (member ... ...)) 
nil (edebug-after (edebug-before 84) 86 (message "Completion function %S uses a 
deprecated calling convention" ...)) (edebug-after (edebug-before 87) 90 (let* 
... ...))))))) (edebug-after (edebug-before 93) 99 (if (edebug-after 0 94 res) 
(edebug-after (edebug-before 95) 98 (cons (edebug-after 0 96 fun) (edebug-after 
0 97 res))))))
  (if (edebug-after (edebug-before 1) 13 (let* ((val (edebug-after 0 2 which))) 
(cond ((eq val (quote all)) t) ((eq val (quote safe)) (edebug-after 
(edebug-before 3) 6 (member (edebug-after 0 4 fun) (edebug-after 0 5 
completion--capf-safe-funs)))) ((eq val (quote optimist)) (edebug-after 
(edebug-before 7) 12 (not (edebug-after ... 11 ...)))) (t nil)))) (edebug-after 
(edebug-before 14) 100 (let ((res (edebug-after (edebug-before 15) 17 (funcall 
(edebug-after 0 16 fun))))) (edebug-after (edebug-before 18) 92 (cond 
((edebug-after (edebug-before 19) 28 (and ... ...)) (edebug-after 
(edebug-before 29) 38 (if ... nil ...)) (edebug-after (edebug-before 39) 68 
(and ... ... ...))) ((edebug-after (edebug-before 69) 78 (not ...)) 
(edebug-after (edebug-before 79) 91 (if ... nil ... ...))))) (edebug-after 
(edebug-before 93) 99 (if (edebug-after 0 94 res) (edebug-after (edebug-before 
95) 98 (cons (edebug-after 0 96 fun) (edebug-after 0 97 res))))))))

  completion--capf-wrapper(pcomplete-completions-at-point optimist)


  run-hook-wrapped(completion--capf-wrapper pcomplete-completions-at-point 
optimist)
  company--capf-data-real()
  company--capf-data()




In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo 
version 1.15.10)
 of 2020-03-15 built on foo-XPS-13-9360
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description:     Ubuntu 18.04.4 LTS

Recent messages:
Result: nil

Result: nil

Result: t

Result: pcomplete-completions-at-point
No apropos matches for ‘eshell-complete-lisp-symbol()’
Type "q" to restore previous buffer.
uncompressing em-cmpl.el.gz...done

Configured using:
 'configure --with-imagemagick --with-xwidgets --with-cairo
 --with-x-toolkit=gtk3'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS XWIDGETS LCMS2

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  flycheck-mode: t
  which-function-mode: t
  show-paren-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  flyspell-mode: t
  which-key-mode: t
  flx-ido-mode: t
  workgroups-mode: t
  ido-everywhere: t
  projectile-mode: t
  me-minor-mode: t
  xterm-mouse-mode: t
  winner-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-meta hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-meta
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-vars hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-vars
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-compose hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-compose
/home/foo/.emacs.d/el-get/mu4e/mu4e/org-mu4e hides 
/usr/local/share/emacs/site-lisp/mu4e/org-mu4e
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-main hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-main
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-proc hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-proc
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-view hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-view
/home/foo/.emacs.d/el-get/mu4e/mu4e/org-old-mu4e hides 
/usr/local/share/emacs/site-lisp/mu4e/org-old-mu4e
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-contrib hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-contrib
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-lists hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-lists
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-mark hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-mark
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-context hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-context
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-speedbar hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-speedbar
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-utils hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-utils
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-draft hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-draft
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-headers hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-headers
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-message hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-message
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e-actions hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e-actions
/home/foo/.emacs.d/el-get/mu4e/mu4e/mu4e hides 
/usr/local/share/emacs/site-lisp/mu4e/mu4e

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils apropos misearch
multi-isearch eieio-opt speedbar sb-image ezimage dframe help-fns
cl-print edebug company-oddmuse company-keywords company-etags etags
company-gtags company-dabbrev-code company-dabbrev company-files
company-capf company-cmake company-xcode company-clang company-semantic
company-eclim company-template company-bbdb company-anaconda
anaconda-mode url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap xref project pythonic
python tramp-sh tramp tramp-compat tramp-loaddefs trampver ucs-normalize
shell parse-time company add-log pcase flycheck json map rx which-func
imenu paren virtualenvwrapper gud yasnippet-snippets yasnippet
flyspell-correct-ido flyspell-correct flyspell ispell org-bullets
key-chord which-key smex flx-ido flx windmove em-unix em-term term
disp-table ehelp em-script em-prompt em-ls em-hist em-pred em-glob
em-dirs em-cmpl em-basic em-banner em-alias esh-var esh-io esh-cmd
esh-opt esh-ext esh-proc esh-arg esh-groups eshell esh-module esh-util
esh-mode elec-pair jka-compr workgroups2 anaphora f dash s edmacro
exwm-systemtray xcb-systemtray xcb-xembed exwm-config ido exwm
exwm-input xcb-keysyms xcb-xkb exwm-manage exwm-floating xcb-cursor
xcb-render exwm-layout exwm-workspace exwm-core xcb-ewmh xcb-icccm xcb
xcb-xproto xcb-types xcb-debug kmacro server zenburn-theme projectile
grep compile ibuf-ext ibuffer ibuffer-loaddefs thingatpt xt-mouse winner
elhome initsplit cus-edit cus-start cus-load wid-edit byte-code-cache
el-get-lock el-get-bundle el-get el-get-autoloading el-get-list-packages
el-get-dependencies el-get-build el-get-status pp el-get-methods
el-get-fossil el-get-svn el-get-pacman el-get-github-zip
el-get-github-tar el-get-http-zip el-get-http-tar el-get-hg el-get-go
el-get-git-svn el-get-fink el-get-emacswiki el-get-http el-get-notify
el-get-emacsmirror el-get-github el-get-git el-get-elpa package
epg-config url-handlers url-parse auth-source eieio eieio-core cl-macs
eieio-loaddefs password-cache url-vars seq byte-opt el-get-darcs
el-get-cvs el-get-bzr el-get-brew el-get-builtin el-get-apt-get
el-get-recipes el-get-byte-compile subr-x el-get-custom cl-extra
help-mode el-get-core autoload radix-tree lisp-mnt bytecomp byte-compile
cconv cl gv dired dired-loaddefs org-element cl-seq avl-tree generator
org advice org-macro org-footnote org-pcomplete pcomplete org-list
org-faces org-entities noutline outline easy-mmode org-version
ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp
ob-comint comint ansi-color ring ob-core ob-eval org-compat org-macs
org-loaddefs format-spec find-func cal-menu easymenu calendar
cal-loaddefs cl-loaddefs cl-lib time-date mule-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting xwidget-internal cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 666176 37101)
 (symbols 48 50095 1)
 (miscs 40 714 716)
 (strings 32 144654 7742)
 (string-bytes 1 4282162)
 (vectors 16 109108)
 (vector-slots 8 2414630 107394)
 (floats 8 234 606)
 (intervals 56 825 149)
 (buffers 992 21))