bug#53755: 27.2; C-u M-x gnus-summary-sort-by-most-recent-number / -date not ordering threads as expected

[Emilia Blåsten]

Dear All,

I would like to report a bug in thread sorting in the Gnus summary
buffer. The issue arises in the following two cases (at least):

* C-u M-x gnus-summary-sort-by-most-recent-number in the summary buffer

* (setq gnus-thread-sort-functions '(not
  gnus-thread-sort-by-most-recent-number)) in .gnus.el

It also appears for similar functions ending in "-date".

Background: When calling the above without C-u or without the (not ...),
things work well, with most recent emails displayed on top of the
summary buffer and threads taking the slot corresponding to their most
recent article. For example:

Using gnus-summary-sort-by-most-recent-number:
E  2022-02-02 21:04  Your upcoming appointment is st  XX removed sender XX
O  2022-02-02 20:13  Re: Postdoc                      XX removed sender XX
RA 2022-01-06 12:17   ╰►                              XX removed sender XX
O  2022-01-04 18:33    ╰►                             XX removed sender XX
OA 2022-02-02 19:43  Re: \tilde e_M                   XX removed sender XX
O  2022-02-02 19:39   ╰►                              XX removed sender XX
OA 2022-01-31 21:15    ╰► Fwd: \tilde e_M             XX removed sender XX
E  2022-02-02 17:03  Blok arvio asunnosta             XX removed sender XX
E  2022-02-02 16:47  Pahoittelemme!                   XX removed sender XX
E  2022-02-02 16:27  Call for Paper: [Axioms] (ISSN:  XX removed sender XX
...

The issue: The documentation of gnus-thread-sorting-functions would
imply that having a (not ...) or doing C-u ... would reverse the
above. That is not what happens. Instead I get the following (pay
attention to the location of the thread "Re: Postdoc"):

Using C-u gnus-summary-sort-by-most-recent-number:
...
O  2022-01-03 18:57  [trans-info-europe] Trans & out  XX removed sender XX
O  2022-01-03 18:57  [trans-info-europe] Trans & out  XX removed sender XX
O  2022-01-04 18:33  Re: Postdoc                      XX removed sender XX
RA 2022-01-06 12:17   ╰►                              XX removed sender XX
O  2022-02-02 20:13    ╰►                             XX removed sender XX
O  2022-01-05 10:19  Page2                            XX removed sender XX
O  2022-01-05 10:19  Page3                            XX removed sender XX
...

The thread is sorted according to its oldest message on 2022-01-04
instead of its newest 2022-02-02. Furthermore the order of the messages
inside the thread became reversed. Based on the documentation I would
expect something like this instead, where most recent messages and most
recently active threads are to the bottom of the buffer:

Expected sorting with C-u gnus-summary-sort-by-most-recent-number:
...
E  2022-02-02 16:27  Call for Paper: [Axioms] (ISSN:  XX removed sender XX
E  2022-02-02 16:47  Pahoittelemme!                   XX removed sender XX
E  2022-02-02 17:03  Blok arvio asunnosta             XX removed sender XX
OA 2022-02-02 19:43  Re: \tilde e_M                   XX removed sender XX
O  2022-02-02 19:39   ╰►                              XX removed sender XX
OA 2022-01-31 21:15    ╰► Fwd: \tilde e_M             XX removed sender XX
O  2022-02-02 20:13  Re: Postdoc                      XX removed sender XX
RA 2022-01-06 12:17   ╰►                              XX removed sender XX
O  2022-01-04 18:33    ╰►                             XX removed sender XX
E  2022-02-02 21:04  Your upcoming appointment is st  XX removed sender XX


In summary, there are two thing happening:

1) Threads seem to become sorted in reverse by their oldest message
instead of in reverse by their newest message.

2) The order of the messages inside threads changes without the
documentation mentioning it.

I suspect that both of these issues follow from the fact that the
function gnus-thread-latest-date (and likely gnus-thread-highest-number)
are used for a dual purpose according to the following from gnus-sum.el:

; Since this is called not only to sort the top-level threads, but
; also in recursive sorts to order the articles within a thread, each
; article will be processed many times.  Thus it speeds things up
; quite a bit to use gnus-date-get-time, which caches the time value.
(defun gnus-thread-latest-date (thread)

However I am not too sure of this, and I have not yet managed to
investigate the whole thread sorting logic.

Kindest regards,
Emilia

PS. My summary view is configured as follows:
;; Interface stuff
(setq gnus-summary-line-format "%U%R %&user-date;  %B%-31,31s  %f\n"
      gnus-user-date-format-alist '((t . "%Y-%m-%d %H:%M"))
      gnus-summary-thread-gathering-function 'gnus-gather-threads-by-references
      gnus-thread-sort-functions 'gnus-thread-sort-by-most-recent-number
      gnus-article-sort-functions 'gnus-thread-sort-by-number
      gnus-sum-thread-tree-false-root ""
      gnus-sum-thread-tree-indent " "
      gnus-sum-thread-tree-leaf-with-other "├► "
      gnus-sum-thread-tree-root ""
      gnus-sum-thread-tree-single-leaf "╰► "
      gnus-sum-thread-tree-vertical "│")



---- M-x report-emacs-bug generated: ----

In GNU Emacs 27.2 (build 1, aarch64-unknown-linux-android)
 of 2022-01-09 built on localhost
Recent messages:
Reading active file via nndraft...done
Checking new news...done
nnimap read 0k from XXXremoved email serverXXX
No more unseen articles
Auto-saving...
previous-line: Beginning of buffer [2 times]
Making completion list... [2 times]
completing-read-default: Command attempted to use minibuffer while in minibuffer
Quit
Mark set

Configured using:
 'configure --disable-dependency-tracking
 --prefix=/data/data/com.termux/files/usr
 --libdir=/data/data/com.termux/files/usr/lib
 --sbindir=/data/data/com.termux/files/usr/bin --disable-rpath
 --disable-rpath-hack --host=aarch64-linux-android --disable-autodepend
 --with-gif=no --with-gnutls --with-jpeg=no --without-gconf
 --without-gsettings --without-lcms2 --without-x --with-png=no
 --with-tiff=no --with-xml2 --with-xpm=no --without-dbus
 --without-selinux --with-modules --with-pdumper=yes --with-dumping=none
 --with-json emacs_cv_sanitize_address=yes emacs_cv_prog_cc_no_pie=no
 ac_cv_lib_elf_elf_begin=no gl_cv_func_dup2_works=no
 ac_cv_func_setrlimit=no --disable-nls --enable-shared --enable-static
 --libexecdir=/data/data/com.termux/files/usr/libexec 'CFLAGS=
 -fstack-protector-strong -Oz' 'CPPFLAGS=
 -I/data/data/com.termux/files/usr/include'
 'LDFLAGS=-L/data/data/com.termux/files/usr/lib
 -Wl,-rpath=/data/data/com.termux/files/usr/lib -fopenmp -static-openmp
 -Wl,--enable-new-dtags -Wl,--as-needed -Wl,-z,relro,-z,now''

Configured features:
NOTIFY INOTIFY GNUTLS LIBXML2 ZLIB MODULES THREADS JSON PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Summary

Minor modes in effect:
  gnus-agent-summary-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  nyan-mode: t
  global-auto-revert-mode: t
  xterm-mouse-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug cl-print rect misearch multi-isearch eieio-opt speedbar
sb-image ezimage dframe deuglify gnus-cus gnus-demon gnus-diary nndiary
gnus-draft gnus-dup gnus-fun gnus-html gnus-kill gnus-logic gnus-mh
mh-comp mh-scan mh-gnus mh-e mh-compat mh-buffers mh-loaddefs
gnus-registry registry eieio-base gnus-salt gnus-topic gnus-uu yenc
gnus-vm sendmail magit-bookmark bookmark dired-aux help-fns radix-tree
url-cache winner tramp-archive tramp-gvfs dbus flow-fill shr-color color
qp mule-util sort gnus-cite mm-archive mail-extr gnus-async gnus-bcklg
gnus-ml disp-table nndraft nnmh nnfolder utf-7 epa-file gnutls
network-stream gnus-agent gnus-srvr gnus-score score-mode nnvirtual
gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig nntp
gnus-cache pp solar cal-dst diary-lib diary-loaddefs org-duration
cal-iso ol-eww eww mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnir
gnus-sum shr svg dom gnus-group gnus-undo gnus-start gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range
gnus-win gnus nnheader wid-edit ol-docview doc-view jka-compr ol-bbdb
ol-w3m face-remap term/xterm xterm magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos
magit-apply magit-wip magit-log which-func imenu magit-diff smerge-mode
diff diff-mode git-commit rx log-edit pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
server magit-mode transient cl-extra help-mode magit-git magit-section
magit-utils crm dracula-theme pcase x2bib org-ref-arxiv org-ref-pubmed
org-ref-isbn org-ref-helm helm-bibtex helm-net xml org-ref-core
org-ref-glossary org-ref-bibtex avy doi-utils url-http url url-proxy
url-privacy url-expand url-methods url-history mailcap url-auth
url-cookie url-domsuf url-util url-gw nsm message rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg epg-config gnus-util
rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader org-ref-utils org-ref-export
citeproc citeproc-itemgetters citeproc-biblatex citeproc-bibtex
ol-bibtex citeproc-cite citeproc-subbibs citeproc-sort citeproc-name
citeproc-formatters citeproc-number rst compile citeproc-proc
citeproc-disamb citeproc-itemdata citeproc-generic-elements
citeproc-macro citeproc-choose citeproc-date citeproc-context
citeproc-prange citeproc-style citeproc-locale citeproc-term citeproc-rt
citeproc-lib citeproc-s thingatpt queue ox-org ox-odt rng-loc rng-uri
rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns
nxml-enc xmltok nxml-util ox-latex ox-icalendar ox-html table ox-ascii
ox-publish ox org-ref-misc-links org-ref-label-link org-ref-ref-links
org-ref-citation-links xref project org-ref-bibliography-links hydra lv
bibtex-completion biblio biblio-download biblio-dissemin biblio-ieee
biblio-hal biblio-dblp biblio-crossref biblio-arxiv timezone biblio-doi
biblio-core let-alist url-queue ido hl-line parsebib bibtex f s dash
helm-files image-mode exif helm-buffers helm-occur helm-tags helm-locate
helm-grep helm-regexp helm-utils helm-help helm-types helm
async-bytecomp helm-global-bindings helm-easymenu helm-source
helm-multi-match helm-lib async helm-config ob-python python tramp-sh
tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat
shell parse-time iso8601 ls-lisp ob-R ob-octave ob-calc calc-store
calc-trail calc-ext calc calc-loaddefs calc-macs calfw-org org-capture
org-element avl-tree generator org-agenda org-refile calfw edmacro
kmacro holidays hol-loaddefs cl org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete
comint ansi-color ring org-list org-faces org-entities time-date
noutline outline easy-mmode org-version ob-emacs-lisp ob-core ob-eval
org-table ol regexp-opt org-keys org-compat advice org-macs org-loaddefs
format-spec find-func suomalainen-kalenteri cal-menu calendar
cal-loaddefs nyan-mode image finder-inf autorevert filenotify xt-mouse
info tool-bar package easymenu browse-url url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select mouse jit-lock font-lock syntax
facemenu font-core term/tty-colors frame minibuffer cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads inotify
multi-tty make-network-process emacs)

Memory information:
((conses 16 746274 125723)
 (symbols 48 51244 165)
 (strings 32 210772 25665)
 (string-bytes 1 6819907)
 (vectors 16 85295)
 (vector-slots 8 1828014 40632)
 (floats 8 1051 1754)
 (intervals 56 7078 6146)
 (buffers 1000 67))