Re: Updating *Completions* as you type

[sbaugh]

Juri Linkov <juri@linkov.net> writes:
>>>>> I don't think this is realistic to add an individual option in all cases.
>>>>
>>>> That's not necessary.  We could also do possibility C that I described
>>>> before:
>>>>
>>>>>> C.
>>>>>> - Remove display-sort-function from the metadata
>>>>>> - add the 'read-kill category to the metadata
>>>>>> - add 'read-kill to completion-category-defaults
>>>>>> (diff is 3 lines)
>>>>
>>>> That seems simple and straightforward.
>>>
>>> Removing display-sort-function is still less safe
>>> than just adding a category.
>>
>> Why do you say that?
>>
>> The reason that comes to mind is that there are replacement completion
>> UIs which will need to explicitly add support for the category.  So
>> removing display-sort-function will affect them immediately, when they
>> might not yet have support for getting display-sort-function from
>> completion-category-defaults.
>>
>> That is true.
>>
>> But that actually suggests a further argument in this direction: if we
>> use user options which change the display-sort-function in the table
>> metadata, we'll have support for all completion UIs out of the box.
>>
>> That seems really desirable!  So maybe we do want a solution like A
>> where we add a user option?  Since that user option will work for all
>> completion UIs.
>>
>> Announcing "you can now customize the sorting order of a bunch of
>> completing-read-based things in this new way" but having that new way
>> only work for the default completion UI is a bit sad, although of course
>> they can support the new way eventually.
>
> This is what I believe they should do: we add a category,
> and they support it as well.

OK, I'm fine with that, but when we do that, I think the per-table
option should override the per-category option.

>>>>> I still don't understand why do you worry about this precedence when
>>>>> the user option completion-category-overrides is nil by default.
>>>>>
>>>>> Could you describe a use cases when such precedence might become a 
>>>>> problem?
>>>>
>>>> If some table needs an individual option (because the sorting needs to
>>>> affect the completion generation), but the table shares a category with
>>>> other tables, then that individual option will be overridden by the
>>>> category configuration.
>>>
>>> Agreed, this is a problem.
>>>
>>>> For example, project-prompt-project-name allows one to complete over
>>>> project names.  If I wanted to sort its completions by some detail of
>>>> the underlying project (how recently the git repo was updated, maybe),
>>>> that would require the table to change behavior.  So it would need an
>>>> individual option.
>>>
>>> Or an individual subcategory.
>>>
>>>> However, project-prompt-project-name uses the same category as
>>>> project-find-file.  So if the user configured sorting for
>>>> project-find-file, it will override the table-specific option for
>>>> project-prompt-project-name.
>>>
>>> I believe they should use two different subcategories, e.g.
>>> 'project-file' and 'project-name'.
>>
>> I agree, but...
>>
>>>> I suppose another option is to simply declare that every table has to
>>>> have a unique category.  That would make "category" a misnomer though.
>>>
>>> Even such subcategories as 'project-name' make sense to use in other
>>> possible cases when reading a project name.
>>
>> ...if the project-name category is used for other tables too, but the
>> option is supposed to be specific to an individual completion table,
>> then we have the same problem again.
>
> And an alternative to add separate options to all these tables
> doesn't look more attractive.

Yes, but we don't have to do that, I'm OK with a category-based
approach.  I just think we should reserve the *ability* to use
table-specific options, by making a table-specific display-sort-function
override the category-specific display-sort-function.

Anyway, we're going around in circles a bit here.  How about this patch
which only adds the new historical option to completions-sort?  I think
we're in agreement on everything in this patch, and maybe installing it
will get some user feedback which we can use when coming back to this
later.

>From c4435be3689278d380099f2341ac242bf74639df Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh@catern.com>
Date: Tue, 17 Oct 2023 09:09:55 -0400
Subject: [PATCH] Add historical option to completions-sort

Support sorting candidates in *Completions* by the order they show up
in the minibuffer history.

Also add minibuffer-sort-alphabetically and
minibuffer-sort-by-history, which are usable for both completions-sort
and display-sort-function.

* lisp/minibuffer.el (completions-sort): Document 'historical option.
(minibuffer-completion-help): Support 'historical option.
(minibuffer-sort-alphabetically)
(minibuffer-completion-base, minibuffer-sort-by-history): Add.
---
 lisp/minibuffer.el | 63 +++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 57 insertions(+), 6 deletions(-)

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 5c12d9fc914..dfb30e4364c 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -1314,14 +1314,26 @@ completion-cycle-threshold
 (defcustom completions-sort 'alphabetical
   "Sort candidates in the *Completions* buffer.
 
-The value can be nil to disable sorting, `alphabetical' for
-alphabetical sorting or a custom sorting function.  The sorting
-function takes and returns a list of completion candidate
-strings."
+Completion candidates in the *Completions* buffer are sorted
+depending on the value.
+
+If nil, sorting is disabled.
+If `alphabetical', candidates are sorted by
+`minibuffer-sort-alphabetically'.
+If `historical', candidates are sorted by
+`minibuffer-sort-by-history'.
+If a function, the function is called to sort the candidates.
+The sorting function takes and returns a list of completion
+candidate strings.
+
+If the completion-specific metadata provides a
+`display-sort-function', that is used instead and this value is
+ignored."
   :type '(choice (const :tag "No sorting" nil)
                  (const :tag "Alphabetical sorting" alphabetical)
+                 (const :tag "Historical sorting" historical)
                  (function :tag "Custom function"))
-  :version "29.1")
+  :version "30.1")
 
 (defcustom completions-group nil
   "Enable grouping of completion candidates in the *Completions* buffer.
@@ -1647,6 +1659,43 @@ minibuffer--sort-preprocess-history
                      (substring c base-size)))
                  hist)))))
 
+(defun minibuffer-sort-alphabetically (completions)
+  "Sort COMPLETIONS alphabetically.
+
+COMPLETIONS are sorted alphabetically by `string-lessp'.
+
+This is a suitable function to use for `completions-sort' or to
+include as `display-sort-function' in completion metadata."
+  (sort completions #'string-lessp))
+
+(defvar minibuffer-completion-base nil
+  "The base for the current completion.
+
+This is the part of the current minibuffer input which is not
+being completed on.  This is primarily relevant for file names,
+where this is the directory component of the file name.")
+
+(defun minibuffer-sort-by-history (completions)
+  "Sort COMPLETIONS by their position in `minibuffer-history-variable'.
+
+COMPLETIONS are sorted first by `minibuffer-sort-alphbetically',
+then any elements occuring in the minibuffer history list are
+moved to the front based on the order they occur in the history.
+If a history variable hasn't been specified for this call of
+`completing-read', COMPLETIONS are sorted only by
+`minibuffer-sort-alphbetically'.
+
+This is a suitable function to use for `completions-sort' or to
+include as `display-sort-function' in completion metadata."
+  (let ((alphabetized (sort completions #'string-lessp)))
+    ;; Only use history when it's specific to these completions.
+    (if (eq minibuffer-history-variable
+            (default-value minibuffer-history-variable))
+        alphabetized
+      (minibuffer--sort-by-position
+       (minibuffer--sort-preprocess-history minibuffer-completion-base)
+       alphabetized))))
+
 (defun minibuffer--group-by (group-fun sort-fun elems)
   "Group ELEMS by GROUP-FUN and sort groups by SORT-FUN."
   (let ((groups))
@@ -2409,6 +2458,7 @@ minibuffer-completion-help
       (let* ((last (last completions))
              (base-size (or (cdr last) 0))
              (prefix (unless (zerop base-size) (substring string 0 base-size)))
+             (minibuffer-completion-base (substring string 0 base-size))
              (base-prefix (buffer-substring (minibuffer--completion-prompt-end)
                                             (+ start base-size)))
              (base-suffix
@@ -2473,7 +2523,8 @@ minibuffer-completion-help
                                             (funcall sort-fun completions)
                                           (pcase completions-sort
                                             ('nil completions)
-                                            ('alphabetical (sort completions 
#'string-lessp))
+                                            ('alphabetical 
(minibuffer-sort-alphabetically completions))
+                                            ('historical 
(minibuffer-sort-by-history completions))
                                             (_ (funcall completions-sort 
completions)))))
 
                       ;; After sorting, group the candidates using the
-- 
2.42.1