bug#27193: 25.2; tmm should use completing-read-default

[Ryan Thompson]

(Note: I think several more replies came in while I was writing this; I haven't read those yet.)

Ok, so if I'm understanding correctly, the thrust of your argument is that completing-read is a very general function that supports a very wide range of behaviors/usages, to such a degree that tmm's usage does not seem like an anomalous or unusual way to use it. I might not agree on what constitutes a typical usage of completing-read, but I can definitely agree that it is a very general function, and in practice no alternative value of completing-read-function that works substantially differently from completing-read-default is likely to be capable of replicating all of that functionality.

So given your recommendation that alternative completion functions should "punt" cases they can't handle back to completing-read-default, along with the high likelihood that every alternate completion system is going to have something that it can't handle and needs to punt on, would you be interested in a patch that implements that punting mechanism within completing-read itself, so that ido-ubiqutous, ivy, helm, and others don't all have to invent their own punting code? I'm thinking along the lines of wrapping the call to completing-read-function with a condition-case that catches a "completing-read-fallback" signal, and calls completing-read-default instead. So any completion system that wants to punt just has to throw the appropriate signal.

Some replies to specific points, which may or may not be moot:

 

Please think about providing a different implementation of tmm, which doesn't use `completing-read'. That will let us know whether `completing-read' is used only as an implementation detail.

Magit implements a very similar system of using single-key shortcuts to activate items from a textual menu, with multiple levels of menus. It doesn't use completing-read.

Of course it does completion.  Erase chars in the minibuffer from the right and then hit TAB - completion. It's not a completion to write home about, but it's completion.

I would say that the fact that tmm does completion on hitting TAB is an internal implementation detail leaking out, i.e. an unintended side-effect of the fact that it uses completing-read internally. The main functionality of tmm is single-key shortcut access to menu items, not completion of user-entered text with matching strings from a set of choices.

 

 

So pluggable completion backends don't make any sense for tmm,

 

That too sounds overly broad. I think that you have too readily convinced yourself that this and that don't make any possible sense, and you are prone to nail things down to prevent what you see as the nonsensical. If such were truly nonsensical then they would likely never exist, and there would be no reason to forbid them.

 

I sense that you think of "pluggable completion backends", that is, uses of `completing-read-function', as necessarily something similar to what your code does.

 

I don't think I'm making a broad statement. My claim is that settings that affect how completion works should not affect tmm because tmm isn't doing completion (or rather, tmm is doing completion only incidentally due to a leaky abstraction). I still stand by this claim, regardless of whether tmm's use of completing-read is considered typical or not. (If you accept this claim, there's probably some other completion-related settings that tmm should also ignore.)

[FWIW I don't think of `completing-read-function' as providing only a "pluggable completion backend". Do you think of `isearch-search-fun-function' as a "pluggable search backend"? Backend? frontend? There are many angles to something like completion or search or... There is not just a backend and a frontend. Likewise, what you call an "alternative completion system".

Well, I'm not sure exactly what to call it. I suppose "alternative values of completing-read-function" is more technically correct, but it's a real mouthful.

 Really, we're just talking about a completion function - a value for `completing-read-function'. That has wide scope - there are few constraints on what it must do. I don't see the justice in adding a constraint for tmm that it cannot do anything except `completing-read-default'.]

 

and I can't imagine any other value of completing-read-function that would make sense for tmm besides completing-read-default.

 

All you need to imagine is something similar to `completing-read-default' but slightly different in some respect. That's all you need to imagine, but nothing prevents more imagination.'

I was implicitly assuming that no one would bother to implement a new completion function from scratch unless it differed substantially from an existing one.

 

And what you or I can imagine is not really the point. The question is whether tmm.el must limit itself to `completing-read-default'. I don't see why it should, just because we know of some values of `completing-read-function' that don't do the right thing for tmm.

 

Looking at the "git blame" output for tmm, that call to completing-read has definitely not been updated since completing-read-function was introduced except for minor bugfixes,

 

That's irrelevant.

 

so it makes sense that tmm would be expecting one and only one implementation of completing-read.

 

That does not follow at all.

 

This kind of argument could (inappropriately, IMO) be
applied to any number of completely normal uses of
`completing-read'.

I see no reason to impose a dichotomy of either a
`completing-read-function' similar to yours or else
`completing-read-default'.  There are likely other
benign values of the variable, besides just
`completing-read-default'.

 

I'm not trying to set a general precedent here. tmm is the only code that I'm aware of that uses completing-read in this way.

 

I agree that it is not a common way of using it. It doesn't follow that the only possible value of `completing-read-function' that is compatible with tmm is `completing-read-default'. Not at all.

 

It sounds like (and no, I haven't looked into it;
it just sounds like it) you have some particular
`completing-read-function' values in mind, which
you have found are incompatible with tmm's use of
`completing-read'.

 

The alternative completing-read-function providers that I am aware of are of are ido-ubiquitous (written by me), ivy, and helm. I'll also include icicles, even though uses some other mechanism besides modifying completing-read-function. ido-ubiquitous and ivy both have code to deactivate themselves when completing-read is called by tmm because otherwise their completion systems would break it, while icicles and helm simply break tmm when enabled. Thus, to my knowledge there is currently no other completing-read-function that doesn't break tmm (except for those that make exceptions specifically for tmm).

 

Again, it is irrelevant that there are uses of `completing-read-function' that break tmm. And what you or I am aware of, or even what might exist anywhere today, does not define the scope of possibilities.

 

[I drink values of the variable `liquid'. Some values, such as strong sulfuric acid, are quite incompatible with my proper functioning. That doesn't mean that the only possible value or the only compatible value for me is the default value, `water'.

 

I drink blindly - I don't know what's in the glass. The only requirement my mouth imposes is that the variable value be a liquid. It is up to whoever fills my glass to DTRT.]

 

And if those uses of `completing-read-function' are incompatible with tmm, and they thus deactivate themselves for tmm commands, that is exactly the right approach, IMO. It is exactly that which I suggested to you (without knowing that that is what you already do).

 

[Tmm does work with Icicles, BTW. (But Icicles does not use `completing-read-function', among other reasons because it wants to work also with older Emacs versions.)]

 

If so, that's not an argument for preventing the use of
other values of `completing-read-function' with tmm.
(Clearly the value `completing-read-default' is fine,
for instance.)  That's not an argument for tmm to do
something to prevent all use of `completing-read-function'.

Instead, it's an argument for the code that defines and
uses a particular `completing-read-function' to take
care of the contexts where it makes sense, and to stop
itself from being used in other contexts, where it might
cause trouble.

Only that code knows the kinds of context where its own
`completing-read-function' makes sense and where it does
not.  Code such as tmm should not try to guess what kinds
of trouble different values of `completing-read-function'
might cause.

I don't think tmm should throw up its hands and say, "Gee,
there might be some values of `completing-read-function'
that are troublesome, so let's just prevent all use of
that variable."  That doesn't make sense, to me.

 

Based on my explanation above, that is precisely what I think tmm should do: avoid using completing-read-function entirely.

 

I know you think that, but I don't see why. There are surely values of `completing-read-function' that do not bother tmm. We know of one already: `completing-read-default'. Why would you suppose that there can be no others?

 

To look at it another way, tmm was originally written to use completing-read as an implementation detail, and later the function that used to be called completing-read was renamed to completing-read-default, but tmm was never updated to use the new name. This patch rectifies that.

 

That's completely imagination. `completing-read-default' is not the new name of what was once `completing-read'. `completing-read' has never used code like `completing-read-default'. (It has always been written in C.)

 

What I would say that perhaps you will think goes a bit in your direction is this: If you can rewrite `tmm.el' so that it has the same behavior without using `completing-read', and the code is at least as simple and easy to maintain, then I'd say go ahead and do that.

 

That would support your idea that `completing-read' is only an implementation detail etc., and the question of `completing-read-function' would become moot.

I've provided Magit's popups as an example of similar functionality implemented without completing-read. I'm not a user of tmm, I just want to make sure my completion function doesn't break it.

 

 

If you want additional suggestions, maybe describe just
what the problem is that your completion function causes
for tmm.  It's hard to offer suggestions if you only
state that it is incompatible, without going into any
detail.  (Not that you must ask for input about this.
But if you would like some then giving more detail might
help.)

Please use your own judgment (as I said, I don't really
care much about `tmm'), but a priori this sounds like
overkill.

It sounds a bit like trying to bend Emacs to fit your
`completing-read-function'.  I can understand such a
motivation, believe me; I don't ascribe a bad intention
to you.  A guess is that you are not sure what to do,
to prevent inappropriate application of your value of
`completing-read-function' in this or that context.

If you think it causes trouble in some contexts, or it
is not able to handle some contexts properly, then
I'd suggest you consider walling it off from those use
cases.  It might take some time to discover which
contexts it causes trouble for, but that's OK - you
could add them as you discover them.  Tmm sounds like
a start. 

The right approach, IMO, is to teach your code when to
use its `completing-read-function' and when not to use
it.  Put differently, consider teaching your
`completing-read-function' when and where to hold back
and just punt to the default behavior.

 

This is exactly how ido-ubiquitous and ivy both currently work: they essentially have a blacklist of callers for which they revert to standard completion. tmm is on the blacklist for both packages.

 

That's great. Problem solved. That's the right approach, IMO.

 

Certainly, for any alternative completion system

 

Any? Again with the superlatives. There is more to the world...

 

there will be cases where it needs to fall back to standard completion. In my view, the completion system should be able to determine purely based on the calling context (i.e. its arguments and any relevant dynamically-bound variables) whether or not it needs to punt. Making this decision based on the name of the caller instead of the context to make this decision is admitting that not enough context was provided. I view it as a workaround, not a desirable design pattern, and someday in the future I hope to be able to remove the blacklist from ido-ubiquitous.

In the case of tmm, the best heuristic I can think of would be to inspect the key bindings of all the letters and numbers. If any of them are locally rebound in the minibuffer to something other than self-insert-command, then punt.

 

That would be silly, IMHO. There can be plenty of uses of `completing-read' where letter or number keys are bound to commands other than `self-insert-command'.

 

[FWIW, though not directly relevant, when Icicle mode is on there are no keys bound to `self-insert-command' in any of the minibuffer completion keymaps. (But there are keys bound to `icicle-self-insert'.)]

My heuristic was meant specifically for ido-ubiquitous: the heuristic is that if any letters or numbers are bound to something other than self-insert-command, the caller is probably doing something fancy that won't work well with ido completion. In any case, it was merely meant as one example of how a completion function might infer from context whether it should punt without having to identify callers by name.

Anyway, regardless of whether this patch is accepted or not, ido-ubiquitous will still need a blacklist either way, since functions like read-file-name that are already covered by normal ido-mode will always be on that blacklist. So I'm not exactly submitting this patch to make my implementation easier. I just thought that having tmm ignore completing-read-function would make it less fragile and mean one less obstacle for anyone implementing a new complting-read-function.