[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Emacs-diffs] emacs-25 f8208b6: Document the user-level features of

From: Dmitry Gutov
Subject: Re: [Emacs-diffs] emacs-25 f8208b6: Document the user-level features of the Xref package
Date: Sun, 10 Jan 2016 21:05:35 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:43.0) Gecko/20100101 Thunderbird/43.0

On 01/10/2016 06:50 PM, Eli Zaretskii wrote:

Sorry, I don't understand: AFAIK (and you seem to confirm) we do use
these tools in some situations, so the text is correct in saying that
they are used when appropriate.

No, these are programs that we just use in some circumstances.

The text says: "Each such tool is used as a @dfn{backend} by commands described in this section". This is untrue, they're not used as "backends" in Xref terms. And most of them are only used by one command (singular, not plural): xref-find-references, and only when an actual backend doesn't provide a replacement logic.

Is it that I used "backend" in a meaning
that's different from what the xref implementation uses?

Yes. You can call them "programs", or "external tools" like CEDET's docs prefer to call them.

This part also makes using the term "backend" for them inaccurate: "Each command detects which backends are available for the current major mode".

Xref backends *do* depend on the current major mode. Or can depend, at least. But whether we use ggtags, cscope, or grep, doesn't depend on the current major mode at all, just on whether there is an index file generated by the respective external program, in some root directory.

semantic-symref-filepattern-alist uses the major-mode value to derive the file wildcards to search for, but not, again the tools to use.

If so,
that's deliberate: this is a user manual, and that was the only
reasonable way I could think of to convey the idea of several backends
without talking about technicalities like xref-backend-references,
what symref does, when do we use the fallbacks, what are generic
functions, etc.

A backend is a value that can be returned by xref-find-backend. By diluting the term, you all but make sure that the same users will become confused later, when they peek under the covers, to try to fix something, or use xref in their own programs, etc.

If you think the users are fine with only superficial understanding, let's keep the description at that level, and not include misleading specifics.

The idea the user will hopefully take out of this is that we use
several potential helpers under the hood some of which are based on
the mentioned tools that are not part of Emacs proper.  The tools are
mentioned to encourage the users to install them, so that their Emacs
environment is more complete and capable.

Let's call them "external tools", then.

     +  The commands in this section visit and search all the files listed
     +in the @code{xref} backend's database, one by one.  For these
     +commands, the database serves only to specify a sequence of files to

That may be true for tags-search, but that's never true for 

It's true when xref-find-references uses IDUtils, for example.  And
even when it doesn't, the command does use _some_ database, even if
that database is made up of doc strings loaded into memory plus

The specific sentence that isn't true is: "For these commands, the database serves only to specify a sequence of files to search."

No, not only. It serves for more, IIUC.

Again, here you present unnecessary specifics (backend has a database, database is only a list of files), which aren't really true in all cases.

Once again, look at this from the user POV, not from the POV of
someone who knows how the implementation works.  Users need some
mental model of what's going on to make some sense and order out of a
heap of potentially unrelated features.  The above should provide
users with a conceptual framework that facilitates coming up with such
a mental model.  At least I hope it will.

I'd rather the imaginary framework didn't stray too far from reality.

If the current backend defines xref-backend-references, then we just ask it, 
and show the results.

If it doesn't, we delegate to the first available "CEDET tool", but they do 
perform the regexp search, not just list the files. Then, xref-collect-references takes 
the list of matches and verified, for each one, whether the match begins and ends on a 
symbol boundary (Grep can't check that).

I'm not describing xref alone, I'm describing what xref _and_ its
backend do together.  Differentiating between the two parts of the job
is not useful for the purposes of the user manual.

You _don't know_ what an arbitrary backend does. Xref just asks it, and the backend returns a list. You can describe what elisp and etags backends do, but that's not what's happening here.

In any case, what you describe above is not something that can be
reasonably said in a user manual, can it?  You describe the (current)
implementation, whereas the user manual has quite different goals.

No, it's you who seem to be describing the implementation, and end up doing it not accurately. I'd be entirely fine with not describing the implementation at all.

You are saying that the description I came up with loses some
accuracy, which is something I'm aware of, and which is inevitable
when describing complex software made up of several independent
modules whose features only partially overlap.

I think we should either describe things accurately, or omit describing what we don't know. "Magic happens" is fine, the user just needs to know the initial requirements and the end result.

On the other hand, I haven't seem some crucial information mentioned:

- The the databases used by these tools can get out of date, and it's the user's responsibility to update them. (Hopefully, we will solve this problem sometime).

- Each of the external tools, except for Grep, supports only a fixed set of programming languages. If, in a multi-language project, we open a file using an exotic-ish language, and ggtags database is present in the project root, xref-find-references will return an empty result, erroneously.

If you have suggestions for how to convey these ideas without going
into implementation, I'm all ears, of course.

I'd prefer something like:

Xref has a set of commands, which ask for data from the currently active Xref backend. And use its identifier-at-point and identifier-completion-table logic as well.

And there's this special command called xref-find-references. We wanted functionality like that very much, but turns out that both built-in backends (elisp and etags), aren't good at this sort of thing.

So we've written a default implementation for xref-backend-references, which is used when a backend can't give us better data. That implementation uses the "external tools" feature borrowed from CEDET, that uses one of the tools from the list, provided the necessary program is present on the computer, and a database file has been generated, falling back to Grep. We could always use Grep here, but ggtags and friends are faster. On the flip side, their databases don't support all languages, and can get out of date, we're very sorry about that.

Oh, and by the way, that default implementation also uses the Project package, to determine which directories to scan.

What list? xref definitely uses one database at a time (per project root, if 
we're talking about the default implementation).

Most of the commands described in the section from which this text
comes are in etags.el, for which the description is accurate.  The
only other command is M-? (xref-query-replace), for which this is

M-? is xref-find-references.

somewhat less accurate, but not entirely incorrect: a single database
is still a special case of "a list of all relevant databases".

This looks confusing to me.

Once again, feel free to suggest more useful ways of conveying the
ideas such that they make sense to users.  The above was the best I
could come up with, and I'll be happy to hear about possible
improvements, if they exist.

In this case, I'm not entirely sure why that piece of text exists.

     address@hidden tags-search
The last sentence makes it seem like it goes through the xref machinery, 
whereas it doesn't.

The last sentence mentions backends because the parent section is
built upon that concept: that these commands use the appropriate
backend as they see fit.

And this is blatantly inaccurate in this case. I think that tells us that the sections' organization should be different.

That concept was introduced as an attempt to
make sense of the fact that xref implemented only part of what's
available with etags.  I do indeed think that IWBNI xref provided a
front-end for these features as well

But then, it kind of already does, doesn't it? You can initiate a regexp search with project-find-regexp (and even choose the directory with C-u), and perform the query-replace from the results buffer.

It would be easy to create dired-specific version of this command, that will use the marked directories, if we want to. Just let me know what to call it.

This makes it seem like tags-loop-continue is applicable for use after 
xref-find-* commands as well. But it isn't.

Well, it explicitly references tags-search and comes directly after
the description of that command.  Would it be better if the first
sentence said that explicitly, like below?


   Having found one match with @kbd{M-x tags-search}, you probably
   want to find all the rest.

Maybe this command, as well as tags-query-replace, the other etags-only 
commands, and the related etags variables, should still remain in a separate 

No, I don't think so.  It would make two sections out of one, which
would mean less order in the description.

I really think there should be separation. etags-specific commands, and commands that _can_ dispatch to etags (but not necessarily do), and go through Xref, are very different beasts. They provide different UI, too.

Is xref entirely not involved in completion-at-point-functions?

Entirely, yes. But I expect some major and minor modes will simultaneously add elements to completion-at-point-functions and xref-backend-functions, and those elements could dispatch to the same underlying logic under the covers. But the APIs are separate.

Do we
still use etags there?

etags by default, elisp-completion-at-point in emacs-lisp-buffers. Other major and minor modes set their completion functions as well.

The original text mentioned etags, and I
imagined that was replaced by xref now.  (I couldn't trace the code to
see what was it actually doing.)

You can look at the value of completion-at-point-functions in a c-mode, and an emacs-lisp-mode buffers.

       mode is enabled, it tries to use the Semantic parser data for
       completion (@pxref{Semantic}).  If Semantic mode is not enabled or
       fails at performing completion, it tries to complete using the
     -selected tags table (@pxref{Tags}).  If in Emacs Lisp mode, it
     +available @code{xref} backend (@pxref{Xref}).

Also not true. Like mentioned, we don't use xref backends for completion. 
Unifying in-buffer completion and xref identifier completion might be 
beneficial, but it's not easy to do.

I only changed the cross-reference, since there's no node named "Tags"
anymore; and etags is one of xref backends.

etags contains an Xref backend (which is mostly a set of functions that have both "xref" and "etags" in their names), but I wouldn't call the whole etags package an Xref backend.

In particular, tags-completion-at-point-function is not used by the Etags Xref backend. They both delegate to tags-lazy-completion-table, however.

So I think this sentence confuses things by implying that Xref backends are responsible for in-buffer completion. I.e., in practical terms, implying that one could register a new Xref backend (by adding an element to xref-backend-functions), and that would take care of completion support. Which isn't true.

Are you saying that
completion-at-point has been changed not to use the tags table when
the Semantic mode is not enabled?  Or are you saying that this text
was incorrect even before my changes, and we didn't use tags tables
even before xref was added?

I wouldn't call the text entirely accurate before (since it only lists Emacs Lisp mode as an exception, implying all other modes use etags for completion). But it's the new text that I'm really worried about.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]