Re: Generation of tags for the current project on the fly

[Dmitry Gutov]

On 1/12/18 9:52 PM, Eli Zaretskii wrote:

> In my workflows, I do that all the time, because I don't always
> remember the details of the functions I need to call in the code I'm
> writing.

Sure, but not as often as you use completion-at-point, probably. Anyway, 
what I said was an approximation/simplification. People's workflows are 
bound to be different.

> > Failing to find a tag is a valid result (some identifiers can be absent,
> > or defined somewhere else, e.g. in the libraries), and doing a rescan
> > each time that happens might be more annoying.
>
> If you maintain that scanning is fast, then the annoyance should be
> minimal.

If scanning is fast, invalidate-on-save should be good enough. And it's 
easier to implement (already is).

> > > We could offer generating a tags table if we don't find one in the
> > > tree, instead of generating it automatically.
> >
> > And then what? Visit it?
>
> No, just do what you intended, but only after an approval.  It could
> be that the user thought she already visited a tags table, or some
> other mistake.

OK, so if the user says yes, we "temporarily visit" to auto-generated 
tags table. Then the user saves a file and that table get invalidated 
(or via some other mechanism), and we want to index it again. Ask again?

> > > > For reference, indexing the Emacs sources takes ~1.1sec here.
> > >
> > > Was that with cold cache or warm cache?
> >
> > Warm, probably. But that's the relevant time, isn't it?
>
> Not necessarily.  The first time a tree is scanned could well be the
> shortly after you start working on a project.

Not sure what you mean. The tree has to be scanned *sometime* at least 
once, hasn't it?

> > 'make tags' makes 1 second on my machine, with an NVMe disk.
>
> I bet it will be even faster with a RAM disk.  But we shouldn't base
> our decisions on such configurations, as that isn't the norm yet, I
> think.

NVMe is a bus for an actual storage device, though. Anyway, 1 second and 
4 seconds are different, but not hugely different. And we haven't 
optimized everything we could, yet.

For instance, could you try to see how long takes the generation of the 
file list alone? And populating the buffer with it. But without passing 
it to etags.

> > > IOW, I don't think this is so fast that we could do that without user
> > > approval.
> >
> > The argument here is that if the user called xref-find-definitions, it's
> > better to do a (long-ish) scan and show something, instead of failing.
>
> It could be a mistake, or the user could reconsider given the
> question.  We do that with visiting large files, for example.

That's a valid argument. On the other hand, they might not know how long 
the indexing will take anyway.

> > > I don't understand why you didn't use the commonly used form:
> > >
> > >      find . -name "*.rb" -o -name "*.js" ... | etags -o- -
> >
> > Because the project API doesn't make this easy. Anyway, generating the
> > full list of files is relatively fast in comparison.
>
> Invoking 'find' will always be faster, as it's optimized for
> traversing directory trees.

'git ls-files' will probably be faster still.

> > > I think
> > > using 'find' is also faster.
> >
> > find is used under the covers. The difference is just that the
> > invocations of etags are only happening later.
>
> No, the difference is also that in my example etags runs in parallel
> with 'find', not in sequence.

That's what I was trying to say.

> > 'make tags' is very much specific to Emacs.
>
> No, TAGS is a standard target in GNU Makefile's.

OK, good to know. Two questions, then:

- Can we make it output the tags to stdout?
- Can we detect than a given Makefile has a proper TAGS target (that can 
output to stdout)?

Not sure yet how to handle the TAGS files inclusions, though.