bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames

[Dmitry Gutov]

Hi Philipp,

Sorry for the long pause.

On 05.09.2021 20:14, Philipp wrote:
> > Am 18.07.2021 um 02:53 schrieb Dmitry Gutov <dgutov@yandex.ru>:
> >
> > On 05.07.2021 22:05, Philipp wrote:
> >
> > > > The difficulty is having a method like project-files return one format for some 
> > > > users, and another for users who want to take advantage of this performance 
> > > > improvement. Or we break the compatibility and/or introduce a new method with 
> > > > this new behavior.
> > > A general design approach in OOP is to not treat abstract virtual functions 
> > > (generic functions in ELisp terminology) as part of the public interface of a 
> > > type; i.e., abstract functions can be implemented, but shouldn't be called 
> > > outside of the module that defines them (project.el in this case).  That allows 
> > > for changes like this: implementers could freely return the new fileset 
> > > structure because only project.el would call project-files.  Not sure how much 
> > > ELisp code adheres to this principle, though.
> >
> > When you say "abstract virtual functions", do you mean OOP as in C++ OOP? I'm 
> > not sure about standard practices there, but this sounds more like C++ and less like OOP 
> > in general.
> >
> > I'm looking as generic functions here as part of an interface signature (like 
> > Java or Go interface). They are programmed against (which is the case with 
> > project.el) and are supposed to be stable.
>
> I think the idea is applicable to most programming languages that have some 
> form of subtype polymorphism.  Basically, for a normal (monomorphic) function, 
> you can make the parameter types more general or the return type over time more 
> specific over time without breaking compatibility.  For a polymorphic function 
> that's only specialized but not called outside the defining entity, e.g. a 
> private virtual function in C++ or a method marked as @ForOverride 
> (https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/ForOverride.java)
>  in Java, it's the other way round: you can make the parameter types more 
> specific and the return type more generic over time.  That implies that for a 
> polymorphic function that's also called outside the defining entity, you can't 
> change any of the types without breaking compatibility.  Thus the suggestion to 
> separate the interface for callers from the interface for 
> subclasses/specializers.

I'm not quite familiar with the practice, and since we don't do type 
parameterization here, the justification probably doesn't apply.

> > > If there's too much code (outside of project.el) that relies on project-files 
> > > returning a list, we need to indeed fall back to some of the other options.
> >
> > A new method seems to be the way forward. Or, say, an ad-hoc argument which 
> > determines whether file names should be relative.
>
> I guess you also can't introduce new parameters without breaking compatibility 
> either.

That is also true.

So we'll probably go this way: I'm already experimenting with a method 
called project-files-filtered in the scratch/etags-regen branch.

> That would only leave the new method possibility.  We could then say that 
> nothing outside project.el should call it to avoid the above problem.  Ideally, 
> the byte compiler would support a declaration form similar to @ForOverride to 
> warn about such invocations.

I get the idea of having a Template Method pattern where the public 
method is something else (this approach seems to be described in 
ForOverride.java), but I'm not sure how to apply it to our case. I.e., 
what kind of data the "public" function would accept and return.

If you mean it would return a list of absolute file names, that would 
actually limit its usefulness.

And/or perhaps you imagined that our projects could have a "private" 
method which would not be "exported" to outside code, but which 
project-find-regexp would be able to use? I don't really like that 
approach, outside code should be able to reimplement the same features.