Re: Subprojects in project.el (Was: Eglot, project.el, and python virtual environments)

[Dmitry Gutov]

On 28/11/2022 06:10, Tim Cross wrote:

> > OT1H, large projects are relatively rare. OT2H, having a need for subprojects 
> > seems to be
> > correlated with working on large projects.
> >
> > What is the common case, in your experience, and how is it better solved? 
> > Globally
> > customizing a list of "markers", or customizing a list of subprojects for every 
> > "parent"
> > project?
>
> In my personal experience, sub-projects have been more about project
> structure and not size. I would agree they are more prevalent in large
> projects, but can exist in medium and even smaller projects.

Sure.

> I don't think I have a preference for customizing a list of markers or a
> list of sub project definitions per project. I suspect different
> approaches will work better in different scenarios and neither is a
> clear 'winner'. However, as pointed out by Stephan, terminology
> confusion/meaning may well be contributing to the confusion here. Not
> only am I unsure everyone is thinking the same thing when talking about
> sub-projects, I'm not sure everyone is even talking about the same thing
> when referencing 'project'.

I suppose another way to look at it is, subprojects could be defined by 
technical boundaries (e.g. this dir is a frontend SPA, and that dir is 
our backend), or by social (we have a monorepo, but this dir belongs to 
our team).

The former approach should be easier to support reliably using markers, 
and the latter -- not necessarily so.

But I'd be happy to find out that 98% of our users' cases can be handled 
with markers.

> I wrote a lot about how I use projects and sub-projects in my work flow
> and then realised it probably isn't helping that much. It struck me that
> perhaps the issue is that the notion of sub-projects isn't really that
> useful in itself and may actually be more detrimental than useful.

It all depends on the individual workflows, for sure.

> When you think about it, a sub-project is really just a more narrow
> project focus. A project is really just a collection of files and
> environment settings which can be considered, for some purpose, as a
> 'unit' in itself. It might define the set of files used when considering
> find and replace for a symbol, when looking for symbol completion
> candidates, or file/buffer switching, opening, linting, cross
> referencing etc. It may correspond to a VCS repository, but it may
> not. It could cut across repositories, or it could be made up of
> multiple repositories or it could simply be some bespoke virtual project
> concept specific to a particular use case.

Enviroment settings -- we do not support (yet?). But depending on the 
person and the goal, dir-locals.el can help quite well.

Regarding cutting across repositories, etc, there is a line for cases 
wre can easily (or with minimal customization) support ooutb, with 
auto-detection that a lot of the users prefer. Versus very custom shapes 
which require one to write a custom backend.

But if someone has a particularly handy way to define an 
arbitrarily-shaped project, they can submit that backend for inclusion 
as well. We could call it "free-form", or something.

> I guess what I want is the ability to define arbitrary collections of
> files and environment settings as a project, have a way to select/target
> a project and an API which various tools can use to get the files or
> environment settings to then operate on. Whether one project can be
> considered a sub-project of another project is less relevant compared to
> the ability to select/identify the target project. Automatic definition
> of projects based on VCS repositories is great and a real time saver,
> but the ability to define what makes up a project manually is also
> important. The ability of the system to automatically determine which
> project is 'active' (for example, based on the location of the file
> being opened) is good and having the system prompt you when it isn't
> clear or when there are multiple options is useful, but just being able
> to run a command to set the current project would also be
> sufficient. However, how one project relates to another project i.e. sub
> project, main project, etc, seem of limited use compared to just having
> the ability to select a sub-set of the files and environment settings of
> a project, whether we call these sub project or nested projects or
> whatever, seems of limited benefit.

Regarding the latter, there are a couple of requests now to be able to 
run a particular action (project-find-file or project-find-regexp) 
against either the current project or the "parent" project (determined 
by the locations of the root dirs), based on the prefix argument. That 
seems reasonable enough.