Re: [Gnu-arch-users] Re: Archives use cases

[Clark McGrew]

On Tue, 2003-10-07 at 18:14, Miles Bader wrote: 
> Clark McGrew <address@hidden> writes:
> > Does anybody have experience dealing with archives with large numbers of
> > categories?[1]  How have you organized the project?  What are your
> > experiences keeping rein on category naming issues?
> 
> Keep in mind that it seems to be an implicit assumption in arch that
> archives will tend to be developer/time specific, so while some problems
> might require naming conventions to be used for category/branch names,
> there will tend to be an upper bound on how many names are actually in
> any particular archive (unless you're absurdly prolific :-).

I can see how that would work with a project that is developing a small
number of packages, but how does that work in a larger project.  Using
an example dear to my heart, the superk software is a collection of many
different packages, and a large number of the libraries will be needed
to build any given program.

Consider a hypothetical first year graduate student faced with several
megabytes of source code, and unfamiliar with all of it.  She has been
asked to compile and run something by tomorrow so the results can be
discussed *now*.  How is this student suppose to find all of the
categories needed to run her program.  Particularly if the packages are
spread over several other peoples archives?  A config package will solve
the distribution problem, but the next step will be to make a change in
the code.  It is very helpful if newcomers are faced with a logical
naming scheme.  Side documentation helps, but good organization is
essential.  

The other consideration is that although we physicists tend to work like
a disorganized free software project, we try very hard to separate the
individual from the code.  That's because much of the code is developed
by graduate students and post-docs who will be leaving the project in a
couple years; however, the code will survive for decades.  I think that
we need a central archive (or small set of archives) that keep the
official version of the code, and those archives must be structured to
handle a large number of categories.  I also suspect that it won't be
feasible to change archives frequently.  

As you can see, I expect a revision control system to be used for two
related tasks.  First, it needs to keep track of the on-going software
development.  Second, it needs to serve as a distribution system that
propagates the code to a large number of individuals.  

My current thinking is to divide the archives by working group and
"require" the working groups to provide a config category.  A top level
config package will then bring all of the working groups together.

superk--release/
     configs/
     base-libraries--release/
        configs/
        skread--release/
        zbs--release/
        geom--release/
        iolib--release/
        etc...
     atmpd--release/
        configs/
        etc...
     lowe--release/
     upmu--release/
     online--release/
     offline--release/
     sk-k2k--release/
     etc...

The goal is that the levels are isolated.  For instance superk--release
won't worry about the specifics in base-libraries and hopefully each
sub-group becomes a manageable namespace.  In a larger collaboration, it
might be worthwhile to add further depth.  This also allows for multiple
top level categories that can be customized (for instance, a top level
superk-online--release might be a cut down version to be installed on
the hardware control computers). 

The next step is to play with it and see if I can make it work from a
technical point of view.  I'm getting the impression that I'm heading
into new territory as far as using tla in a largish academic
collaboration.  I see real advantages gained by using distributed
development archives where each person keeps local branches for
categories that they are working on.  The trick will be to setup a
structure to keep the flexibility from degenerating into chaos.

Thanks,

-Clark
-- 
Clark McGrew                    Univ. at Stony Brook, Physics and Astronomy
<address@hidden>        631-632-8299