Re: [Gnu-arch-users] arch lkml

[Tom Lord]

    > From: address@hidden (Eric W. Biederman)

    > Arch makes a lot of distinctions about caches and mirrors and
    > normal archives.  It is my feel that this is showing the
    > limitations of arch.  Why can't these all handled in the same
    > way?  Code in an archive.

Scalability.

Look at it this way: we have some set of "raw data" (all the data stored
by `commit', `tag', or `import' across all of the archives in some
domain of consideration (e.g., "the free software developer community"
or "the developers employed by XYZZY Corp.").

That raw data expands over time according to some simple, core,
transactional rules (i.e., what `commit', `import', and `tag' mean).

At the same time, we have an _open_ended_ number of access patterns
for people reading that data.  Extremely open-ended with variations on
access patterns reflecting network topologies, what parts of the data
are needed more quickly than others, what kind of indexing is needed,
who's doing what concurrently, etc.

One idea is to look for a "silver bullet" archive format:  one that
will satisfy all of those access patterns and, at the same time,
preserve the transactional semantics of the three core operations.
This is, of course, quixotic quest.

A better idea, in my opinion, is to (a) optimize the heck out of those
three core operations;  (b) make it is as cheap as practical to move
data around, especially across networks;   (c) keep the data in a
format that makes it very _easy_ to create _ancillary_ data structures
that optimize the access patterns of a particular situation;  (d)
start building those ancillary data structures;  (e) do all this using
techniques that are simple enough to get Right.



-t