Re: [Gnu-arch-users] Arch Cache & cached archives

[Aaron Bentley]

Eric Wong wrote:
> Aaron Bentley <address@hidden> wrote:
>
> > This is a description of the preliminary Arch Cache implementation with 
> > notes on future directions.  Please consider it a draft; comments and 
> > critiques are welcome.
>
>
> <snip>
>
> > Comparison with local mirrors
> > ==============================
> > - The user never downloads anything they don't need
> > - Commits are possible
> > - Never out of date
> > - Disconnected operation is not yet supported
>
>
> How about making the cache implementation compatible with a local
> mirror?

I don't think it makes sense to use the archive storage format for the 
cache.

An archive and requires full information to be present for a version. 
That is, if we store a log file, we must also store the changeset or 
import.  If we store patch-9, we must also store patch-8.

It also doesn't store some kinds of information that make sense to 
cache, like ancestor.

And it does store a whole bunch of anciliary files that should be 
irrelevent, like signature files and checksums.

My idea was that in order for the cache to serve as a mirror, we'd need 
to run a cache-fill command.  e.g.

tla cache-fill address@hidden
or
tla cache-fill address@hidden/tla--devo--1.3

This would ensure that all the revisions contained were buildable from 
the cache alone.  If it had suitable local data, it would build the 
necessary cacherevs.  Otherwise, it would download them.

Additional cache-fill options might be available, like tla cache-fill 
--update, which would download the latest changesets, logs, etc for 
versions that were already in the cache.  (or recently-used?)

>  I would like to be able to run [ar]browse (no options) as
> quickly as I do with local mirrors, and I'd like to streamline it with
> disconnected operation.

Of course, that's impossible while maintaining the "Never out of date" 
feature, and I wanted to keep that.  Now that Tom's talking about 
tunable expiry, so you could possibly cache that information for a 
longer period.

> archive-mirror:
>
> archive-mirror would have a --dirs option to create a directory skeleton
> for the local archive.  All the directories that are present on the
> remote archive are copied over, but remain empty (with the exception of
> sub directories).

The underlying implementation doesn't currently have a way to represent 
empty versions, but I'll likely change that.  I don't currently have a 
way to list versions, either.  Both of those will need to change before 
the cache can be substituted for a mirror anyway.

So yes, I can see cache-fill --dirs --expire 3600 creating a skeleton 
that would last for an hour.

> [ar]browse:
>
> [ar]browse would have an option to choose to use _only_ cached
> information, or access the remote archive as needed (--cached/--uncached?).
> I'm undecided on which method is better as the default, but as long as
> users have the option to choose, both the cache and local-mirror usages
> will be supported.

Err.  That's trickier to implement using the current archives.h interface.

> The remote option would be somewhat equivalent of what I currently do
> (tla archive-mirror && tla abrowse) with my local mirrors except I'll be
> able to avoid downloading changesets and patch-logs I don't need.

remote option?

Aaron