[Gnu-arch-users] Re: error in tree-lint

[Miles Bader]

Pau Aliagas <address@hidden> writes:
> But now I have a BIG question: the website in question is 1Gb amb a few
> thousands of files. I'm not sure how should I manage it (and not, I cannot
> split it, sorry). Should I build cached revs automatically (using hook)?  
> Should I add libraries? I need speed when commiting changes and I don't
> want the backups to take forever, so I'0m not sure waht to do.
> 
> Changes are usually small: create a new dir, add a few config files, a few
> images and that is it. Another type of change is patching a few general
> files to modify the website behaviour and look and feel. As you can see, 
> not many changes.

Here's my understanding:

   Cached revs:

      A cached rev is in the archive (and thus on the `server' for
      remote archives), and tells arch to save a snapshot of the whole
      source tree at that revision.  This means any following revisions
      can use it as a base instead of applying earlier patches to
      construct it.  They are mainly useful if you've got _lots_ of
      changesets (hundreds and hundreds), or if for some reason you
      don't want to rely on earlier changesets (for instance, if they're
      in a different archive).

      If you've got a huge source tree, and not many changes, I think you
      probably don't want to make any cached revs yet.

      They are stored as a compressed tar file of the whole source tree,
      and so are fairly space efficient for a particular cached
      revision, but cached revisions don't share storage, and so you
      don't want _too_ many of them.

   Libraries:

      A library is stored on the client, and is similarly a cached copy
      of a particularly revision (though the details are somewhat
      different).

      It can be useful if the archive is remote, and you want to save
      the bandwidth of fetching a big source tree (or even fetching
      individual changesets, when your bandwidth is _very_ low), or make
      it easier to operate disconnected.

      Libraries are stored as a simple unpacked copy of the source tree,
      so take exactly as much space as it does -- however, libraries for
      related revisions (e.g. patch-1, patch-2, &c) try to share storage
      for identical files, which makes it cheaper to keep multiple of
      them around.

      HOWEVER, when tla creates a library for a particular revision,
      currently it automatically goes and creates a library for _each_
      revision preceding it.  They all share disk space via hard-links,
      so it's not quite as bad as it seems, but if you've got hundreds
      of revisions, all those trees can take up a lot of space (I've had
      to go back and manually delete all the earlier revisions in a
      particularly long sequence to get all my disk space back).

   Pristine trees:

      A pristine tree is stored underneath the {arch} directory in
      individual project trees.  tla currently tends to create rather
      too many pristine trees, and doesn't automatically delete them,
      which can also eat up all your disk space if you're not careful
      (pristine trees _don't_ seem to share storage using hard-links,
      either between each other or with libraries or the source tree).

      I think you can avoid making pristine trees by using the
      --no-pristines option to `tla get', but I'm not exactly sure how it
      deals with operations that need pristine trees (like `tla what-changed'
      or `tla commit'); I guess it must make a temporary pristine tree,
      which could be pretty expensive for a big tree.

My experience with tla is mostly with emacs, which is `medium sized' --
about 100MB for the source tree -- and I find the most annoying things
are:

  * Pristine trees using up all my disk space.

  * Massive disk thrashing when doing a commit (at which time tla
    basically walks the whole source tree doing a `diff' to see what's
    changed).  I've got 512MB of RAM in my machine, and linux does a
    fairly good job of disk caching, but apparently even emacs' 100MB
    source tree is just too much.

If your changes are small, it doesn't seem like either cached revs or
libraries will help you that much (until you have many, many,
revisions); maybe what you really need is tons of RAM so the OS can
keep the whole source tree in the disk cache... :-)

[Note that another important point is the style of `id tagging' you use;
I'm currently using `explicit' tags, which keeps a small file to hold
the id-tag for each source file, but this eats up disk space and seems
generally less good than the `tagline' method, which stores the id tag
inside the source-file itself.  The recently announced Xouvert fork of
XFree86 is apparently using tagline id-tags, and I'm think of switching
myself.]

-Miles
-- 
We are all lying in the gutter, but some of us are looking at the stars.
-Oscar Wilde