[Gnu-arch-users] Re: archzoom

[Miles Bader]

address@hidden (Ludovic Courtès) writes:
> I think you already mentioned that, according to you, this inefficiency
> was more an implementation issue rather than a design issue, is that
> correct?

I'm not really talking about one specific inefficiency, just my
impression as a heavy user of tla.  As far as I can figure it's really
the product of many different inefficiencies together, some due to
implementation issues such as not caching enough, not pipelining http
requests, or not doing backward deltas, and some inherent to arch's
design.

There does seem to be a general "don't care" attitude about doing system
calls in tla though; during the heyday of arch, there were many possible
optimizations discussed which could potentially greatly speed up
operation in "common cases", but were never implemented for various
reasons (or were implemented and not merged).

As to the specific example you gave, a few thoughts:

The basic pattern of system calls when doing tla changes, using
taglines, is for each _unchanged_ DIR/FILE in the tree:

   three times (working dir, then twice in revision library):
      lstat64("DIR/FILE")                      = 0
      open("DIR/.arch-ids/FILE.id", O_RDONLY) = -1 ENOENT (No such file or 
directory)
      open("DIR/FILE", O_RDONLY)              = 4
      fstat64(4)                              = 0
      lseek(4, -1026, SEEK_END)               = 529
      read(4"...", 1025)                      = 1025
      close(4)                                = 0

   in revlib:
      lstat64("ROOT/DIR/FILE") = 0

   in working dir:
      lstat64("ROOT/DIR/FILE") = 0

So basically an "establish file tagging" pass (not sure why it does the
revlib twice though), and an "actually compare files" pass.

The "actually compare files" pass is as probably as good as you can get
but the "establish file tagging" passes are not.  I think given the
existance of the ,,inode-sigs files, which are basically "guesses" at
the tree state, one could in many cases replace the all those
open/reads/etc with a single stat64 for most files, only using the
open/read to confirm taglines for files which don't match any "guess".
Then of course you could cache the stat results so the "actually compare
files" pass needn't do any system calls at all.  Then "tla changes"
could approach the speed of a simple diff.

[Only doing stats for most files is a big win in many cases -- in NFS
obviously, but also in normal linux, even with file content caching
easing the pain of the reads over time:  there's always the first time
which must hit the disk, and the memory consumed for caching
file-contents in a large tree is often not trivial at all, leading to
thrashing.]

-Miles
-- 
We live, as we dream -- alone....