Re: Re: [Gnu-arch-users] arch with a small PATH_MAX [WAS: much longer subject :)]

[Ron Parker]

On Wed, 14 Jul 2004 18:38:01 -0500, John Meinel <address@hidden> wrote:

> As mentioned, since the current longest FQFN for an archive is 233
> characters, we may be okay for directly sharable files. But simply doing
> the c/b/v/r fix would change this dramatically to not really worrying
> about it anymore. However, this is a change to the archive, and should
> be handled with care.

Actually that was the relative path to the file.  According to James' email:

> Nah. I don't mind. The pwd is /var/www/mirrors/pub, which would make the
> longest fully qualified filename about 251 charactesr long.

There has to be some overhead for the server itself.

> I am concerned with the ++revision-lock/+contents directory. Because
> that starts with a long path, and I'm not sure what form it takes as it
> is renamed into the new patch-nn directory.

I may have to look into this.

> | **  Category-Branch-Version-Revision Reduction
> |
> | ***  Cons
> |
> | It breaks upward compatibility if implemented universally.
> 
> True, this may be the only thing that really kills it.

The key being, if done universally.  If done locally only on
POSIX-minimal systems where paths are known to be too long, it could
be a help.

> As mentioned elsewhere, vi directory viewing shows the full path, and
> many people show the full path on the command line. So I feel this is a
> small, though not null, issue. Also, how often do you go into these
> directories. For Tom, probably quite a bit. Most other people stay away
> from it, though.

Don't discard the whole grove (there's not quite an entire forest
here), because of one tree. Vi may not be the premium example, but
when debugging tla I have found it convenient to have the non-reduced
form for the directories name.

> | *  An Alternative Solution
> |
> | Category-Branch-Version-Revision reduction could be used for
> | strictly-local files without affecting the external arch
> | community.  While, this still could theoretically exceed the
> | POSIX-minimal PATH_MAX, it lessens the likelihood of it
> | happening.
> 
> Agreed, though I would put the ++pristine-trees into strictly local
> instead of indirectly shared.

I could see pristines in either group. At the very least I know the
official tla 1.2 distribution includes them.  I do not know if this
was out of convenience or for deeper reasons.

Originally it irritated me, because of the bloat to the distribution. 
But in light of the recent "fork" and "submit" discussions I could see
a possible use for them.

> | This would only be needed on POSIX-minimal systems.  Instead of
> | patching and distributing modified versions of tar, diffutils and
> | patch, tla could implement filters to their input or output.  I
> | know its ugly, but custom GNU utils might just be uglier.
> 
> Because fork() is an expensive call on Windows, one desire is to move
> all of the tar/diff/patch utilities into libraries and keep it internal
> in tla. (gzip already has the very nice zlib library.)

I have seen this expressed by some.  I see it as orthogonal to solving
the PATH_MAX limit, but possibly as part of creating a pure Win32 API
version of tla.  Given that the number of forks tla performs is
relatively small I suspect that reducing the number of fork() calls
would yield little improvement.

>From the tests I have run, and from comparing the relative speed of
tla, CVS and SourceSafe I suspect that the primary speed issue on
Windows is the file-system, not fork. SourceSafe is monolithic and
runs within a single process.  It takes roughly the same amount of
time to do a checkout on SourceSafe as it does on CVS and I suspect
tla as well.  Let's just say that the CPU is never pegged but the
drive thrashes. I have not yet been able to test tla against the
project where I work, due to spaces in file names and other issues. 
(Yes, I know about cehteh's escaping support.)

The best way to speed up SCM on Windows seems to be having your
archive on a separate machine across a fast network link or on a
separate local drive.

I understand Johanne's desire to librify the utilities for a pure
Win32 solution. But it does not matter for the sake of getting a fully
functional tla on Windows (or any other) POSIX-minimal system.

Also a pure Win32 solution may or may not be able to support the full
variety of hooks that are available under a POSIX subsystem or
emulation, like SFU or Cygwin.  Please understand I am not trying to
disparage Johanne's work or the idea of a pure Win32 port. I can see
the value in have both a Win32 API version of tla and a Cygwin port. 
I have both versions of vim installed on my system for very good
reasons.

> So in essence it changes for redistributing patched utils to
> incorporating them into arch. One of the complaints against tla is that
> it relies on external tools, which could be broken unexpectedly. I don't
> subscribe to that fear (they can be fixed unexpectedly too), but it
> would eliminate it at the same time.

This is the old static versus dynamic library argument in another
form. I think a deeper reason is needed to make this worth doing.  But
it appears others are interested in doing this work.

However, let me say that if tar were librified the on-the-fly
filtering could be done internally to the libtar code and a filter
would not be necessary. IMO, if and until that happens filtering the
input and output is a relatively lightweight task.

> Writing a tar/patch/diff filter seems ugly enough, coupled with the
> desire for libtar, to consider it a bad route to go.

It seems simple enough to me, a couple of pipes, some file block
processing or some basic pattern matching and it is done.

> The idea of a new "proxy-http" protocol sounds quite good. It isn't
> strictly upward compatible, but someone wanting to connect to an IIS
> served archive doesn't have an option right now.

If this is a reference to my pm-http comments, then proxy in this
context could be very misleading.  The http protocol and libneon
already support regular proxying.

I see the real value of a new protocol in allowing newer versions of
tla to directly support POSIX-minimal servers without the need for an
extra proxy. It doesn't even have to have a separate protocol, it
could just be handled by the simple addition of a file to the archive,
as mentioned in my original post.