Re: [Gnu-arch-users] arch with a small PATH_MAX [WAS: much longer subject :)]

[John Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I think you wrote a good summary, and good discussion. I think there are
a couple topics to discuss, so we might think about creating multiple
threads here.

I think your "Terms / Assumptions/ Goals" are all accurate.

I would argue that you don't want to include a ++pristine-trees in a
shared tarball of a working directory, though it takes work to pull it
out. (revlibs do this automatically). If you consider ++pristine-trees
as strictly local, I think it gives you a little more working room.

Parker, Ron wrote:

|
| It is 264-characters long and would nearly fit on a POSIX-minimal
| system.
|
| The longest FQFN I have in a ++saved... directory is:
|
|       /home/rdparker/tla-1.3/++saved.src.1089815347.15624.1/\
|       {arch}/++pristine-trees/unlocked/package-framework/\
|       package-framework--devo/package-framework--devo--1.0/\
|       address@hidden/\
|       package-framework--devo--1.0--patch-5/{arch}/\
|       package-framework/package-framework--patch-detection/\
|       package-framework--patch-detection--1.0/\
|       address@hidden/patch-log/patch-2
|
| It contains 358 characters and exceeds the POSIX minimum by ~40%.
| Anecdotally in playing with arch, I believe I have seen paths in
| excess of 400 characters but never greater than 500.
|
|

| ** Directly Sharable Files
|

As mentioned, since the current longest FQFN for an archive is 233
characters, we may be okay for directly sharable files. But simply doing
the c/b/v/r fix would change this dramatically to not really worrying
about it anymore. However, this is a change to the archive, and should
be handled with care.

I am concerned with the ++revision-lock/+contents directory. Because
that starts with a long path, and I'm not sure what form it takes as it
is renamed into the new patch-nn directory.

| **  Category-Branch-Version-Revision Reduction
|
| ***  Cons
|
| It breaks upward compatibility if implemented universally.

True, this may be the only thing that really kills it.

|
| It does lose semantic information when viewing a directory in
| isolation.  (Think the vi or emacs directory viewers or shell
| prompts with the last fragment of the current-working-directory
| displayed.)  Granted IMO this is primarily an issue for arch
| developers.  But I include myself marginally in that category.
|

As mentioned elsewhere, vi directory viewing shows the full path, and
many people show the full path on the command line. So I feel this is a
small, though not null, issue. Also, how often do you go into these
directories. For Tom, probably quite a bit. Most other people stay away
from it, though.

| *  An Alternative Solution
|
| Category-Branch-Version-Revision reduction could be used for
| strictly-local files without affecting the external arch
| community.  While, this still could theoretically exceed the
| POSIX-minimal PATH_MAX, it lessens the likelihood of it
| happening.

Agreed, though I would put the ++pristine-trees into strictly local
instead of indirectly shared.

|
| There are also individual directory names that are fairly long
| compared to the POSIX minimum.  For those things like the
| ,,commit directories:
|
|       .../\
|       
|
,,address@hidden
| .1/\
|       hackerlab--devo--1.0--patch-93.patches/new-files-archive/\
|       {arch}/hackerlab/hackerlab--devo/hackerlab--devo--1.0/\
|       address@hidden/patch-log/patch-93/
|       
| a simplification could be made reducing that one directory name
| to:
|
|       .../,,commit.hackerlab--1089817914.15755.1/...
|

This is also a very helpful, very local fix, that I feel has a lot of merit.

| which retains the trailing subdirectories and avoids
| completely loosing any semantic meaning.  While having the extra
| information in the ,,commit directory name is nice, it can be
| found within the commit directory itself.  I think that is a
| reasonable compromise on a POSIX-challenged machine.  If the data
| needs to be explicit, it could go in a file just inside the
| ,,commit directory.
|
| <bad-idea?>
| </bad-idea?>

I think so. Relocating directories would add a lot of logic to see if
any dirs have been relocated, and to make sure that if you found them by
accident you would know not to traverse them yet.

|
| This would only be needed on POSIX-minimal systems.  Instead of
| patching and distributing modified versions of tar, diffutils and
| patch, tla could implement filters to their input or output.  I
| know its ugly, but custom GNU utils might just be uglier.

Because fork() is an expensive call on Windows, one desire is to move
all of the tar/diff/patch utilities into libraries and keep it internal
in tla. (gzip already has the very nice zlib library.)

So in essence it changes for redistributing patched utils to
incorporating them into arch. One of the complaints against tla is that
it relies on external tools, which could be broken unexpectedly. I don't
subscribe to that fear (they can be fixed unexpectedly too), but it
would eliminate it at the same time.

Writing a tar/patch/diff filter seems ugly enough, coupled with the
desire for libtar, to consider it a bad route to go.

|
| *  Directly-Shared files
|
| As his longest at a total 251 characters.  Ouch, that is getting
| close to the POSIX minimum limit, so there may be a need for a
| solution.  The big problem is how to do this in a upwardly
| compatible manner.  If you don't want to change the protocol, and
| your server can't support the current protocol, what do you do?
| How about a helper program?  A little proxy that could be run to
| access POSIX-minimal archives in a normal manner?  Say a proxy
| that would convert requests for
|
|       http://www.inthefaith.net/rdp/{archives}/\
|       address@hidden/package-framework/\
|       package-framework--devo/package-framework--devo--1.0/\
|       patch-1/\
|       package-framework--devo--1.0--patch-1.patches.tar.gz
|
| into:
|
|       http://www.inthefaith.net/rdp/{archives}/\
|       address@hidden/package-framework/devo/1.0/\
|       patch-1/patches.tar.gz
|
| The POSIX-minimal systems themselves could either use the proxy
| as well, or access the archive directly by archive-register'ing
| it as pm-http://www.inthefaith.net/rdp/{archives}.....
| Optionally a POSIX-minimal server could be indicated by a
| =meta-info/windows-blows, with the simple text "yes it does".  Of
| course if the mere existence of =meta-info indicates a mirror, a
| =short-paths file could be used with the simple text of "windows
| blows".

The idea of a new "proxy-http" protocol sounds quite good. It isn't
strictly upward compatible, but someone wanting to connect to an IIS
served archive doesn't have an option right now.

| Okay, I spent entirely too long working on this but I hope it
| generates some discussion and presents some alternative solutions
| that may have not been considered before.  I feel there are
| things here that would play nicely with any of the existing ideas
| out there.  I only focused on the CBV reduction because it
| retains the most semantic information.  Do your worst and let me
| know what you all think.
|

Overall, I think you did a good job, and I think CBV reduction is the
"best" version. At least it is the simplest one that retains the most
info, and would have good performance characteristics.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFA9cPZJdeBCYSNAAMRAqfuAJ4vcu7y2WSGeEL5G/ly/kTZpANXgACg2JrR
45NqAGjPCGW/eTwEAxJXEGI=
=WolM
-----END PGP SIGNATURE-----