Re: [Bug-tar] Re: Auto-exclude of cache directories for tar, cpio

[Bryan Ford]

Hi Joerg,

Sorry for the late response - I wasn't on this list (and wasn't even aware of 
it in fact :)) and thus didn't see your reply at all until Paul mentioned it 
to me just now.

Joerg Schilling writes:
>From: Paul Eggert <address@hidden>
>>Bryan Ford <address@hidden> writes:
>>>      http://www.brynosaurus.com/cachedir/
>
>Star uses since 6 years (as an extension to the *BSD tar implementation
>of the -F option) ".mirror" and ".exclude". The file .mirror is created
>by typical mirroring software....

Excellent - but neither ".mirror" nor ".exclude" has the semantics I'm looking 
for, at least in their obvious intent.  ".exclude" files are basically 
out-of-band commands from a user or system administrator to a backup program, 
specifically instructing the backup program to exclude that directory.  
Normal applications shouldn't know or care about how backups are done, and 
thus shouldn't normally be giving instructions to backup programs by writing 
".exclude" files automatically.

".mirror" files come closer to my intended semantics, in that they are 
intended to be written automatically by _applications_ (i.e., the mirror 
software) rather than the user or administrator, in order to express 
something about the semantics of the data they wrote into a particular 
directory.  The user/administrator can then give an appropriate flag (e.g., 
-FFF) to ask their backup software to be sensitive to that 
automatically-generated semantic information in the appropriate way.  And 
indeed a mirror is in some sense a cache, in that it consists of content 
that's presumably regenerable by re-downloading it from the source.  But a 
mirror serves a different purpose: it's intended to be used directly _by 
users_, not just invisibly by some application.  If a mirror disappears, its 
local users will stop being able to find stuff where they're used to finding 
it, and they'll have to complain to the sysadmin until it gets restored.  In 
contrast, if an application cache (e.g., a Mozilla web cache) disappears, the 
application just silently re-creates it next time it runs, and users notice 
nothing except perhaps that things take a bit longer next time.  Users know 
and care about mirrors, while caches are supposed to be entirely invisible.  
I don't think Mozilla et al should write ".mirror" files into their cache 
directories, because their cache directories aren't mirrors - they're caches.

I just looked at the man page for star, and I notice that star can not only 
look for .mirror and .exclude files, but also SCCS and RCS directories, .o's, 
and core, errs, and a.out files - these are all different things with 
different semantics, which we often desire to leave out of backups for 
related but not identical reasons.  None of those existing conventions 
exactly matches the purpose I'm addressing with this proposal, namely 
allowing applications to tag their cache directories in a semantically 
appropriate and meaningful way.  But at the same time it doesn't seem like it 
would be too hard to enhance star (or any other archiver that already 
supports such conventions) to be aware of one additional one that _does_, by 
definition, exactly suit the desired purpose.

Thanks,
Bryan