Re: [Gnu-arch-users] Re: arch with 'special files'

[Josh England]

John Arbash Meinel wrote:
> Josh England wrote:
> ...
> 
>> I agree with this, and what you said in your last post.  Asking tla to
>> handle translating different metadata types to their functional
>> equivalents on every platform is just asking for trouble.  I think arch
>> should transparently handle *very few* metadata types implicitly: file
>> permissions, optional file ownership, and optional CR/LF issues.  All of
>> these translate well enough between platforms.
>>
>>
>>
> I agree.
> 
>> For other metadata, consider again the single generic metadata string.
>> This string could be arbitrarily long. tla would only need to diff it to
>> tell that the metadata for any given file has changed.
>> The string itself could be anything.  For example, a file called
>> /tmp/foo could have a metadata string like this:
>>
>> "clrf=unix,xattr::joe=bob,win32=ignore"
>>
>>
>>
> If you make it a '\n' delimited string, then you could use standard diff
> techniques for delta compression.

Since this is user-level metadata, tla cannot be expected to know how to
make a meaningful delta for it.  All tla really needs to do is have a
flag for each file (that has attached metadata) indicating whether or
not the metadata has changed.  It is completely up to the user to make
sense of *how* the metadata has changed and what to do about it.

>> or this:
>> "<?xml version="1.0"?> <file_attr> joe <value> bob </value></file_attr>"
>>
>> or even this (which is probably a bad idea):
>> "#!/bin/sh chown 0:0 ${ARCH_FILE}"
>>
>> The point is, you could do ANYTHING with that single string, and tla's
>> job would be very easy: tell you if the string has changed.  This is SVN
>> properties on steroids.
>>
>>
>>
> Well, specifically, how do you "tell if the string has changed"?
> Obviously this metadata would need to be stored somewhere. And tla would
> need a "tla set-metadata" and "tla get-metadata". tla could revision
> control the metadata.
> 
> But I think what you are asking is such that when you do "tla get" at
> each step of the get, it might invoke some sort of user defined hook
> whenever metadata changes.
> I agree, a single string is probably sufficient. Then any layers above
> tla can do what they need to.

No, 'tla get' should not call a hook for every file.  That would be bad.

> But what you really want is like a hook such as "metadata-changed" which
> is called after a get, with a list of all the files whose metadata has
> changed.

Yes.

> Naturally a "dumb" wrapper could just crawl the tree and use "tla
> get-metadata" for each file, however this is really inefficient, and
> doesn't scale to large trees very well.
> 
> You could have the command "tla get-changed-metadata" but then tla needs
> to somehow keep track of what metadata has changed. Which it could do by
> diffing the current metadata against the old metadata (which is a whole
> tree thing), or by looking at the changeset, but oddly enough, when it
> does the apply-delta it already is looking at all of the metadata.

This sounds good.

> Then there is the concern that apply-delta is generally done in another
> directory which is then copied into the working directory (think about
> revlibs, etc). So when does the "metadata-changed" hook occur. In the
> initial apply, before the copy, or after, or do we have one for both?
> Does your revlib have the metadata applied to it, or does it just keep
> track of it, and only your working tree has the metadata applied?

Not being knee-deep in tla code, I'm not sure what the best approach
would be, either.  Since this is user-level metadata, it seems to me the
only time it is meaningful to check it is after a 'tla update'.  This is
when a hook or some wrapper script would call 'tla get-changed-metadata'
for a list of changed files, and then 'tla get-metadata' on each file
and do any necessary actions based on the metadata change.  Having the
'metadata-changed' hook occur as one of the last things done by 'tla
update' seems like a good place for it.

>> Its alright for people to want the software to transparently handle
>> every situation known (and unknown) to man, but it is simply not very
>> feasible.  Software of this nature should really follow the 'Keep It
>> Simple' philosophy, which I think tla has done a great job of so far.
>> Giving tla this kind of generic store/retreive of a simple string would
>> *enable* some very cool things to be done by the user in surrounding
>> wrappers and hooks, while keeping the core tla engine fairly simple.  I
>> could envision any number of people building some very cool mechanisms
>> on top of this kind of feature.
>>
>> -JE
>>
>>
> I agree, but we need a good definition of how this feature can be used
> first.
> 
> Naturally the easiest thing is to just allow "tla get-metadata" "tla
> set-metadata" and not worry about the hooks.
> And this metadata could be handled in the same fashion that tla uses to
> handle the arch ids.
> So you could have a .arch-ids/filename.id wherein you have RFC8422
> formatted information:
> id: aonteuhaeou-taoenuh-aoheunanothu
> perms: x
> line-endings:
> 
> <extra stuff here>
> 
> Then stuff in the Header section would be arch controlled and
> understood, and the free-form is everything else.
> 
> You could even overload taglines. So you would have:
> 
> /* arch-tag: aoauea-aoeuht-aoeuhao
> * arch-perms: ????
> * arch-metainfo: whatever,you want goes here.
> */
> 
> Is this the *right* way to go? I'm not really sure. It seems we should
> put all meta-info together. But most certainly tla should never change
> values inside the file. (other than applying a patch which modifies it.)

This looks good also, except for the inability to have a "\n" in the
arch-metainfo.  Maybe some other wacky delimeter could be used:

/* arch-tag: aoauea-aoeuht-aoeuhao
* arch-perms: ????
* arch-metainfo:
this=my wacky
multiline
metadata
string
-*- end-arch-metainfo -*-
*/


-JE