Re: [Bug-wget] [PATCH] Keep fetched URLs in POSIX extended attributes

[Ander Juaristi]

Also, shouldn't we print something when xattrs are not supported? In
Linux fsetxattr() returns -1 when that is the case.

<pedantic_comment>
Extended attributes are not supported by all filesystems, eg:

dd if=/dev/zero bs=64M count=1 of=./minix.img
mkfs.minix ./minix.img
mount -o loop -t minix minix.img /tmp/minix-test

And then:
wget --xattr -O /tmp/minix-test/archive.html http://archive.org
getfattr -d /tmp/minix-test/archive.html
<empty>

Yeah, Minix is not used in the real world, but NTFS is (sometimes in the
form of portable USB drives) and I think it also does not support
extended attributes. Didn't try it - more hassle to set up in 5 minutes.

If we trust Wikipedia [1], other filesystems from the MS world (such as
FAT) do not support xattrs either.
</pedantic_comment>

[1] https://en.wikipedia.org/wiki/Extended_file_attributes#Linux

On 21/07/16 06:33, Sean Burford wrote:
> Hi,
> 
> I find it useful to keep track of where files are downloaded from.  POSIX
> extended attributes provide a lightweight portable method of keeping this
> information across Linux, OS/X, FreeBSD and many other platforms.
> 
> This compliments wget's existing WARC support, which serves a related but
> different use case closer to tcpdump or tar for web pages.  Extended
> attributes can provide a quick answer to "where did I get this file from
> again?"
> 
> This patch changes:
> *   autoconf detects whether extended attributes are available and enables
> the code if they are.
> *   The new flags --xattr and --no-xattr control whether xattr is enabled.
> *   The new command "xattr = (on|off)" can be used in ~/.wgetrc or
> /etc/wgetrc
> *   The original and redirected URLs are recorded as shown below.
> *   This works for both single fetches and recursive mode.
> 
> Here is an example, where http://archive.org redirects to
> https://archive.org:
> $ wget --xattr http://archive.org
> ...
> $ getfattr -d index.html
> user.xdg.origin.url="https://archive.org/";
> user.xdg.referrer.url="http://archive.org/";
> 
> These attributes were chosen based on those stored by Google Chrome (
> https://bugs.chromium.org/p/chromium/issues/detail?id=45903) and curl (
> https://github.com/curl/curl/blob/master/src/tool_xattr.c)
>

signature.asc
Description: OpenPGP digital signature