bug#12366: [gnu-prog-discuss] Writing unwritable files

[Paolo Bonzini]

[For bug-coreutils: the context here is that sed -i, just like perl -i,
breaks hard links and thus destroys the content of files with 0400
permission].

Il 06/09/2012 12:38, John Darrington ha scritto:
>      That's expected of programs that break hard links.
> 
> I wonder how many users who are not hackers expect it?  I suspect most 
> would not.
> 
> Why is it not possible or not desirable to check the mode of the file
> before renaming?

Because there are valid use cases in which you _want_ to break hard links.

Also because it would be twice as slow and require twice as much disk
space.  The choices would be:

1) copy the input file to a temporary file, then write to the original
file while processing the copy; required space = input_size +
max(input_size, output_size)

2) write to a temporary file while processing the original, then copy it
over to the input; required space = output_size + max(input_size,
output_size).

3) same as (1) or (2) with memory replacing a temporary file.  Goes
totally against the idea of sed as a _stream_ editor.

>      Here is what the sed manual thinks.
> 
> Thanks for pointing that out (it must be a recent addition; my installed
> version doesn't have this text).

Actually it is older than the version control repository, so 2004 or
older.  But it is under "Reporting bugs", not under the -i option
(patches welcome ;)).

You'll also find it under /usr/share/doc/sed-4.2.1/BUGS.

>      `-i' clobbers read-only files
>           In short, `sed -i' will let you delete the contents of a read-only
>           file, and in general the `-i' option (*note Invocation: Invoking
>           sed.) lets you clobber protected files.  This is not a bug, but
>           rather a consequence of how the Unix filesystem works.
>      
>           The permissions on a file say what can happen to the data in that
>           file, while the permissions on a directory say what can happen to
>           the list of files in that directory.  `sed -i' will not ever open
>           for writing  a file that is already on disk.  Rather, it will work
>           on a temporary file that is finally renamed to the original name:
>           if you rename or delete files, you're actually modifying the
>           contents of the directory, so the operation depends on the
>           permissions of the directory, not of the file.  For this same
>           reason, `sed' does not let you use `-i' on a writeable file in a
>           read-only directory, and will break hard or symbolic links when
>           `-i' is used on such a file.
> 
> I don't think that this addresses the issue at hand.  The program does 
> something
> non-intuitive that can lead to loss of data, and it wouldn't be reasonable to 
> blame 
> the user in that instance.

I agree, but it's not me who designed the Unix filesystem permissions.

> (Some) other GNU programs don't behave like this.  For 
> example "truncate foo" on a readonly foo will exit with an error

Truncate does not process foo for input at the same time, so it isn't't
really relevant.

> , as will  "dd if=foo of=foo count=10".

dd has a well-defined behavior for overlapping input and output, and
this well-defined behavior in fact mandates that dd doesn't break hard
links.

> Likewise, "shuf foo -o foo".

I consider "shuf foo -o foo" (on a read-write file) to be insecure.
Besides, it works by chance just because it reads everything in memory
first.  If it used mmap to process the input file, "shuf foo -o foo"
would be broken, and the only way to fix it would be to do the same as
"sed -i".

shuf could in fact introduce a "shuf -i" mode that would be consistent
with the way "sed -i" works, including the ability to create a backup
file _and_ the breaking of hard links.

Paolo