[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#32228: sed -i does not buffer, causing excessive writes

From: Vidar Holen
Subject: bug#32228: sed -i does not buffer, causing excessive writes
Date: Fri, 20 Jul 2018 13:23:35 -0700


I'm using noticing that `sed -i` does not do any kind of output
buffering. While behavior is correct, this causes an excessive number of
small writes:

    $ strace -e write ./sed -i 's/foo/bar/g' file.txt
    write(4, "                    GNU GENERAL "..., 47) = 47
    write(4, "                       Version 3"..., 47) = 47
    write(4, "\n", 1)                       = 1

Compare this to redirection:

    $ strace -e write ./sed 's/foo/bar/g' file.txt > tmpfile
    write(1, "                    GNU GENERAL "..., 4096) = 4096
    write(1, "om or adapt all or part of the w"..., 4096) = 4096
    write(1, ".\n\n  You may make, run and propa"..., 4096) = 4096

On my system, this makes `sed -i` take 7x longer than redirection+mv:
3.7s vs 0.5s for 100MB, for 675 vs 10 writes on the same input.

I'm seeing this on Debian with the latest sed commit (c52a676) and libc
2.27-2. The root cause appears to be `ck_mkstemp` using `fdopen`, which
unlike `fopen` does not buffer by default.

Enabling buffering should be simple and effective, resulting in a nice

Vidar Holen

reply via email to

[Prev in Thread] Current Thread [Next in Thread]