Re: Buffered I/O and ed shell escape command

[Alexander Jones]

I wouldn't put too much stock into comparing the GNU version of ed with
OpenBSD's, since they both ultimately derive from the same implementation
(written in the early-to-mid 1990s by Andrew Moore) and thus could share
long-standing bugs. That being said, I concur with your reading of the
POSIX spec that the modified command string should be printed before the
command is executed, so this is likely a bug that should be fixed in GNU
and reported to the various BSDs.

On Tue, Dec 28, 2021 at 4:33 PM Sören Tempel <soeren@soeren-tempel.net>
wrote:

> Hi,
>
> I did some experiments with the shell escape command today and noticed
> that the command behavior depends on the output stream on my system:
>
>         $ cat /tmp/ed-cmd
>         !echo foo
>         !!
>         $ ed < /tmp/ed-cmd
>         foo
>         !
>         echo foo
>         foo
>         !
>         $ ed < /tmp/ed-cmd > /tmp/ed-out.txt
>         foo
>         !
>         foo
>         echo foo
>         !
>
> If the standard output is line buffered (i.e. a TTY) everything works as
> expected. However, if standard output is a file, the modified command
> string of the second command (echo foo) is written *after* the command
> was executed. This is IMHO not conforming to POSIX which mandates that
> the modified command string must be written *before* the command is
> executed.
>
> Looking at the source code, the problem seems to be that buffered output
> streams are not flushed before the given command is executed using
> system(3). As such, the executed command may write data (unbuffered) to
> standard output and after the command terminated, data potentially
> buffered by ed is written to it. As such, the output order depends on
> whether standard output is line buffered.
>
> My hypothesis seems to be confirmed by the fact that applying the
> following patch fixes the outlined problem:
>
> diff -upr ed-1.18-pre3.orig/main_loop.c ed-1.18-pre3/main_loop.c
> --- ed-1.18-pre3.orig/main_loop.c       2021-12-28 20:40:49.170833983 +0100
> +++ ed-1.18-pre3/main_loop.c    2021-12-28 20:40:53.594162095 +0100
> @@ -678,6 +678,7 @@ pflabel:      if( !check_addr_range2( ad
>      case '!': if( unexpected_address( addr_cnt ) ) return ERR;
>                fnp = get_shell_command( ibufpp );
>                if( !fnp ) return ERR;
> +              fflush ( NULL ); /* flush any buffered I/O */
>                if( system( fnp + 1 ) < 0 )
>                  { set_error_msg( "Can't create shell process" ); return
> ERR; }
>                if( !scripted() ) fputs( "!\n", stdout );
>
> However, I also tested this with the OpenBSD implementation of ed(1) and
> this implementation also exhibits different behavior depending on
> standard output buffering. I am therefore not sure if I am simply
> misunderstanding the specification of the shell escape command in POSIX
> or if this is indeed a bug.
>
> I would welcome any sort of feedback on my outlined observations.
>
> Greetings,
> Sören
>
>