Re: Make run in parallel mode with output redirected to a regular file can randomly drop output lines

[Frank Heckenbach]

Eli Zaretskii wrote:

> > From: Frank Heckenbach <address@hidden>
> > 
> > > > void write (int fd, void *data, size_t size)
> > > > {
> > > >   if (getflags (fd) & O_APPEND)
> > > >     {
> > > >       lock_mutex (get_mutex (fd));
> > > >       off_t pos = get_size (fd);
> > > >       do_write (fd, pos, data, size);
> > > >       set_pos (fd, pos + size);
> > > >       unlock_mutex (get_mutex (fd));
> > > >     }
> > > >   else
> > > >     {
> > > >       // no mutex here!
> > > >       off_t pos = get_pos (fd);
> > > >       do_write (fd, pos, data, size);
> > > >       set_pos (fd, pos + size);
> > > >     }
> > > > }
> > > 
> > > If the 'else' clause uses a single file pointer system-wise, there's
> > > no overwriting because the pointer is not moved between writes.
> > 
> > I still can't follow you. Just imagine this function is run by two
> > different processes simultaneously with the same FD without
> > O_APPEND. Both fetch the current position (get_pos) and get the same
> > value. Then both write (do_write) at this same position, overwriting
> > each other. Finally, both update the file pointer (set_pos), but
> > again, only the 2nd one becomes effective.
> 
> There's no reason for them to call get_pos.  do_write moves the
> pointer as a side effect.

Not sure what we're arguing here. I'm discussing a hypothetical
implementation of a POSIX-conformant system. In my hypothetical
scenario, do_write does not move the position. Sure, there are
alternatives, I never denied that.

And even if it did, how is do_write implemented? At some point, it
must retrieve the current position, write there and update the
position. If this isn't protected, the problem exists.

Maybe this goes back to what we discussed a few weeks ago WRT the
implementation of seek. It's not like seek (explicit or implicit
during writes) moves the hard disk heads which just sit there
waiting for the next write. It updates the current file position
which is just a variable in (kernel space) memory, and write uses
this variable (among other things) to decide where to place the
data. Like all shared variables, unsynchronized concurrent access
can cause problems.