Re: [Nmh-workers] use of 'Unseen-Sequence' with rcvstore (Was: Re: refile handling of corrupt .mh_sequences)

[Paul Fox]

Long ago, on Mon, 20 Jan 2014, Ken wrote:
 > >Does anyone know if the man page is accurate (ie., if it corresponds to
 > >the source code)? 
 > 
 > It is NOT accurate.  We've had file locking in sequences for more than
 > a decade, but the documentation wasn't updated.  David Levine updated
 > the rcvstore man page post-1.5, and it now says:
 > 
 > LOCKING AND −unseen
 >   If you use the “Unseen‐Sequence” profile entry, rcvstore could  try  to
 >   read  and  update  its sequence state while another nmh process is also
 >   trying to do so.  This can cause the sequence state to lose track.   To
 >   avoid  this  just  between asynchronous invocations of rcvstore, do not
 >   use it without an external locking  mechanism,  for  example,  a  proc‐
 >   mailrc(5)  local  lockfile,  if  you  use the “Unseen‐Sequence” profile
 >   entry.

I'm confused now.  Back in July I had a problem because I had some
"rcvstore -sequence foo" commands being run almost simultaneously from
cron, and the "foo" entry in .mh_sequences didn't always end up
containing all of the messages that had been rcvstore'd.  I realized
it must be a locking issue, so I implemented some external locking in
my scripts, and the problem went away.

Later, I saw the above man page paragraph.  Since it didn't cover my
case (i.e., a seqeunce other than "unseen"), I updated rcvstore(1)
to generalize it to any sequence rcvstore might update with -sequence,
as well.  That version is in git, at present:

  Locking and sequences
    If  you use the “Unseen-Sequence” profile entry, or if you use the -se‐
    quence argument, rcvstore might try to update its sequence state  while
    another  nmh  process  is  also trying to do so.  This can cause one of
    those updates to be lost.  To avoid this (just between asynchronous in‐
    vocations  of  rcvstore), always use it with an external locking mecha‐
    nism.  A procmailrc(5) local lockfile would work, or if not using proc‐
    mail, then consider using dotlockfile(1) or flock(1).

I didn't ask the list for review at the time, so I'm doing it now:  is
the above paragraph correct?

At the time, reading just that man page, unsurprisingly I inferred
that there was no locking of .mh_sequences in nmh at all.

But just now, while looking in the mail archives for something else, I
found Ken's 2014 message, quoted above.  If locking _is_ supported,
then I don't understand:  if we have locking implemented, then why is
that paragraph necessary, and why does .mh_sequences not get all of
the updates from simultaneous rcvstore invocations.

paul
=----------------------
paul fox, pgf@foxharp.boston.ma.us (arlington, ma, where it's 70.7 degrees)