Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation of FUA and FLUSH

[Alex Bligh]

On 1 Apr 2016, at 11:10, Wouter Verhelst <address@hidden> wrote:

> On Fri, Apr 01, 2016 at 10:28:03AM +0100, Alex Bligh wrote:
>> 
>> On 1 Apr 2016, at 09:35, Wouter Verhelst <address@hidden> wrote:
>> 
>>>> +* All write commands (that includes both `NBD_CMD_WRITE` and
>>>> +  `NBD_CMD_TRIM`) that the server completes (i.e. replies to)
>>>> +  prior to processing to a `NBD_CMD_FLUSH` MUST be written to non-volatile
>>>> +  storage prior to replying to that `NBD_CMD_FLUSH`. The server SHOULD 
>>>> ensure
>>>> +  that all write command received prior to processing the `NBD_CMD_FLUSH`
>>>> +  (whether they are replied to or not) are written to non-volatile
>>>> +  storage prior to processing an `NBD_CMD_FLUSH`; note this is a
>>>> +  stronger condition than the previous 'MUST' condition. This
>>> 
>>> This seems to make little sense. Are you saying that suddenly now
>>> sending a reply for FLUSH with outstanding writes is wrong? If not, the
>>> above should be clarified.
>> 
>> The MUST sentence does not cover that situation as it only refers
>> to completed writes.
>> 
>> The SHOULD sentence says that's a 'SHOULD NOT' situation in respect
>> of writes that have PROCESSED (i.e actioned) whether or not they
>> have been replied to. Of course the client has no way of knowing
>> whether they have been PROCESSED without a reply.
>> 
>> Personally I think the SHOULD clause is pretty pointless and is
>> unnecessary, but that's where the conversation got to n years
>> ago I believe.
> 
> I'm still not sure what it's supposed to mean, though. Clearly, you
> should at the very least reword it, if not...
> 
>> Happy to delete the last sentence if that's wrong.
> 
> ... remove it instead.

If I can't even explain it, it doesn't bode well!

I think there are three types of writes that are relevant at the point
of replying to a FLUSH:

Type A: writes that are truly 'in flight', i.e. have been sent, but have
not been 'processed' (i.e. write(2) has not been called).

Type B: writes that are have been processed (i.e. write(2) has not been
called) but the reply has not yet been sent to the client - either the
reply hasn't been made yet or (more likely) it's sitting in some queue.

Type C: writes that have been processed and a reply sent.


The first sentence (the 'MUST') refers to Type C writes and says these
MUST be persisted to permanent storage prior to replying to the FLUSH.

Type A writes may be in a buffer server side or even client side, so
no one expects anything to happen with respect to those.

The 'SHOULD' bit concerns Type B writes. It's saying that if you've
actually done the write call (but not yet replied) you SHOULD persist
these to disk prior to replying to the FLUSH. This type of situation
doesn't (last time I looked) happen in nbd-server.c because it does
not process requests in parallel. However it may happen elsewhere.
The question is what to do with them.

I'm happy dropping anything about these writes and only dealing with
Type C writes, but the 'SHOULD' is where I thought we got to before
(I think I sent the quote from the mailing list).


-- 
Alex Bligh