Re: [Qemu-devel] O_DIRECT on glusterfs (was Re: [PATCH v9 4/4] block: Support GlusterFS as a QEMU block backend)

[Anand Avati]

On 10/03/2012 08:50 AM, Paolo Bonzini wrote:
> Il 24/09/2012 11:13, Bharata B Rao ha scritto:
> > +
> > +    if ((bdrv_flags&  BDRV_O_NOCACHE)) {
> > +        open_flags |= O_DIRECT;
> > +    }
> > +
>
> If I understand correctly what I was told, this prevents the brick
> server from using its own buffer cache.  This is quite different from
> what we do for example over NFS (where the host does no caching, but
> nothing prevents it on the remote server).
>
> I think these 3 lines should be removed.  We're bypassing the host
> buffer cache just by virtue of using a userspace driver, and that's what
> cache=none cares about.
>
> Paolo

O_DIRECT also has an effect on the behavior of the "client side" (the 
part within the qemu) of Gluster stack as well. I presume the intention 
of O_DIRECT is to minimize use of memory (whether as host' page cache or 
buffered data in user space). To that end it is a good idea to leave 
O_DIRECT flag set.

The behavior of whether gluster bricks need to get the O_DIRECT 
propagated or not is a different issue. We are exploring the possibility 
of not sending O_DIRECT flag over the wire to mimic NFS behavior. That 
would be independent of the qemu block driver setting the open flag.

Avati