Re: [ANNOUNCE] libblkio v0.1.0 preview release

[Stefan Hajnoczi]

On Thu, Apr 29, 2021 at 03:22:59PM +0100, Richard W.M. Jones wrote:
> On Thu, Apr 29, 2021 at 03:05:50PM +0100, Stefan Hajnoczi wrote:
> > The purpose of this preview release is to discuss both the API design
> > and general direction of the project. API documentation is available
> > here:
> > 
> >   https://gitlab.com/libblkio/libblkio/-/blob/v0.1.0/docs/blkio.rst
> 
> libvirt originally, and now libnbd, keep a per-thread error message
> (stored in thread-local storage).  It's a lot nicer than having to
> pass &errmsg to every function.  You can just write:
> 
>  if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
>    fprintf (stderr,
>             "failed to connect to remote server: %s (errno = %d)\n",
>             nbd_get_error (), nbd_get_errno ());
>    exit (EXIT_FAILURE);
>  }
> 
> (https://libguestfs.org/libnbd.3.html#ERROR-HANDLING)
> 
> It means you can extend the range of error information available in
> future.  Also you can return a 'const char *' and the application
> doesn't have to worry about lifetimes, at least in the common case.

Thanks for sharing the idea, I think it would work well for libblkio
too.

Do you ignore the dlclose(3) memory leak?

> > Examples are available here:
> > 
> >   https://gitlab.com/libblkio/libblkio/-/tree/v0.1.0/examples
> > 
> > The goal is to eventually include the following drivers:
> > - Linux io_uring
> > - NVMe (VFIO and vfio-user)
> > - virtio-blk (VFIO, vfio-user, vhost-user-blk, and vhost-vdpa-blk)
> >
> > There are a few reasons why libblkio is needed:
> > 
> > 1. QEMU, Ceph, GlusterFS, MariaDB, and other programs have been adding
> >    more low-level block I/O code, most of it duplicated. Usually only
> >    one or two of Linux AIO, io_uring, userspace drivers, vhost-user
> >    drivers, etc are implemented. This makes it difficult to benefit from
> >    the latest advances in high-performance block I/O.
> > 
> > 2. Coding to a standard API makes it possible to introduce new
> >    optimizations or hardware interfaces without costly changes to the
> >    software stack.
> > 
> > 3. A client library is needed so applications can take advantage of
> >    qemu-storage-daemon's vhost-user-blk exports.
> > 
> > 4. Implementing block I/O as a library allows QEMU to use Rust for new
> >    code without messy QEMU internal API bindings. Note that libblkio
> >    currently does not provide a Rust crate, it only offers a C API.
> 
> This is where I get confused about what this library actually does.
> It's not just a nicer wrapper around io_uring, but what is it actually
> doing?

It's a library for what QEMU calls protocol drivers (POSIX files,
userspace NVMe driver, etc). In particular, anything offering
multi-queue block I/O fits into libblkio.

It is not intended to replace libnbd or other network storage libraries.
libblkio's properties API is synchronous to keep things simple for
applications. Attaching to network storage needs to be asynchronous,
although the libblkio API could be extended if people want to support
network storage.

Stefan

signature.asc
Description: PGP signature