[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug 1863333] Re: Assigning NVMe disk to a domain causes VFIO_MAP_DMA er

From: Alex Williamson
Subject: [Bug 1863333] Re: Assigning NVMe disk to a domain causes VFIO_MAP_DMA errors
Date: Thu, 20 Feb 2020 23:39:53 -0000

This seems to be due to the vfio-helper code assuming it can map an
arbitrarily large IOVA range starting at 64K base address.  x86
processors typically have a reserved range near the top of the 32-bit
address space which is used for MSI support which is used by the
interrupt remapper where we cannot create an overlapping DMA mapping
window.  Therefore once you have something approaching a 4G VM, you'll
see the initial -EINVAL and I assume things fall apart from there.
Changing the base IOVA address in vfio-helpers.c seems to be sufficient,

#define QEMU_VFIO_IOVA_MIN 0x100000000ULL

This might be a sufficient legacy fix, but we do now expose valid IOVA
ranges through the VFIO API which would allow this driver to dynamically
pick IOVA ranges based on what the platform actually supports.

You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

  Assigning NVMe disk to a domain causes VFIO_MAP_DMA errors

Status in QEMU:

Bug description:
  I'm seeing some errors when assigning my NVMe disk to qemu. This is
  the full command line:

  /home/zippy/work/qemu/qemu.git/x86_64-softmmu/qemu-system-x86_64 \
  -name guest=fedora,debug-threads=on \
  -S \
  -machine pc-i440fx-4.1,accel=kvm,usb=off,dump-guest-core=off \
  -cpu host \
  -m size=4194304k,slots=16,maxmem=1099511627776k \
  -overcommit mem-lock=off \
  -smp 4,sockets=1,dies=1,cores=2,threads=2 \
  -object iothread,id=iothread1 \
  -object iothread,id=iothread2 \
  -object iothread,id=iothread3 \
  -object iothread,id=iothread4 \
  -mem-prealloc \
  -mem-path /hugepages2M/libvirt/qemu/2-fedora \
  -numa node,nodeid=0,cpus=0,mem=4096 \
  -uuid 63840878-0deb-4095-97e6-fc444d9bc9fa \
  -no-user-config \
  -nodefaults \
  -chardev socket,id=charmonitor,fd=31,server,nowait \
  -mon chardev=charmonitor,id=monitor,mode=control \
  -rtc base=utc \
  -no-shutdown \
  -global PIIX4_PM.disable_s3=0 \
  -global PIIX4_PM.disable_s4=0 \
  -boot menu=on,strict=on \
  -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
  -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 \
  -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
  -netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=34 \
  -chardev pty,id=charserial0 \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -chardev socket,id=charchannel0,fd=35,server,nowait \
  -spice port=5900,addr=,disable-ticketing,seamless-migration=on \
  -device virtio-vga,id=video0,virgl=on,max_outputs=1,bus=pci.0,addr=0x2 \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 \
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
  -msg timestamp=on

  And these are the errors I see:

  2020-02-14T09:06:18.183167Z qemu-system-x86_64: VFIO_MAP_DMA failed: Invalid 
  2020-02-14T09:10:49.753767Z qemu-system-x86_64: VFIO_MAP_DMA failed: Cannot 
allocate memory
  2020-02-14T09:11:04.530344Z qemu-system-x86_64: VFIO_MAP_DMA failed: No space 
left on device
  2020-02-14T09:11:04.531087Z qemu-system-x86_64: VFIO_MAP_DMA failed: No space 
left on device
  2020-02-14T09:11:04.531230Z qemu-system-x86_64: VFIO_MAP_DMA failed: No space 
left on device

  I'm doing nothing with the disk inside the guest, but:

    # dd if=/dev/vda of=/dev/null status=progress

  (the disk appears as /dev/vda in the guest). Surprisingly, I do not
  see these errors when I use the traditional PCI assignment (-device
  vfio-pci). My versions of kernel and qemu:

  moe ~ # uname -r
  moe ~ # /home/zippy/work/qemu/qemu.git/x86_64-softmmu/qemu-system-x86_64 
  QEMU emulator version 4.2.50 (v4.2.0-1439-g5d6542bea7-dirty)
  Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

To manage notifications about this bug go to:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]