Re: Discard commands not working with virtio-blk despite using recent QEMU version

[John Snow]

On 6/1/20 12:00 PM, David Scherfgen wrote:
> Dear mailing list,
> 
> I'm using QEMU 4.2.0 (coming with Ubuntu 20.04 LTS) to run a virtual
> webserver. It uses a ZFS volume as its disk over the VirtIO bus
> (virtio-blk).
> 
> After equipping my server with SSDs and moving my virtual
> webserver's disk to them (actually a ZFS volume in a mirror
> configuration), I was surprised to find out that discard commands didn't
> work. When running fstrim on the guest, I got:
> 
> fstrim: /: FITRIM ioctl failed: Operation not supported
> 
> This surprised me because I read about virtio-blk being adapted so that
> it can handle discard commands. I checked the source code for QEMU 4.2.0
> and indeed the changes to virtio-blk to support discard were in there.
> 
> Switching to the SCSI bus solved the problem. But nevertheless,
> shouldn't it also be working with virtio-blk now?
> 
> Further below, I'm posting the QEMU command lines.
> 
> Thanks in advance for any ideas on this.
> 
> Best regards
> David Scherfgen
> 
> ----------
> 
> QEMU command lines:
> 
> With the VirtIO bus (discard not working):
> 
> qemu-system-x86_64 -enable-kvm -name guest=webserver,debug-threads=on -S
> -object
> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-30-webserver/master-key.aes
> -machine pc-i440fx-4.2,accel=kvm,usb=off,dump-guest-core=off -cpu qemu64
> -m 49152 -overcommit mem-lock=off -smp 8,sockets=1,cores=4,threads=2
> -uuid 7d619885-9c53-2fa3-ca69-032e6fba6a90 -no-user-config -nodefaults
> -chardev socket,id=charmonitor,fd=32,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
> -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
> -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x5 -blockdev
> {"driver":"host_device","filename":"/dev/zvol/tank-ssd/webserver","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}
> -blockdev
> {"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on
> -netdev tap,fd=34,id=hostnet0,vhost=on,vhostfd=35 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:50:56:00:f2:8e,bus=pci.0,addr=0x3
> -vnc 0.0.0.0:10000 <http://0.0.0.0:10000>,password -device
> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -cpu host -k de
> -sandbox
> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
> -msg timestamp=on
> 
> With the SCSI (discard working):
> 
> qemu-system-x86_64 -enable-kvm -name guest=webserver,debug-threads=on -S
> -object
> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-31-webserver/master-key.aes
> -machine pc-i440fx-4.2,accel=kvm,usb=off,dump-guest-core=off -cpu qemu64
> -m 49152 -overcommit mem-lock=off -smp 8,sockets=1,cores=4,threads=2
> -uuid 7d619885-9c53-2fa3-ca69-032e6fba6a90 -no-user-config -nodefaults
> -chardev socket,id=charmonitor,fd=32,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
> -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
> -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x5 -blockdev
> {"driver":"host_device","filename":"/dev/zvol/tank-ssd/webserver","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}
> -blockdev
> {"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}
> -device
> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1,write-cache=on
> -netdev tap,fd=34,id=hostnet0,vhost=on,vhostfd=35 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:50:56:00:f2:8e,bus=pci.0,addr=0x3
> -vnc 0.0.0.0:10000 <http://0.0.0.0:10000>,password -device
> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -cpu host -k de
> -sandbox
> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
> -msg timestamp=on

Hi, I didn't follow personally the development of adding TRIM to
virtio-blk, so below are some naive questions:

- What kernel does the guest run? I think the kernel needs to support it
for that block driver too.

- The filesystem driver in the guest needs to support it as well. (What
FS are you using in the guest? Is that ZFS? Not all implementations
support TRIM as far as I know, depends on version and so on.)

- does discard need to be enabled for the device model as well as the
block backend? (-device virtio-blk-pci,discard=on) ... The patch makes
it look like it should be enabled by default, but it's something to try.