0

I'm trying to boot a QEMU VM from a vhost-user-blk-pci device, which appears to be generally possible (https://github.com/spdk/spdk/issues/1728). In my case, vhost gets the image via SPDK's NVMe-oF driver. However, QEMU does not find a bootable device. What I am doing:

  1. Start vhost

     bin/vhost -S /var/tmp -s 1024 -m 0x3 -A 0000:82:00.1
    
  2. Connect to NVMe-oF server and create blk controller

     ./rpc.py bdev_nvme_attach_controller -t tcp -a 10.0.0.4 -s 4420 -f ipv4 -n nqn.2024-10.placeholder:bd --name placeholder
    
     ./rpc.py vhost_create_blk_controller --cpumask 0x1 vhost.0 placeholdern1
    
  3. Attempt to launch QEMU with blk controller as boot device (does not find anything bootable)

     taskset -c 2,3 qemu-system-x86_64 \
         -enable-kvm \
         -m 1G \
         -smp 8 \
         -nographic \
         -object memory-backend-file,id=mem0,size=1G,mem-path=/dev/hugepages,share=on \
         -numa node,memdev=mem0 \
         -chardev socket,id=spdk_vhost_blk0,path=/var/tmp/vhost.0,reconnect=1 \
         -device vhost-user-blk-pci,chardev=spdk_vhost_blk0,bootindex=1,num-queues=2
    

Things I've checked:

  • I can mount an NMVe-oF disk to the VM just fine using the same sequence of commands (giving QEMU an additional bootable drive) (just booting from it won't work)
  • the image on the NVMe-oF server boots just fine if I provide it locally (via the host-kernel NVMe-oF driver that I can't use in production) and declare it in the QEMU options as a drive
  • QEMU does not appear to have an NVMe-oF driver itself that I could use instead (it does have an NVMe driver)

QEMU version 7.2.15 (Debian 1:7.2+dfsg-7+deb12u12)

SPDK version SPDK v25.01-pre git sha1 8d960f1d8

1 Answer 1

1

Someone at the SPDK community slack helped me figure out the problem.

My NVMe-oF target is using a block size of 4096 byte, while QEMU's SeaBIOS has a hard limit on 512 for vhost-user boot disks.

So the solution is configuring the NVMe-oF target to use 512 byte blocks or switch QEMU to a BIOS that supports larger ones.

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.