+
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.
Redistribution and use in source and binary forms, with or without
with the Linux* KVM hypervisor by implementing the vhost-net offload API.
The sample application performs simple packet switching between virtual machines based on Media Access Control
(MAC) address or Virtual Local Area Network (VLAN) tag.
-The splitting of ethernet traffic from an external switch is performed in hardware by the Virtual Machine Device Queues
+The splitting of Ethernet traffic from an external switch is performed in hardware by the Virtual Machine Device Queues
(VMDQ) and Data Center Bridging (DCB) features of the IntelĀ® 82599 10 Gigabit Ethernet Controller.
Background
It was found that virtio-net performance was poor due to context switching and packet copying between host, guest, and QEMU.
The following figure shows the system architecture for a virtio-based networking (virtio-net).
-.. _figure_16:
+.. _figure_qemu_virtio_net:
-**Figure16. QEMU Virtio-net (prior to vhost-net)**
+.. figure:: img/qemu_virtio_net.*
-.. image19_png has been renamed
+ System Architecture for Virtio-based Networking (virtio-net).
-|qemu_virtio_net|
The Linux* Kernel vhost-net module was developed as an offload mechanism for virtio-net.
The vhost-net module enables KVM (QEMU) to offload the servicing of virtio-net devices to the vhost-net kernel module,
The following figure shows the system architecture for virtio-net networking with vhost-net offload.
-.. _figure_17:
+.. _figure_virtio_linux_vhost:
-**Figure 17. Virtio with Linux* Kernel Vhost**
+.. figure:: img/virtio_linux_vhost.*
-.. image20_png has been renamed
+ Virtio with Linux
-|virtio_linux_vhost|
Sample Code Overview
--------------------
The DPDK vhost-net sample code demonstrates KVM (QEMU) offloading the servicing of a Virtual Machine's (VM's)
virtio-net devices to a DPDK-based application in place of the kernel's vhost-net module.
-The DPDK vhost-net sample code is based on vhost library. Vhost library is developed for user space ethernet switch to
+The DPDK vhost-net sample code is based on vhost library. Vhost library is developed for user space Ethernet switch to
easily integrate with vhost functionality.
The vhost library implements the following features:
.. note::
**Any vhost cuse specific requirement in the following sections will be emphasized**.
-Two impelmentations are turned on and off statically through configure file. Only one implementation could be turned on. They don't co-exist in current implementation.
+Two implementations are turned on and off statically through configure file. Only one implementation could be turned on. They don't co-exist in current implementation.
The vhost sample code application is a simple packet switching application with the following feature:
The following figure shows the architecture of the Vhost sample application based on vhost-cuse.
-.. _figure_18:
+.. _figure_vhost_net_arch:
-**Figure 18. Vhost-net Architectural Overview**
+.. figure:: img/vhost_net_arch.*
-.. image21_png has been renamed
+ Vhost-net Architectural Overview
-|vhost_net_arch|
The following figure shows the flow of packets through the vhost-net sample application.
-.. _figure_19:
+.. _figure_vhost_net_sample_app:
-**Figure 19. Packet Flow Through the vhost-net Sample Application**
+.. figure:: img/vhost_net_sample_app.*
-.. image22_png has been renamed
+ Packet Flow Through the vhost-net Sample Application
-|vhost_net_sample_app|
Supported Distributions
-----------------------
* Fedora* 20
+.. _vhost_app_prerequisites:
+
Prerequisites
-------------
The vhost cuse code uses the following packages; fuse, fuse-devel, and kernel-modules-extra.
The vhost user code don't rely on those modules as eventfds are already installed into vhost process through
-unix domain socket.
+Unix domain socket.
#. Install Fuse Development Libraries and headers:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is recommended for testing purposes that the DPDK testpmd sample application is used in the guest to forward packets,
-the reasons for this are discussed in Section 22.7, "Running the Virtual Machine (QEMU)".
+the reasons for this are discussed in `Running the Virtual Machine (QEMU)`_.
The testpmd application forwards packets between pairs of Ethernet devices,
it requires an even number of Ethernet devices (virtio or otherwise) to execute.
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... \
+ qemu-system-x86_64 ... \
-netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> \
-device virtio-net-pci, netdev=hostnet1,id=net1 \
-netdev tap,id=hostnet2,vhost=on,vhostfd=<open fd> \
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... \
+ qemu-system-x86_64 ... \
-chardev socket,id=char1,path=<sock_path> \
-netdev type=vhost-user,id=hostnet1,chardev=char1 \
-device virtio-net-pci,netdev=hostnet1,id=net1 \
CONFIG_RTE_LIBRTE_VHOST=n
- vhost user is turned on by default in the lib/librte_vhost/Makefile.
- To enable vhost cuse, uncomment vhost cuse and comment vhost user manually. In future, a configure will be created for switch between two implementations.
+ vhost user is turned on by default in the configure file config/common_linuxapp.
+ To enable vhost cuse, disable vhost user.
.. code-block:: console
- SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_cuse/vhost-net-cdev.c vhost_cuse/virtio-net-cdev.c vhost_cuse/eventfd_copy.c
- #SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_user/vhost-net-user.c vhost_user/virtio-net-user.c vhost_user/fd_man.c
+ CONFIG_RTE_LIBRTE_VHOST_USER=y
After vhost is enabled and the implementation is selected, build the vhost library.
#. Build the application:
- .. code-block:: console
-
- make
-
- .. note::
-
- Note For zero copy, need firstly disable CONFIG_RTE_MBUF_SCATTER_GATHER,
- CONFIG_RTE_LIBRTE_IP_FRAG and CONFIG_RTE_LIBRTE_DISTRIBUTOR
- in the config file and then re-configure and compile the core lib, and then build the application:
-
- .. code-block:: console
-
- vi ${RTE_SDK}/config/common_linuxapp
-
- change it as follows:
-
- ::
-
- CONFIG_RTE_MBUF_SCATTER_GATHER=n
- CONFIG_RTE_LIBRTE_IP_FRAG=n
- CONFIG_RTE_LIBRTE_DISTRIBUTOR=n
-
.. code-block:: console
cd ${RTE_SDK}
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/vhost
+ cd ${RTE_SDK}/examples/vhost/build/app
#. Run the vhost-switch sample code:
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- -p 0x1 --dev-basename usvhost --dev-index 1
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
vhost user: a socket file named usvhost will be created under current directory. Use its path as the socket path in guest's qemu commandline.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- -p 0x1 --dev-basename usvhost
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
.. note::
Please note the huge-dir parameter instructs the DPDK to allocate its memory from the 2 MB page hugetlbfs.
+.. note::
+
+ The number used with the --socket-mem parameter may need to be more than 1024.
+ The number required depends on the number of mbufs allocated by vhost-switch.
+
+.. _vhost_app_parameters:
+
Parameters
~~~~~~~~~~
-**Basename and Index.**
+**Basename.**
vhost cuse uses a Linux* character device to communicate with QEMU.
-The basename and the index are used to generate the character devices name.
-
- /dev/<basename>-<index>
+The basename is used to generate the character devices name.
-The index parameter is provided for a situation where multiple instances of the virtual switch is required.
+ /dev/<basename>
-For compatibility with the QEMU wrapper script, a base name of "usvhost" and an index of "1" should be used:
+For compatibility with the QEMU wrapper script, a base name of "usvhost" should be used:
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- -p 0x1 --dev-basename usvhost --dev-index 1
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
**vm2vm.**
The vm2vm parameter disable/set mode of packet switching between guests in the host.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --vm2vm [0,1,2]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --vm2vm [0,1,2]
**Mergeable Buffers.**
The mergeable buffers parameter controls how virtio-net descriptors are used for virtio-net headers.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --mergeable [0,1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --mergeable [0,1]
**Stats.**
The stats parameter controls the printing of virtio-net device statistics.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --stats [0,n]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --stats [0,n]
**RX Retry.**
The rx-retry option enables/disables enqueue retries when the guests RX queue is full.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry [0,1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry [0,1]
**RX Retry Number.**
The rx-retry-num option specifies the number of retries on an RX burst,
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry 1 --rx-retry-num 5
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry 1 --rx-retry-num 5
**RX Retry Delay Time.**
The rx-retry-delay option specifies the timeout (in micro seconds) between retries on an RX burst,
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry 1 --rx-retry-delay 20
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry 1 --rx-retry-delay 20
**Zero copy.**
-The zero copy option enables/disables the zero copy mode for RX/TX packet,
-in the zero copy mode the packet buffer address from guest translate into host physical address
-and then set directly as DMA address.
-If the zero copy mode is disabled, then one copy mode is utilized in the sample.
-This option is disabled by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy [0,1]
-
-**RX descriptor number.**
-The RX descriptor number option specify the Ethernet RX descriptor number,
-Linux legacy virtio-net has different behaviour in how to use the vring descriptor from DPDK based virtio-net PMD,
-the former likely allocate half for virtio header, another half for frame buffer,
-while the latter allocate all for frame buffer,
-this lead to different number for available frame buffer in vring,
-and then lead to different Ethernet RX descriptor number could be used in zero copy mode.
-So it is valid only in zero copy mode is enabled. The value is 32 by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy 1 --rx-desc-num [0, n]
-
-**TX descriptornumber.**
-The TX descriptor number option specify the Ethernet TX descriptor number, it is valid only in zero copy mode is enabled.
-The value is 64 by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy 1 --tx-desc-num [0, n]
+Zero copy mode is removed, due to it has not been working for a while. And
+due to the large and complex code, it's better to redesign it than fixing
+it to make it work again. Hence, zero copy may be added back later.
**VLAN strip.**
The VLAN strip option enable/disable the VLAN strip on host, if disabled, the guest will receive the packets with VLAN tag.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --vlan-strip [0, 1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --vlan-strip [0, 1]
+
+.. _vhost_app_running:
Running the Virtual Machine (QEMU)
----------------------------------
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1,id=net1 ...
+ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1, \
+ id=net1 ...
* Ensure the guest's virtio-net network adapter is configured with offloads disabled.
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
+ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1, \
+ id=net1, csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
* Redirect QEMU to communicate with the DPDK vhost-net sample code in place of the vhost-net kernel module(vhost cuse).
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> ...
+ qemu-system-x86_64 ... -netdev tap,id=hostnet1,vhost=on, \
+ vhostfd=<open fd> ...
* Enable the vhost-net sample code to map the VM's memory into its own process address space.
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -mem-prealloc -mem-path / dev/hugepages ...
+ qemu-system-x86_64 ... -mem-prealloc -mem-path /dev/hugepages ...
.. note::
#!/usr/bin/python
fd = os.open("/dev/usvhost-1", os.O_RDWR)
- subprocess.call("qemu-system-x86_64 ... . -netdev tap,id=vhostnet0,vhost=on,vhostfd=" + fd +"...", shell=True)
+ subprocess.call
+ ("qemu-system-x86_64 ... -netdev tap,id=vhostnet0,vhost=on,vhostfd="
+ + fd +"...", shell=True)
.. note::
- This process is automated in the QEMU wrapper script discussed in Section 24.7.3.
+ This process is automated in the `QEMU Wrapper Script`_.
Mapping the Virtual Machine's Memory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -mem-prealloc -mem-path / dev/hugepages ...
+ qemu-system-x86_64 ... -mem-prealloc -mem-path /dev/hugepages ...
.. note::
- This process is automated in the QEMU wrapper script discussed in Section 24.7.3.
- The following two sections only applies to vhost cuse. For vhost-user, please make corresponding changes to qemu-wrapper script and guest XML file.
+ This process is automated in the `QEMU Wrapper Script`_.
+ The following two sections only applies to vhost cuse.
+ For vhost-user, please make corresponding changes to qemu-wrapper script and guest XML file.
QEMU Wrapper Script
~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ qemu-wrap.py -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1
- -netdev tap,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1 -hda <disk img> -m 4096
+ qemu-wrap.py -machine pc-i440fx-1.4,accel=kvm,usb=off \
+ -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1 \
+ -netdev tap,id=hostnet1,vhost=on \
+ -device virtio-net-pci,netdev=hostnet1,id=net1 \
+ -hda <disk img> -m 4096
which will become the following call to QEMU:
.. code-block:: console
- /usr/local/bin/qemu-system-x86_64 -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1
- -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> -device virtio-net-pci,netdev=hostnet1,id=net1,
- csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off -hda <disk img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
+ qemu-system-x86_64 -machine pc-i440fx-1.4,accel=kvm,usb=off \
+ -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1 \
+ -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> \
+ -device virtio-net-pci,netdev=hostnet1,id=net1, \
+ csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \
+ -hda <disk img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
Libvirt Integration
~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ mkdir /dev/cgroup
- user@target:~$ mount -t cgroup none /dev/cgroup -o devices
+ mkdir /dev/cgroup
+ mount -t cgroup none /dev/cgroup -o devices
* Restart the libvirtd system process
emul_path = "/usr/local/bin/qemu-system-x86_64"
* Configure the "us_vhost_path" variable to point to the DPDK vhost-net sample code's character devices name.
- DPDK vhost-net sample code's character device will be in the format "/dev/<basename>-<index>".
+ DPDK vhost-net sample code's character device will be in the format "/dev/<basename>".
.. code-block:: xml
- us_vhost_path = "/dev/usvhost-1"
+ us_vhost_path = "/dev/usvhost"
Common Issues
~~~~~~~~~~~~~
-**QEMU failing to allocate memory on hugetlbfs.**
+* QEMU failing to allocate memory on hugetlbfs, with an error like the following::
-file_ram_alloc: can't mmap RAM pages: Cannot allocate memory
+ file_ram_alloc: can't mmap RAM pages: Cannot allocate memory
-When running QEMU the above error implies that it has failed to allocate memory for the Virtual Machine on the hugetlbfs.
-This is typically due to insufficient hugepages being free to support the allocation request.
-The number of free hugepages can be checked as follows:
+ When running QEMU the above error indicates that it has failed to allocate memory for the Virtual Machine on
+ the hugetlbfs. This is typically due to insufficient hugepages being free to support the allocation request.
+ The number of free hugepages can be checked as follows:
-.. code-block:: console
+ .. code-block:: console
+
+ cat /sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages
- user@target:cat /sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages
+ The command above indicates how many hugepages are free to support QEMU's allocation request.
-The command above indicates how many hugepages are free to support QEMU's allocation request.
+* User space VHOST when the guest has 2MB sized huge pages:
+
+ The guest may have 2MB or 1GB sized huge pages. The user space VHOST should work properly in both cases.
+
+* User space VHOST will not work with QEMU without the ``-mem-prealloc`` option:
+
+ The current implementation works properly only when the guest memory is pre-allocated, so it is required to
+ use a QEMU version (e.g. 1.6) which supports ``-mem-prealloc``. The ``-mem-prealloc`` option must be
+ specified explicitly in the QEMU command line.
+
+* User space VHOST will not work with a QEMU version without shared memory mapping:
+
+ As shared memory mapping is mandatory for user space VHOST to work properly with the guest, user space VHOST
+ needs access to the shared memory from the guest to receive and transmit packets. It is important to make sure
+ the QEMU version supports shared memory mapping.
+
+* In an Ubuntu environment, QEMU fails to start a new guest normally with user space VHOST due to not being able
+ to allocate huge pages for the new guest:
+
+ The solution for this issue is to add ``-boot c`` into the QEMU command line to make sure the huge pages are
+ allocated properly and then the guest should start normally.
+
+ Use ``cat /proc/meminfo`` to check if there is any changes in the value of ``HugePages_Total`` and ``HugePages_Free``
+ after the guest startup.
+
+* Log message: ``eventfd_link: module verification failed: signature and/or required key missing - tainting kernel``:
+
+ This log message may be ignored. The message occurs due to the kernel module ``eventfd_link``, which is not a standard
+ Linux module but which is necessary for the user space VHOST current implementation (CUSE-based) to communicate with
+ the guest.
+
+.. _vhost_app_running_dpdk:
Running DPDK in the Virtual Machine
-----------------------------------
The "wait and retry" algorithm is implemented in DPDK testpmd as a forwarding method call "mac_retry".
The following sequence diagram describes the algorithm in detail.
-.. _figure_20:
+.. _figure_tx_dpdk_testpmd:
-**Figure 20. Packet Flow on TX in DPDK-testpmd**
+.. figure:: img/tx_dpdk_testpmd.*
-.. image23_png has been renamed
+ Packet Flow on TX in DPDK-testpmd
-|tx_dpdk_testpmd|
Running Testpmd
~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -- n 4 -socket-mem 128 -- --burst=64 -i
+ cd ${RTE_SDK}/x86_64-native-linuxapp-gcc/app
+ ./testpmd -c 0x3 -n 4 --socket-mem 512 \
+ -- --burst=64 --i --disable-hw-vlan-filter
The destination MAC address for packets transmitted on each port can be set at the command line:
.. code-block:: console
- user@target:~$ x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -- n 4 -socket-mem 128 -- --burst=64 -i --eth- peer=0,aa:bb:cc:dd:ee:ff --eth-peer=1,ff,ee,dd,cc,bb,aa
+ ./testpmd -c 0x3 -n 4 --socket-mem 512 \
+ -- --burst=64 --i --disable-hw-vlan-filter \
+ --eth-peer=0,aa:bb:cc:dd:ee:ff --eth-peer=1,ff:ee:dd:cc:bb:aa
* Packets received on port 1 will be forwarded on port 0 to MAC address
- aa:bb:cc:dd:ee:ff.
+ aa:bb:cc:dd:ee:ff
* Packets received on port 0 will be forwarded on port 1 to MAC address
- ff,ee,dd,cc,bb,aa.
+ ff:ee:dd:cc:bb:aa
The testpmd application can then be configured to act as an L2 forwarding application:
Any packets received on the NIC with these values is placed on the devices receive queue.
When a virtio-net device transmits packets, the VLAN tag is added to the packet by the DPDK vhost sample code.
-.. |vhost_net_arch| image:: img/vhost_net_arch.*
+Running virtio-user with vhost-switch
+-------------------------------------
-.. |qemu_virtio_net| image:: img/qemu_virtio_net.*
+We can also use virtio-user with vhost-switch now.
+Virtio-user is a virtual device that can be run in a application (container) parallelly with vhost in the same OS,
+aka, there is no need to start a VM. We just run it with a different --file-prefix to avoid startup failure.
-.. |tx_dpdk_testpmd| image:: img/tx_dpdk_testpmd.*
+.. code-block:: console
-.. |vhost_net_sample_app| image:: img/vhost_net_sample_app.*
+ cd ${RTE_SDK}/x86_64-native-linuxapp-gcc/app
+ ./testpmd -c 0x3 -n 4 --socket-mem 1024 --no-pci --file-prefix=virtio-user-testpmd \
+ --vdev=virtio-user0,mac=00:01:02:03:04:05,path=$path_vhost \
+ -- -i --txqflags=0xf01 --disable-hw-vlan
-.. |virtio_linux_vhost| image:: img/virtio_linux_vhost.*
+There is no difference on the vhost side.
+Pleae note that there are some limitations (see release note for more information) in the usage of virtio-user.