+
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.
Redistribution and use in source and binary forms, with or without
with the Linux* KVM hypervisor by implementing the vhost-net offload API.
The sample application performs simple packet switching between virtual machines based on Media Access Control
(MAC) address or Virtual Local Area Network (VLAN) tag.
-The splitting of ethernet traffic from an external switch is performed in hardware by the Virtual Machine Device Queues
+The splitting of Ethernet traffic from an external switch is performed in hardware by the Virtual Machine Device Queues
(VMDQ) and Data Center Bridging (DCB) features of the Intel® 82599 10 Gigabit Ethernet Controller.
Background
Virtio networking (virtio-net) was developed as the Linux* KVM para-virtualized method for communicating network packets
between host and guest.
It was found that virtio-net performance was poor due to context switching and packet copying between host, guest, and QEMU.
-The following figure shows the system architecture for a virtio- based networking (virtio-net).
+The following figure shows the system architecture for a virtio-based networking (virtio-net).
-.. _figure_16:
+.. _figure_qemu_virtio_net:
-**Figure16. QEMU Virtio-net (prior to vhost-net)**
+.. figure:: img/qemu_virtio_net.*
-.. image19_png has been renamed
+ System Architecture for Virtio-based Networking (virtio-net).
-|qemu_virtio_net|
The Linux* Kernel vhost-net module was developed as an offload mechanism for virtio-net.
The vhost-net module enables KVM (QEMU) to offload the servicing of virtio-net devices to the vhost-net kernel module,
The following figure shows the system architecture for virtio-net networking with vhost-net offload.
-.. _figure_17:
+.. _figure_virtio_linux_vhost:
-**Figure 17. Virtio with Linux* Kernel Vhost**
+.. figure:: img/virtio_linux_vhost.*
-.. image20_png has been renamed
+ Virtio with Linux
-|virtio_linux_vhost|
Sample Code Overview
--------------------
The DPDK vhost-net sample code demonstrates KVM (QEMU) offloading the servicing of a Virtual Machine's (VM's)
virtio-net devices to a DPDK-based application in place of the kernel's vhost-net module.
-The DPDK vhost-net sample code is a simple packet switching application with the following features:
+The DPDK vhost-net sample code is based on vhost library. Vhost library is developed for user space Ethernet switch to
+easily integrate with vhost functionality.
+
+The vhost library implements the following features:
* Management of virtio-net device creation/destruction events.
-* Mapping of the VM's physical memory into the DPDK vhost-net sample code's address space.
+* Mapping of the VM's physical memory into the DPDK vhost-net's address space.
* Triggering/receiving notifications to/from VMs via eventfds.
* A virtio-net back-end implementation providing a subset of virtio-net features.
+There are two vhost implementations in vhost library, vhost cuse and vhost user. In vhost cuse, a character device driver is implemented to
+receive and process vhost requests through ioctl messages. In vhost user, a socket server is created to received vhost requests through
+socket messages. Most of the messages share the same handler routine.
+
+.. note::
+ **Any vhost cuse specific requirement in the following sections will be emphasized**.
+
+Two implementations are turned on and off statically through configure file. Only one implementation could be turned on. They don't co-exist in current implementation.
+
+The vhost sample code application is a simple packet switching application with the following feature:
+
* Packet switching between virtio-net devices and the network interface card,
including using VMDQs to reduce the switching that needs to be performed in software.
-The following figure shows the architecture of the Vhost sample application.
+The following figure shows the architecture of the Vhost sample application based on vhost-cuse.
-.. _figure_18:
+.. _figure_vhost_net_arch:
-**Figure 18. Vhost-net Architectural Overview**
+.. figure:: img/vhost_net_arch.*
-.. image21_png has been renamed
+ Vhost-net Architectural Overview
-|vhost_net_arch|
The following figure shows the flow of packets through the vhost-net sample application.
-.. _figure_19:
+.. _figure_vhost_net_sample_app:
-**Figure 19. Packet Flow Through the vhost-net Sample Application**
+.. figure:: img/vhost_net_sample_app.*
-.. image22_png has been renamed
+ Packet Flow Through the vhost-net Sample Application
-|vhost_net_sample_app|
Supported Distributions
-----------------------
* Fedora* 19
+* Fedora* 20
+
+.. _vhost_app_prerequisites:
+
Prerequisites
-------------
This section lists prerequisite packages that must be installed.
-Installing Packages on the Host
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Installing Packages on the Host(vhost cuse required)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The vhost sample code uses the following packages; fuse, fuse-devel, and kernel- modules-extra.
+The vhost cuse code uses the following packages; fuse, fuse-devel, and kernel-modules-extra.
+The vhost user code don't rely on those modules as eventfds are already installed into vhost process through
+Unix domain socket.
#. Install Fuse Development Libraries and headers:
yum -y install kernel-modules-extra
+QEMU simulator
+~~~~~~~~~~~~~~
+
+For vhost user, qemu 2.2 is required.
+
Setting up the Execution Environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/ nr_hugepages
+ echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
#. Mount hugetlbs at a separate mount point for 2 MB pages:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is recommended for testing purposes that the DPDK testpmd sample application is used in the guest to forward packets,
-the reasons for this are discussed in Section 22.7, "Running the Virtual Machine (QEMU)".
+the reasons for this are discussed in `Running the Virtual Machine (QEMU)`_.
The testpmd application forwards packets between pairs of Ethernet devices,
it requires an even number of Ethernet devices (virtio or otherwise) to execute.
Observe that in the example, "-device" and "-netdev" are repeated for two virtio-net devices.
+For vhost cuse:
+
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... \
+ qemu-system-x86_64 ... \
-netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> \
-device virtio-net-pci, netdev=hostnet1,id=net1 \
-netdev tap,id=hostnet2,vhost=on,vhostfd=<open fd> \
-device virtio-net-pci, netdev=hostnet2,id=net1
+For vhost user:
+
+.. code-block:: console
+
+ qemu-system-x86_64 ... \
+ -chardev socket,id=char1,path=<sock_path> \
+ -netdev type=vhost-user,id=hostnet1,chardev=char1 \
+ -device virtio-net-pci,netdev=hostnet1,id=net1 \
+ -chardev socket,id=char2,path=<sock_path> \
+ -netdev type=vhost-user,id=hostnet2,chardev=char2 \
+ -device virtio-net-pci,netdev=hostnet2,id=net2
+
+sock_path is the path for the socket file created by vhost.
Compiling the Sample Code
-------------------------
+#. Compile vhost lib:
-#. Go to the examples directory:
+ To enable vhost, turn on vhost library in the configure file config/common_linuxapp.
.. code-block:: console
- export RTE_SDK=/path/to/rte_sdk cd ${RTE_SDK}/examples/vhost-net
+ CONFIG_RTE_LIBRTE_VHOST=n
-#. Set the target (a default target is used if not specified). For example:
+ vhost user is turned on by default in the configure file config/common_linuxapp.
+ To enable vhost cuse, disable vhost user.
.. code-block:: console
- export RTE_TARGET=x86_64-native-linuxapp-gcc
+ CONFIG_RTE_LIBRTE_VHOST_USER=y
- See the DPDK Getting Started Guide for possible RTE_TARGET values.
+ After vhost is enabled and the implementation is selected, build the vhost library.
-#. Build the application:
+#. Go to the examples directory:
.. code-block:: console
- make
-
- .. note::
+ export RTE_SDK=/path/to/rte_sdk
+ cd ${RTE_SDK}/examples/vhost
- Note For zero copy, need firstly disable CONFIG_RTE_MBUF_SCATTER_GATHER,
- CONFIG_RTE_LIBRTE_IP_FRAG and CONFIG_RTE_LIBRTE_DISTRIBUTOR
- in the config file and then re-configure and compile the core lib, and then build the application:
+#. Set the target (a default target is used if not specified). For example:
.. code-block:: console
- vi ${RTE_SDK}/config/common_linuxapp
-
- change it as follows:
+ export RTE_TARGET=x86_64-native-linuxapp-gcc
- ::
+ See the DPDK Getting Started Guide for possible RTE_TARGET values.
- CONFIG_RTE_MBUF_SCATTER_GATHER=n
- CONFIG_RTE_LIBRTE_IP_FRAG=n
- CONFIG_RTE_LIBRTE_DISTRIBUTOR=n
+#. Build the application:
.. code-block:: console
cd ${RTE_SDK}/examples/vhost
make
-#. Go to the eventfd_link directory:
+#. Go to the eventfd_link directory(vhost cuse required):
.. code-block:: console
- cd ${RTE_SDK}/examples/vhost-net/eventfd_link
+ cd ${RTE_SDK}/lib/librte_vhost/eventfd_link
-#. Build the eventfd_link kernel module:
+#. Build the eventfd_link kernel module(vhost cuse required):
.. code-block:: console
Running the Sample Code
-----------------------
-#. Install the cuse kernel module:
+#. Install the cuse kernel module(vhost cuse required):
.. code-block:: console
modprobe cuse
-#. Go to the eventfd_link directory:
+#. Go to the eventfd_link directory(vhost cuse required):
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/vhost-net/eventfd_link
+ cd ${RTE_SDK}/lib/librte_vhost/eventfd_link
-#. Install the eventfd_link module:
+#. Install the eventfd_link module(vhost cuse required):
.. code-block:: console
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/vhost-net
+ cd ${RTE_SDK}/examples/vhost/build/app
#. Run the vhost-switch sample code:
+ vhost cuse:
+
+ .. code-block:: console
+
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
+
+ vhost user: a socket file named usvhost will be created under current directory. Use its path as the socket path in guest's qemu commandline.
+
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- -p 0x1 --dev-basename usvhost --dev-index 1
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
.. note::
Please note the huge-dir parameter instructs the DPDK to allocate its memory from the 2 MB page hugetlbfs.
+.. note::
+
+ The number used with the --socket-mem parameter may need to be more than 1024.
+ The number required depends on the number of mbufs allocated by vhost-switch.
+
+.. _vhost_app_parameters:
+
Parameters
~~~~~~~~~~
-**Basename and Index.**
-The DPDK vhost-net sample code uses a Linux* character device to communicate with QEMU.
-The basename and the index are used to generate the character devices name.
-
- /dev/<basename>-<index>
+**Basename.**
+vhost cuse uses a Linux* character device to communicate with QEMU.
+The basename is used to generate the character devices name.
-The index parameter is provided for a situation where multiple instances of the virtual switch is required.
+ /dev/<basename>
-For compatibility with the QEMU wrapper script, a base name of "usvhost" and an index of "1" should be used:
+For compatibility with the QEMU wrapper script, a base name of "usvhost" should be used:
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- -p 0x1 --dev-basename usvhost --dev-index 1
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- -p 0x1 --dev-basename usvhost
**vm2vm.**
The vm2vm parameter disable/set mode of packet switching between guests in the host.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --vm2vm [0,1,2]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --vm2vm [0,1,2]
**Mergeable Buffers.**
The mergeable buffers parameter controls how virtio-net descriptors are used for virtio-net headers.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --mergeable [0,1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --mergeable [0,1]
**Stats.**
The stats parameter controls the printing of virtio-net device statistics.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --stats [0,n]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --stats [0,n]
**RX Retry.**
The rx-retry option enables/disables enqueue retries when the guests RX queue is full.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry [0,1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry [0,1]
**RX Retry Number.**
The rx-retry-num option specifies the number of retries on an RX burst,
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry 1 --rx-retry-num 5
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry 1 --rx-retry-num 5
**RX Retry Delay Time.**
The rx-retry-delay option specifies the timeout (in micro seconds) between retries on an RX burst,
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir / mnt/huge -- --rx-retry 1 --rx-retry-delay 20
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --rx-retry 1 --rx-retry-delay 20
**Zero copy.**
-The zero copy option enables/disables the zero copy mode for RX/TX packet,
-in the zero copy mode the packet buffer address from guest translate into host physical address
-and then set directly as DMA address.
-If the zero copy mode is disabled, then one copy mode is utilized in the sample.
-This option is disabled by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy [0,1]
-
-**RX descriptor number.**
-The RX descriptor number option specify the Ethernet RX descriptor number,
-Linux legacy virtio-net has different behaviour in how to use the vring descriptor from DPDK based virtio-net PMD,
-the former likely allocate half for virtio header, another half for frame buffer,
-while the latter allocate all for frame buffer,
-this lead to different number for available frame buffer in vring,
-and then lead to different Ethernet RX descriptor number could be used in zero copy mode.
-So it is valid only in zero copy mode is enabled. The value is 32 by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy 1 --rx-desc-num [0, n]
-
-**TX descriptornumber.**
-The TX descriptor number option specify the Ethernet TX descriptor number, it is valid only in zero copy mode is enabled.
-The value is 64 by default.
-
-.. code-block:: console
-
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --zero-copy 1 --tx-desc-num [0, n]
+Zero copy mode is removed, due to it has not been working for a while. And
+due to the large and complex code, it's better to redesign it than fixing
+it to make it work again. Hence, zero copy may be added back later.
**VLAN strip.**
The VLAN strip option enable/disable the VLAN strip on host, if disabled, the guest will receive the packets with VLAN tag.
.. code-block:: console
- user@target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- --vlan-strip [0, 1]
+ ./vhost-switch -c f -n 4 --socket-mem 1024 --huge-dir /mnt/huge \
+ -- --vlan-strip [0, 1]
+
+.. _vhost_app_running:
Running the Virtual Machine (QEMU)
----------------------------------
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -device virtio-net-pci, netdev=hostnet1,id=net1 ...
+ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1, \
+ id=net1 ...
* Ensure the guest's virtio-net network adapter is configured with offloads disabled.
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -device virtio-net-pci, netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_ tso6=off,guest_ecn=off
+ qemu-system-x86_64 ... -device virtio-net-pci,netdev=hostnet1, \
+ id=net1, csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
-* Redirect QEMU to communicate with the DPDK vhost-net sample code in place of the vhost-net kernel module.
+* Redirect QEMU to communicate with the DPDK vhost-net sample code in place of the vhost-net kernel module(vhost cuse).
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> ...
+ qemu-system-x86_64 ... -netdev tap,id=hostnet1,vhost=on, \
+ vhostfd=<open fd> ...
* Enable the vhost-net sample code to map the VM's memory into its own process address space.
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -mem-prealloc -mem-path / dev/hugepages ...
+ qemu-system-x86_64 ... -mem-prealloc -mem-path /dev/hugepages ...
.. note::
The QEMU wrapper (qemu-wrap.py) is a Python script designed to automate the QEMU configuration described above.
It also facilitates integration with libvirt, although the script may also be used standalone without libvirt.
-Redirecting QEMU to vhost-net Sample Code
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Redirecting QEMU to vhost-net Sample Code(vhost cuse)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To redirect QEMU to the vhost-net sample code implementation of the vhost-net API,
an open file descriptor must be passed to QEMU running as a child process.
#!/usr/bin/python
fd = os.open("/dev/usvhost-1", os.O_RDWR)
- subprocess.call("qemu-system-x86_64 ... . -netdev tap,id=vhostnet0,vhost=on,vhostfd=" + fd +"...", shell=True)
+ subprocess.call
+ ("qemu-system-x86_64 ... -netdev tap,id=vhostnet0,vhost=on,vhostfd="
+ + fd +"...", shell=True)
.. note::
- This process is automated in the QEMU wrapper script discussed in Section 22.7.3.
+ This process is automated in the `QEMU Wrapper Script`_.
Mapping the Virtual Machine's Memory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ qemu-system-x86_64 ... -mem-prealloc -mem-path / dev/hugepages ...
+ qemu-system-x86_64 ... -mem-prealloc -mem-path /dev/hugepages ...
.. note::
- This process is automated in the QEMU wrapper script discussed in Section 22.7.3.
+ This process is automated in the `QEMU Wrapper Script`_.
+ The following two sections only applies to vhost cuse.
+ For vhost-user, please make corresponding changes to qemu-wrapper script and guest XML file.
QEMU Wrapper Script
~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ qemu-wrap.py -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1
- -netdev tap,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1 -hda <disk img> -m 4096
+ qemu-wrap.py -machine pc-i440fx-1.4,accel=kvm,usb=off \
+ -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1 \
+ -netdev tap,id=hostnet1,vhost=on \
+ -device virtio-net-pci,netdev=hostnet1,id=net1 \
+ -hda <disk img> -m 4096
which will become the following call to QEMU:
.. code-block:: console
- /usr/local/bin/qemu-system-x86_64 -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1
- -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> -device virtio-net- pci,netdev=hostnet1,id=net1,
- csum=off,gso=off,guest_tso4=off,gu est_tso6=off,guest_ecn=off -hda <disk img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
+ qemu-system-x86_64 -machine pc-i440fx-1.4,accel=kvm,usb=off \
+ -cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1 \
+ -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> \
+ -device virtio-net-pci,netdev=hostnet1,id=net1, \
+ csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \
+ -hda <disk img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
Libvirt Integration
~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ mkdir /dev/cgroup
- user@target:~$ mount -t cgroup none /dev/cgroup -o devices
+ mkdir /dev/cgroup
+ mount -t cgroup none /dev/cgroup -o devices
* Restart the libvirtd system process
emul_path = "/usr/local/bin/qemu-system-x86_64"
- * Configure the "us_vhost_path" variable to point to the DPDK vhost- net sample code's character devices name.
- DPDK vhost-net sample code's character device will be in the format "/dev/<basename>-<index>".
+ * Configure the "us_vhost_path" variable to point to the DPDK vhost-net sample code's character devices name.
+ DPDK vhost-net sample code's character device will be in the format "/dev/<basename>".
.. code-block:: xml
- us_vhost_path = "/dev/usvhost-1"
+ us_vhost_path = "/dev/usvhost"
Common Issues
~~~~~~~~~~~~~
-**QEMU failing to allocate memory on hugetlbfs.**
+* QEMU failing to allocate memory on hugetlbfs, with an error like the following::
-file_ram_alloc: can't mmap RAM pages: Cannot allocate memory
+ file_ram_alloc: can't mmap RAM pages: Cannot allocate memory
-When running QEMU the above error implies that it has failed to allocate memory for the Virtual Machine on the hugetlbfs.
-This is typically due to insufficient hugepages being free to support the allocation request.
-The number of free hugepages can be checked as follows:
+ When running QEMU the above error indicates that it has failed to allocate memory for the Virtual Machine on
+ the hugetlbfs. This is typically due to insufficient hugepages being free to support the allocation request.
+ The number of free hugepages can be checked as follows:
-.. code-block:: console
+ .. code-block:: console
+
+ cat /sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages
+
+ The command above indicates how many hugepages are free to support QEMU's allocation request.
+
+* User space VHOST when the guest has 2MB sized huge pages:
+
+ The guest may have 2MB or 1GB sized huge pages. The user space VHOST should work properly in both cases.
+
+* User space VHOST will not work with QEMU without the ``-mem-prealloc`` option:
- user@target:cat /sys/kernel/mm/hugepages/hugepages-<pagesize> / nr_hugepages
+ The current implementation works properly only when the guest memory is pre-allocated, so it is required to
+ use a QEMU version (e.g. 1.6) which supports ``-mem-prealloc``. The ``-mem-prealloc`` option must be
+ specified explicitly in the QEMU command line.
-The command above indicates how many hugepages are free to support QEMU's allocation request.
+* User space VHOST will not work with a QEMU version without shared memory mapping:
+
+ As shared memory mapping is mandatory for user space VHOST to work properly with the guest, user space VHOST
+ needs access to the shared memory from the guest to receive and transmit packets. It is important to make sure
+ the QEMU version supports shared memory mapping.
+
+* In an Ubuntu environment, QEMU fails to start a new guest normally with user space VHOST due to not being able
+ to allocate huge pages for the new guest:
+
+ The solution for this issue is to add ``-boot c`` into the QEMU command line to make sure the huge pages are
+ allocated properly and then the guest should start normally.
+
+ Use ``cat /proc/meminfo`` to check if there is any changes in the value of ``HugePages_Total`` and ``HugePages_Free``
+ after the guest startup.
+
+* Log message: ``eventfd_link: module verification failed: signature and/or required key missing - tainting kernel``:
+
+ This log message may be ignored. The message occurs due to the kernel module ``eventfd_link``, which is not a standard
+ Linux module but which is necessary for the user space VHOST current implementation (CUSE-based) to communicate with
+ the guest.
+
+.. _vhost_app_running_dpdk:
Running DPDK in the Virtual Machine
-----------------------------------
The "wait and retry" algorithm is implemented in DPDK testpmd as a forwarding method call "mac_retry".
The following sequence diagram describes the algorithm in detail.
-.. _figure_20:
+.. _figure_tx_dpdk_testpmd:
-**Figure 20. Packet Flow on TX in DPDK-testpmd**
+.. figure:: img/tx_dpdk_testpmd.*
-.. image23_png has been renamed
+ Packet Flow on TX in DPDK-testpmd
-|tx_dpdk_testpmd|
Running Testpmd
~~~~~~~~~~~~~~~
.. code-block:: console
- user@target:~$ x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -- n 4 -socket-mem 128 -- --burst=64 -i
+ cd ${RTE_SDK}/x86_64-native-linuxapp-gcc/app
+ ./testpmd -c 0x3 -n 4 --socket-mem 512 \
+ -- --burst=64 --i --disable-hw-vlan-filter
The destination MAC address for packets transmitted on each port can be set at the command line:
.. code-block:: console
- user@target:~$ x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -- n 4 -socket-mem 128 -- --burst=64 -i --eth- peer=0,aa:bb:cc:dd:ee:ff --eth-peer=1,ff,ee,dd,cc,bb,aa
+ ./testpmd -c 0x3 -n 4 --socket-mem 512 \
+ -- --burst=64 --i --disable-hw-vlan-filter \
+ --eth-peer=0,aa:bb:cc:dd:ee:ff --eth-peer=1,ff:ee:dd:cc:bb:aa
* Packets received on port 1 will be forwarded on port 0 to MAC address
- aa:bb:cc:dd:ee:ff.
+ aa:bb:cc:dd:ee:ff
* Packets received on port 0 will be forwarded on port 1 to MAC address
- ff,ee,dd,cc,bb,aa.
+ ff:ee:dd:cc:bb:aa
The testpmd application can then be configured to act as an L2 forwarding application:
Any packets received on the NIC with these values is placed on the devices receive queue.
When a virtio-net device transmits packets, the VLAN tag is added to the packet by the DPDK vhost sample code.
-.. |vhost_net_arch| image:: img/vhost_net_arch.*
+Running virtio_user with vhost-switch
+-------------------------------------
-.. |qemu_virtio_net| image:: img/qemu_virtio_net.*
+We can also use virtio_user with vhost-switch now.
+Virtio_user is a virtual device that can be run in a application (container) parallelly with vhost in the same OS,
+aka, there is no need to start a VM. We just run it with a different --file-prefix to avoid startup failure.
-.. |tx_dpdk_testpmd| image:: img/tx_dpdk_testpmd.*
+.. code-block:: console
-.. |vhost_net_sample_app| image:: img/vhost_net_sample_app.*
+ cd ${RTE_SDK}/x86_64-native-linuxapp-gcc/app
+ ./testpmd -c 0x3 -n 4 --socket-mem 1024 --no-pci --file-prefix=virtio_user-testpmd \
+ --vdev=virtio_user0,mac=00:01:02:03:04:05,path=$path_vhost \
+ -- -i --txqflags=0xf01 --disable-hw-vlan
-.. |virtio_linux_vhost| image:: img/virtio_linux_vhost.*
+There is no difference on the vhost side.
+Pleae note that there are some limitations (see release note for more information) in the usage of virtio_user.
git.droids-corp.org / dpdk.git / blobdiff