1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2010-2014 Intel Corporation.
4 Intel Virtual Function Driver
5 =============================
7 Supported Intel® Ethernet Controllers (see the *DPDK Release Notes* for details)
8 support the following modes of operation in a virtualized environment:
10 * **SR-IOV mode**: Involves direct assignment of part of the port resources to different guest operating systems
11 using the PCI-SIG Single Root I/O Virtualization (SR IOV) standard,
12 also known as "native mode" or "pass-through" mode.
13 In this chapter, this mode is referred to as IOV mode.
15 * **VMDq mode**: Involves central management of the networking resources by an IO Virtual Machine (IOVM) or
16 a Virtual Machine Monitor (VMM), also known as software switch acceleration mode.
17 In this chapter, this mode is referred to as the Next Generation VMDq mode.
19 SR-IOV Mode Utilization in a DPDK Environment
20 ---------------------------------------------
22 The DPDK uses the SR-IOV feature for hardware-based I/O sharing in IOV mode.
23 Therefore, it is possible to partition SR-IOV capability on Ethernet controller NIC resources logically and
24 expose them to a virtual machine as a separate PCI function called a "Virtual Function".
25 Refer to :numref:`figure_single_port_nic`.
27 Therefore, a NIC is logically distributed among multiple virtual machines (as shown in :numref:`figure_single_port_nic`),
28 while still having global data in common to share with the Physical Function and other Virtual Functions.
29 The DPDK fm10kvf, i40evf, igbvf or ixgbevf as a Poll Mode Driver (PMD) serves for the Intel® 82576 Gigabit Ethernet Controller,
30 Intel® Ethernet Controller I350 family, Intel® 82599 10 Gigabit Ethernet Controller NIC,
31 Intel® Fortville 10/40 Gigabit Ethernet Controller NIC's virtual PCI function, or PCIe host-interface of the Intel Ethernet Switch
33 Meanwhile the DPDK Poll Mode Driver (PMD) also supports "Physical Function" of such NIC's on the host.
35 The DPDK PF/VF Poll Mode Driver (PMD) supports the Layer 2 switch on Intel® 82576 Gigabit Ethernet Controller,
36 Intel® Ethernet Controller I350 family, Intel® 82599 10 Gigabit Ethernet Controller,
37 and Intel® Fortville 10/40 Gigabit Ethernet Controller NICs so that guest can choose it for inter virtual machine traffic in SR-IOV mode.
39 For more detail on SR-IOV, please refer to the following documents:
41 * `SR-IOV provides hardware based I/O sharing <http://www.intel.com/network/connectivity/solutions/vmdc.htm>`_
43 * `PCI-SIG-Single Root I/O Virtualization Support on IA
44 <http://www.intel.com/content/www/us/en/pci-express/pci-sig-single-root-io-virtualization-support-in-virtualization-technology-for-connectivity-paper.html>`_
46 * `Scalable I/O Virtualized Servers <http://www.intel.com/content/www/us/en/virtualization/server-virtualization/scalable-i-o-virtualized-servers-paper.html>`_
48 .. _figure_single_port_nic:
50 .. figure:: img/single_port_nic.*
52 Virtualization for a Single Port NIC in SR-IOV Mode
55 Physical and Virtual Function Infrastructure
56 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
58 The following describes the Physical Function and Virtual Functions infrastructure for the supported Ethernet Controller NICs.
60 Virtual Functions operate under the respective Physical Function on the same NIC Port and therefore have no access
61 to the global NIC resources that are shared between other functions for the same NIC port.
63 A Virtual Function has basic access to the queue resources and control structures of the queues assigned to it.
64 For global resource access, a Virtual Function has to send a request to the Physical Function for that port,
65 and the Physical Function operates on the global resources on behalf of the Virtual Function.
66 For this out-of-band communication, an SR-IOV enabled NIC provides a memory buffer for each Virtual Function,
67 which is called a "Mailbox".
69 Intel® Ethernet Adaptive Virtual Function
70 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
71 Adaptive Virtual Function (AVF) is a SR-IOV Virtual Function with the same device id (8086:1889) on different Intel Ethernet Controller.
72 AVF Driver is VF driver which supports for all future Intel devices without requiring a VM update. And since this happens to be an adaptive VF driver,
73 every new drop of the VF driver would add more and more advanced features that can be turned on in the VM if the underlying HW device supports those
74 advanced features based on a device agnostic way without ever compromising on the base functionality. AVF provides generic hardware interface and
75 interface between AVF driver and a compliant PF driver is specified.
77 Intel products starting Ethernet Controller 710 Series to support Adaptive Virtual Function.
79 The way to generate Virtual Function is like normal, and the resource of VF assignment depends on the NIC Infrastructure.
81 For more detail on SR-IOV, please refer to the following documents:
83 * `Intel® AVF HAS <https://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ethernet-adaptive-virtual-function-hardware-spec.pdf>`_
85 The PCIE host-interface of Intel Ethernet Switch FM10000 Series VF infrastructure
86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
88 In a virtualized environment, the programmer can enable a maximum of *64 Virtual Functions (VF)*
89 globally per PCIE host-interface of the Intel Ethernet Switch FM10000 Series device.
90 Each VF can have a maximum of 16 queue pairs.
91 The Physical Function in host could be only configured by the Linux* fm10k driver
92 (in the case of the Linux Kernel-based Virtual Machine [KVM]), DPDK PMD PF driver doesn't support it yet.
96 * Using Linux* fm10k driver:
98 .. code-block:: console
100 rmmod fm10k (To remove the fm10k module)
101 insmod fm0k.ko max_vfs=2,2 (To enable two Virtual Functions per port)
103 Virtual Function enumeration is performed in the following sequence by the Linux* pci driver for a dual-port NIC.
104 When you enable the four Virtual Functions with the above command, the four enabled functions have a Function#
105 represented by (Bus#, Device#, Function#) in sequence starting from 0 to 3.
108 * Virtual Functions 0 and 2 belong to Physical Function 0
110 * Virtual Functions 1 and 3 belong to Physical Function 1
114 The above is an important consideration to take into account when targeting specific packets to a selected port.
116 Intel® X710/XL710 Gigabit Ethernet Controller VF Infrastructure
117 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
119 In a virtualized environment, the programmer can enable a maximum of *128 Virtual Functions (VF)*
120 globally per Intel® X710/XL710 Gigabit Ethernet Controller NIC device.
121 The number of queue pairs of each VF can be configured by ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF`` in ``config`` file.
122 The Physical Function in host could be either configured by the Linux* i40e driver
123 (in the case of the Linux Kernel-based Virtual Machine [KVM]) or by DPDK PMD PF driver.
124 When using both DPDK PMD PF/VF drivers, the whole NIC will be taken over by DPDK based application.
128 * Using Linux* i40e driver:
130 .. code-block:: console
132 rmmod i40e (To remove the i40e module)
133 insmod i40e.ko max_vfs=2,2 (To enable two Virtual Functions per port)
135 * Using the DPDK PMD PF i40e driver:
137 Kernel Params: iommu=pt, intel_iommu=on
139 .. code-block:: console
143 ./dpdk-devbind.py -b igb_uio bb:ss.f
144 echo 2 > /sys/bus/pci/devices/0000\:bb\:ss.f/max_vfs (To enable two VFs on a specific PCI device)
146 Launch the DPDK testpmd/example or your own host daemon application using the DPDK PMD library.
148 Virtual Function enumeration is performed in the following sequence by the Linux* pci driver for a dual-port NIC.
149 When you enable the four Virtual Functions with the above command, the four enabled functions have a Function#
150 represented by (Bus#, Device#, Function#) in sequence starting from 0 to 3.
153 * Virtual Functions 0 and 2 belong to Physical Function 0
155 * Virtual Functions 1 and 3 belong to Physical Function 1
159 The above is an important consideration to take into account when targeting specific packets to a selected port.
161 For Intel® X710/XL710 Gigabit Ethernet Controller, queues are in pairs. One queue pair means one receive queue and
162 one transmit queue. The default number of queue pairs per VF is 4, and can be 16 in maximum.
164 Intel® 82599 10 Gigabit Ethernet Controller VF Infrastructure
165 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
167 The programmer can enable a maximum of *63 Virtual Functions* and there must be *one Physical Function* per Intel® 82599
168 10 Gigabit Ethernet Controller NIC port.
169 The reason for this is that the device allows for a maximum of 128 queues per port and a virtual/physical function has to
170 have at least one queue pair (RX/TX).
171 The current implementation of the DPDK ixgbevf driver supports a single queue pair (RX/TX) per Virtual Function.
172 The Physical Function in host could be either configured by the Linux* ixgbe driver
173 (in the case of the Linux Kernel-based Virtual Machine [KVM]) or by DPDK PMD PF driver.
174 When using both DPDK PMD PF/VF drivers, the whole NIC will be taken over by DPDK based application.
178 * Using Linux* ixgbe driver:
180 .. code-block:: console
182 rmmod ixgbe (To remove the ixgbe module)
183 insmod ixgbe max_vfs=2,2 (To enable two Virtual Functions per port)
185 * Using the DPDK PMD PF ixgbe driver:
187 Kernel Params: iommu=pt, intel_iommu=on
189 .. code-block:: console
193 ./dpdk-devbind.py -b igb_uio bb:ss.f
194 echo 2 > /sys/bus/pci/devices/0000\:bb\:ss.f/max_vfs (To enable two VFs on a specific PCI device)
196 Launch the DPDK testpmd/example or your own host daemon application using the DPDK PMD library.
198 * Using the DPDK PMD PF ixgbe driver to enable VF RSS:
200 Same steps as above to install the modules of uio, igb_uio, specify max_vfs for PCI device, and
201 launch the DPDK testpmd/example or your own host daemon application using the DPDK PMD library.
203 The available queue number (at most 4) per VF depends on the total number of pool, which is
204 determined by the max number of VF at PF initialization stage and the number of queue specified
207 * If the max number of VFs (max_vfs) is set in the range of 1 to 32:
209 If the number of Rx queues is specified as 4 (``--rxq=4`` in testpmd), then there are totally 32
210 pools (ETH_32_POOLS), and each VF could have 4 Rx queues;
212 If the number of Rx queues is specified as 2 (``--rxq=2`` in testpmd), then there are totally 32
213 pools (ETH_32_POOLS), and each VF could have 2 Rx queues;
215 * If the max number of VFs (max_vfs) is in the range of 33 to 64:
217 If the number of Rx queues in specified as 4 (``--rxq=4`` in testpmd), then error message is expected
218 as ``rxq`` is not correct at this case;
220 If the number of rxq is 2 (``--rxq=2`` in testpmd), then there is totally 64 pools (ETH_64_POOLS),
221 and each VF have 2 Rx queues;
223 On host, to enable VF RSS functionality, rx mq mode should be set as ETH_MQ_RX_VMDQ_RSS
224 or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated (max_vfs >= 1).
225 It also needs config VF RSS information like hash function, RSS key, RSS key length.
229 The limitation for VF RSS on Intel® 82599 10 Gigabit Ethernet Controller is:
230 The hash and key are shared among PF and all VF, the RETA table with 128 entries is also shared
231 among PF and all VF; So it could not to provide a method to query the hash and reta content per
232 VF on guest, while, if possible, please query them on host for the shared RETA information.
234 Virtual Function enumeration is performed in the following sequence by the Linux* pci driver for a dual-port NIC.
235 When you enable the four Virtual Functions with the above command, the four enabled functions have a Function#
236 represented by (Bus#, Device#, Function#) in sequence starting from 0 to 3.
239 * Virtual Functions 0 and 2 belong to Physical Function 0
241 * Virtual Functions 1 and 3 belong to Physical Function 1
245 The above is an important consideration to take into account when targeting specific packets to a selected port.
247 Intel® 82576 Gigabit Ethernet Controller and Intel® Ethernet Controller I350 Family VF Infrastructure
248 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
250 In a virtualized environment, an Intel® 82576 Gigabit Ethernet Controller serves up to eight virtual machines (VMs).
251 The controller has 16 TX and 16 RX queues.
252 They are generally referred to (or thought of) as queue pairs (one TX and one RX queue).
253 This gives the controller 16 queue pairs.
255 A pool is a group of queue pairs for assignment to the same VF, used for transmit and receive operations.
256 The controller has eight pools, with each pool containing two queue pairs, that is, two TX and two RX queues assigned to each VF.
258 In a virtualized environment, an Intel® Ethernet Controller I350 family device serves up to eight virtual machines (VMs) per port.
259 The eight queues can be accessed by eight different VMs if configured correctly (the i350 has 4x1GbE ports each with 8T X and 8 RX queues),
260 that means, one Transmit and one Receive queue assigned to each VF.
264 * Using Linux* igb driver:
266 .. code-block:: console
268 rmmod igb (To remove the igb module)
269 insmod igb max_vfs=2,2 (To enable two Virtual Functions per port)
271 * Using DPDK PMD PF igb driver:
273 Kernel Params: iommu=pt, intel_iommu=on modprobe uio
275 .. code-block:: console
278 ./dpdk-devbind.py -b igb_uio bb:ss.f
279 echo 2 > /sys/bus/pci/devices/0000\:bb\:ss.f/max_vfs (To enable two VFs on a specific pci device)
281 Launch DPDK testpmd/example or your own host daemon application using the DPDK PMD library.
283 Virtual Function enumeration is performed in the following sequence by the Linux* pci driver for a four-port NIC.
284 When you enable the four Virtual Functions with the above command, the four enabled functions have a Function#
285 represented by (Bus#, Device#, Function#) in sequence, starting from 0 to 7.
288 * Virtual Functions 0 and 4 belong to Physical Function 0
290 * Virtual Functions 1 and 5 belong to Physical Function 1
292 * Virtual Functions 2 and 6 belong to Physical Function 2
294 * Virtual Functions 3 and 7 belong to Physical Function 3
298 The above is an important consideration to take into account when targeting specific packets to a selected port.
300 Validated Hypervisors
301 ~~~~~~~~~~~~~~~~~~~~~
303 The validated hypervisor is:
305 * KVM (Kernel Virtual Machine) with Qemu, version 0.14.0
307 However, the hypervisor is bypassed to configure the Virtual Function devices using the Mailbox interface,
308 the solution is hypervisor-agnostic.
309 Xen* and VMware* (when SR- IOV is supported) will also be able to support the DPDK with Virtual Function driver support.
311 Expected Guest Operating System in Virtual Machine
312 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314 The expected guest operating systems in a virtualized environment are:
316 * Fedora* 14 (64-bit)
318 * Ubuntu* 10.04 (64-bit)
320 For supported kernel versions, refer to the *DPDK Release Notes*.
322 Setting Up a KVM Virtual Machine Monitor
323 ----------------------------------------
325 The following describes a target environment:
327 * Host Operating System: Fedora 14
329 * Hypervisor: KVM (Kernel Virtual Machine) with Qemu version 0.14.0
331 * Guest Operating System: Fedora 14
333 * Linux Kernel Version: Refer to the *DPDK Getting Started Guide*
335 * Target Applications: l2fwd, l3fwd-vf
337 The setup procedure is as follows:
339 #. Before booting the Host OS, open **BIOS setup** and enable **Intel® VT features**.
341 #. While booting the Host OS kernel, pass the intel_iommu=on kernel command line argument using GRUB.
342 When using DPDK PF driver on host, pass the iommu=pt kernel command line argument in GRUB.
344 #. Download qemu-kvm-0.14.0 from
345 `http://sourceforge.net/projects/kvm/files/qemu-kvm/ <http://sourceforge.net/projects/kvm/files/qemu-kvm/>`_
346 and install it in the Host OS using the following steps:
348 When using a recent kernel (2.6.25+) with kvm modules included:
350 .. code-block:: console
352 tar xzf qemu-kvm-release.tar.gz
354 ./configure --prefix=/usr/local/kvm
357 sudo /sbin/modprobe kvm-intel
359 When using an older kernel, or a kernel from a distribution without the kvm modules,
360 you must download (from the same link), compile and install the modules yourself:
362 .. code-block:: console
364 tar xjf kvm-kmod-release.tar.bz2
369 sudo /sbin/modprobe kvm-intel
371 qemu-kvm installs in the /usr/local/bin directory.
373 For more details about KVM configuration and usage, please refer to:
375 `http://www.linux-kvm.org/page/HOWTO1 <http://www.linux-kvm.org/page/HOWTO1>`_.
377 #. Create a Virtual Machine and install Fedora 14 on the Virtual Machine.
378 This is referred to as the Guest Operating System (Guest OS).
380 #. Download and install the latest ixgbe driver from:
382 `http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=14687 <http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=14687>`_
386 When using Linux kernel ixgbe driver, unload the Linux ixgbe driver and reload it with the max_vfs=2,2 argument:
388 .. code-block:: console
391 modprobe ixgbe max_vfs=2,2
393 When using DPDK PMD PF driver, insert DPDK kernel module igb_uio and set the number of VF by sysfs max_vfs:
395 .. code-block:: console
399 ./dpdk-devbind.py -b igb_uio 02:00.0 02:00.1 0e:00.0 0e:00.1
400 echo 2 > /sys/bus/pci/devices/0000\:02\:00.0/max_vfs
401 echo 2 > /sys/bus/pci/devices/0000\:02\:00.1/max_vfs
402 echo 2 > /sys/bus/pci/devices/0000\:0e\:00.0/max_vfs
403 echo 2 > /sys/bus/pci/devices/0000\:0e\:00.1/max_vfs
407 You need to explicitly specify number of vfs for each port, for example,
408 in the command above, it creates two vfs for the first two ixgbe ports.
410 Let say we have a machine with four physical ixgbe ports:
421 The command above creates two vfs for device 0000:02:00.0:
423 .. code-block:: console
425 ls -alrt /sys/bus/pci/devices/0000\:02\:00.0/virt*
426 lrwxrwxrwx. 1 root root 0 Apr 13 05:40 /sys/bus/pci/devices/0000:02:00.0/virtfn1 -> ../0000:02:10.2
427 lrwxrwxrwx. 1 root root 0 Apr 13 05:40 /sys/bus/pci/devices/0000:02:00.0/virtfn0 -> ../0000:02:10.0
429 It also creates two vfs for device 0000:02:00.1:
431 .. code-block:: console
433 ls -alrt /sys/bus/pci/devices/0000\:02\:00.1/virt*
434 lrwxrwxrwx. 1 root root 0 Apr 13 05:51 /sys/bus/pci/devices/0000:02:00.1/virtfn1 -> ../0000:02:10.3
435 lrwxrwxrwx. 1 root root 0 Apr 13 05:51 /sys/bus/pci/devices/0000:02:00.1/virtfn0 -> ../0000:02:10.1
437 #. List the PCI devices connected and notice that the Host OS shows two Physical Functions (traditional ports)
438 and four Virtual Functions (two for each port).
439 This is the result of the previous step.
441 #. Insert the pci_stub module to hold the PCI devices that are freed from the default driver using the following command
442 (see http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM Section 4 for more information):
444 .. code-block:: console
446 sudo /sbin/modprobe pci-stub
448 Unbind the default driver from the PCI devices representing the Virtual Functions.
449 A script to perform this action is as follows:
451 .. code-block:: console
453 echo "8086 10ed" > /sys/bus/pci/drivers/pci-stub/new_id
454 echo 0000:08:10.0 > /sys/bus/pci/devices/0000:08:10.0/driver/unbind
455 echo 0000:08:10.0 > /sys/bus/pci/drivers/pci-stub/bind
457 where, 0000:08:10.0 belongs to the Virtual Function visible in the Host OS.
459 #. Now, start the Virtual Machine by running the following command:
461 .. code-block:: console
463 /usr/local/kvm/bin/qemu-system-x86_64 -m 4096 -smp 4 -boot c -hda lucid.qcow2 -device pci-assign,host=08:10.0
467 — -m = memory to assign
469 — -smp = number of smp cores
471 — -boot = boot option
473 — -hda = virtual disk image
475 — -device = device to attach
479 — The pci-assign,host=08:10.0 value indicates that you want to attach a PCI device
480 to a Virtual Machine and the respective (Bus:Device.Function)
481 numbers should be passed for the Virtual Function to be attached.
483 — qemu-kvm-0.14.0 allows a maximum of four PCI devices assigned to a VM,
484 but this is qemu-kvm version dependent since qemu-kvm-0.14.1 allows a maximum of five PCI devices.
486 — qemu-system-x86_64 also has a -cpu command line option that is used to select the cpu_model
487 to emulate in a Virtual Machine. Therefore, it can be used as:
489 .. code-block:: console
491 /usr/local/kvm/bin/qemu-system-x86_64 -cpu ?
493 (to list all available cpu_models)
495 /usr/local/kvm/bin/qemu-system-x86_64 -m 4096 -cpu host -smp 4 -boot c -hda lucid.qcow2 -device pci-assign,host=08:10.0
497 (to use the same cpu_model equivalent to the host cpu)
499 For more information, please refer to: `http://wiki.qemu.org/Features/CPUModels <http://wiki.qemu.org/Features/CPUModels>`_.
501 #. Install and run DPDK host app to take over the Physical Function. Eg.
503 .. code-block:: console
505 make install T=x86_64-native-linuxapp-gcc
506 ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 -- -i
508 #. Finally, access the Guest OS using vncviewer with the localhost:5900 port and check the lspci command output in the Guest OS.
509 The virtual functions will be listed as available for use.
511 #. Configure and install the DPDK with an x86_64-native-linuxapp-gcc configuration on the Guest OS as normal,
512 that is, there is no change to the normal installation procedure.
514 .. code-block:: console
516 make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
517 cd x86_64-native-linuxapp-gcc
522 If you are unable to compile the DPDK and you are getting "error: CPU you selected does not support x86-64 instruction set",
523 power off the Guest OS and start the virtual machine with the correct -cpu option in the qemu- system-x86_64 command as shown in step 9.
524 You must select the best x86_64 cpu_model to emulate or you can select host option if available.
528 Run the DPDK l2fwd sample application in the Guest OS with Hugepages enabled.
529 For the expected benchmark performance, you must pin the cores from the Guest OS to the Host OS (taskset can be used to do this) and
530 you must also look at the PCI Bus layout on the board to ensure you are not running the traffic over the QPI Interface.
534 * The Virtual Machine Manager (the Fedora package name is virt-manager) is a utility for virtual machine management
535 that can also be used to create, start, stop and delete virtual machines.
536 If this option is used, step 2 and 6 in the instructions provided will be different.
538 * virsh, a command line utility for virtual machine management,
539 can also be used to bind and unbind devices to a virtual machine in Ubuntu.
540 If this option is used, step 6 in the instructions provided will be different.
542 * The Virtual Machine Monitor (see :numref:`figure_perf_benchmark`) is equivalent to a Host OS with KVM installed as described in the instructions.
544 .. _figure_perf_benchmark:
546 .. figure:: img/perf_benchmark.*
548 Performance Benchmark Setup
551 DPDK SR-IOV PMD PF/VF Driver Usage Model
552 ----------------------------------------
554 Fast Host-based Packet Processing
555 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
557 Software Defined Network (SDN) trends are demanding fast host-based packet handling.
558 In a virtualization environment,
559 the DPDK VF PMD driver performs the same throughput result as a non-VT native environment.
561 With such host instance fast packet processing, lots of services such as filtering, QoS,
562 DPI can be offloaded on the host fast path.
564 :numref:`figure_fast_pkt_proc` shows the scenario where some VMs directly communicate externally via a VFs,
565 while others connect to a virtual switch and share the same uplink bandwidth.
567 .. _figure_fast_pkt_proc:
569 .. figure:: img/fast_pkt_proc.*
571 Fast Host-based Packet Processing
574 SR-IOV (PF/VF) Approach for Inter-VM Communication
575 --------------------------------------------------
577 Inter-VM data communication is one of the traffic bottle necks in virtualization platforms.
578 SR-IOV device assignment helps a VM to attach the real device, taking advantage of the bridge in the NIC.
579 So VF-to-VF traffic within the same physical port (VM0<->VM1) have hardware acceleration.
580 However, when VF crosses physical ports (VM0<->VM2), there is no such hardware bridge.
581 In this case, the DPDK PMD PF driver provides host forwarding between such VMs.
583 :numref:`figure_inter_vm_comms` shows an example.
584 In this case an update of the MAC address lookup tables in both the NIC and host DPDK application is required.
586 In the NIC, writing the destination of a MAC address belongs to another cross device VM to the PF specific pool.
587 So when a packet comes in, its destination MAC address will match and forward to the host DPDK PMD application.
589 In the host DPDK application, the behavior is similar to L2 forwarding,
590 that is, the packet is forwarded to the correct PF pool.
591 The SR-IOV NIC switch forwards the packet to a specific VM according to the MAC destination address
592 which belongs to the destination VF on the VM.
594 .. _figure_inter_vm_comms:
596 .. figure:: img/inter_vm_comms.*
598 Inter-VM Communication