2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 Compiling the Intel® DPDK Target from Source
32 ============================================
36 Parts of this process can also be done using the setup script described in Chapter 6 of this document.
38 Install the Intel® DPDK and Browse Sources
39 ------------------------------------------
41 First, uncompress the archive and move to the uncompressed Intel® DPDK source directory:
43 .. code-block:: console
45 user@host:~$ unzip DPDK-<version>.zip
46 user@host:~$ cd DPDK-<version>
47 user@host:~/DPDK-<version>$ ls
48 app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/
50 The Intel® DPDK is composed of several directories:
52 * lib: Source code of Intel® DPDK libraries
54 * app: Source code of Intel® DPDK applications (automatic tests)
56 * examples: Source code of Intel® DPDK application examples
58 * config, tools, scripts, mk: Framework-related makefiles, scripts and configuration
60 Installation of Intel® DPDK Target Environments
61 -----------------------------------------------
63 The format of a Intel® DPDK target is:
65 ARCH-MACHINE-EXECENV-TOOLCHAIN
69 * ARCH can be: i686, x86_64
71 * MACHINE can be: native, ivshmem
73 * EXECENV can be: linuxapp, bsdapp
75 * TOOLCHAIN can be: gcc, icc
77 The targets to be installed depend on the 32-bit and/or 64-bit packages and compilers installed on the host.
78 Available targets can be found in the DPDK/config directory.
79 The defconfig\_ prefix should not be used.
83 Configuration files are provided with the RTE_MACHINE optimization level set.
84 Within the configuration files, the RTE_MACHINE configuration value is set to native,
85 which means that the compiled software is tuned for the platform on which it is built.
86 For more information on this setting, and its possible values, see the *Intel® DPDK Programmers Guide*.
88 When using the Intel® C++ Compiler (icc), one of the following commands should be invoked for 64-bit or 32-bit use respectively.
89 Notice that the shell scripts update the $PATH variable and therefore should not be performed in the same session.
90 Also, verify the compiler's installation directory since the path may be different:
92 .. code-block:: console
94 source /opt/intel/bin/iccvars.sh intel64
95 source /opt/intel/bin/iccvars.sh ia32
97 To install and make targets, use the make install T=<target> command in the top-level Intel® DPDK directory.
99 For example, to compile a 64-bit target using icc, run:
101 .. code-block:: console
103 make install T=x86_64-native-linuxapp-icc
105 To compile a 32-bit build using gcc, the make command should be:
107 .. code-block:: console
109 make install T=i686-native-linuxapp-gcc
111 To compile all 64-bit targets using gcc, use:
113 .. code-block:: console
115 make install T=x86_64*gcc
117 To compile all 64-bit targets using both gcc and icc, use:
119 .. code-block:: console
121 make install T=x86_64-*
125 The wildcard operator (*) can be used to create multiple targets at the same time.
127 To prepare a target without building it, for example, if the configuration changes need to be made before compilation,
128 use the make config T=<target> command:
130 .. code-block:: console
132 make config T=x86_64-native-linuxapp-gcc
136 The igb_uio module must be compiled with the same kernel as the one running on the target.
137 If the Intel® DPDK is not being built on the target machine,
138 the RTE_KERNELDIR environment variable should be used to point the compilation at a copy of the kernel version to be used on the target machine.
140 Once the target environment is created, the user may move to the target environment directory and continue to make code changes and re-compile.
141 The user may also make modifications to the compile-time Intel® DPDK configuration by editing the .config file in the build directory.
142 (This is a build-local copy of the defconfig file from the top- level config directory).
144 .. code-block:: console
146 cd x86_64-native-linuxapp-gcc
150 In addition, the make clean command can be used to remove any existing compiled files for a subsequent full, clean rebuild of the code.
152 Browsing the Installed Intel® DPDK Environment Target
153 -----------------------------------------------------
155 Once a target is created it contains all libraries and header files for the Intel® DPDK environment that are required to build customer applications.
156 In addition, the test and testpmd applications are built under the build/app directory, which may be used for testing.
157 In the case of Linux, a kmod directory is also present that contains a module to install:
159 .. code-block:: console
161 $ ls x86_64-native-linuxapp-gcc
162 app build hostapp include kmod lib Makefile
164 Loading the Intel® DPDK igb_uio Module
165 --------------------------------------
167 To run any Intel® DPDK application, the igb_uio module can be loaded into the running kernel.
168 The module is found in the kmod sub-directory of the Intel® DPDK target directory.
169 This module should be loaded using the insmod command as shown below (assuming that the current directory is the Intel® DPDK target directory).
170 In many cases, the uio support in the Linux* kernel is compiled as a module rather than as part of the kernel,
171 so it is often necessary to load the uio module first:
173 .. code-block:: console
176 sudo insmod kmod/igb_uio.ko
178 Since Intel® DPDK release 1.7 provides VFIO support, compilation and use of igb_uio module has become optional for platforms that support using VFIO.
183 To run an Intel® DPDK application and make use of VFIO, the vfio-pci module must be loaded:
185 .. code-block:: console
187 sudo modprobe vfio-pci
189 Note that in order to use VFIO, your kernel must support it.
190 VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default,
191 however please consult your distributions documentation to make sure that is the case.
193 Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d).
195 For proper operation of VFIO when running Intel® DPDK applications as a non-privileged user, correct permissions should also be set up.
196 This can be done by using the Intel® DPDK setup script (called setup.sh and located in the tools directory).
198 Binding and Unbinding Network Ports to/from the igb_uioor VFIO Modules
199 ----------------------------------------------------------------------
201 As of release 1.4, Intel® DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use.
202 Instead, all ports that are to be used by an Intel® DPDK application must be bound to the igb_uio or vfio-pci module before the application is run.
203 Any network ports under Linux* control will be ignored by the Intel® DPDK poll-mode drivers and cannot be used by the application.
207 The Intel® DPDK will, by default, no longer automatically unbind network ports from the kernel driver at startup.
208 Any ports to be used by an Intel® DPDK application must be unbound from Linux* control and bound to the igb_uio or vfio-pci module before the application is run.
210 To bind ports to the igb_uio or vfio-pci module for Intel® DPDK use, and then subsequently return ports to Linux* control,
211 a utility script called dpdk_nic _bind.py is provided in the tools subdirectory.
212 This utility can be used to provide a view of the current state of the network ports on the system,
213 and to bind and unbind those ports from the different kernel modules, including igb_uio and vfio-pci.
214 The following are some examples of how the script can be used.
215 A full description of the script and its parameters can be obtained by calling the script with the --help or --usage options.
219 Due to the way VFIO works, there are certain limitations to which devices can be used with VFIO.
220 Mainly it comes down to how IOMMU groups work.
221 Any Virtual Function device can be used with VFIO on its own, but physical devices will require either all ports bound to VFIO,
222 or some of them bound to VFIO while others not being bound to anything at all.
224 If your device is behind a PCI-to-PCI bridge, the bridge will then be part of the IOMMU group in which your device is in.
225 Therefore, the bridge driver should also be unbound from the bridge PCI device for VFIO to work with devices behind the bridge.
229 While any user can run the dpdk_nic_bind.py script to view the status of the network ports,
230 binding or unbinding network ports requires root privileges.
232 To see the status of all network ports on the system:
234 .. code-block:: console
236 root@host:DPDK# ./tools/dpdk_nic_bind.py --status
238 Network devices using IGB_UIO driver
239 ====================================
240 0000:82:00.0 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe
241 0000:82:00.1 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe
243 Network devices using kernel driver
244 ===================================
245 0000:04:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio *Active*
246 0000:04:00.1 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio
247 0000:04:00.2 'I350 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio
248 0000:04:00.3 'I350 Gigabit Network Connection' if=eth3 drv=igb unused=igb_uio
250 Other network devices
251 =====================
254 To bind device eth1, 04:00.1, to the igb_uio driver:
256 .. code-block:: console
258 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=igb_uio 04:00.1
262 .. code-block:: console
264 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=igb_uio eth1
266 To restore device 82:00.0 to its original kernel binding:
268 .. code-block:: console
270 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=ixgbe 82:00.0