2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
34 This chapter describes the packages required to compile the DPDK.
38 If the DPDK is being used on an Intel® Communications Chipset 89xx Series platform,
39 please consult the *Intel® Communications Chipset 89xx Series Software for Linux* Getting Started Guide*.
41 BIOS Setting Prerequisite
42 -------------------------
44 For the majority of platforms, no special BIOS settings are needed to use basic DPDK functionality.
45 However, for additional HPET timer and power management functionality,
46 and high performance of small packets on 40G NIC, BIOS setting changes may be needed.
47 Consult :ref:`Chapter 5. Enabling Additional Functionality <Enabling_Additional_Functionality>`
48 for more information on the required changes.
50 Compilation of the DPDK
51 -----------------------
57 Testing has been performed using Fedora* 18. The setup commands and installed packages needed on other systems may be different.
58 For details on other Linux distributions and the versions tested, please consult the DPDK Release Notes.
62 * coreutils: cmp, sed, grep, arch
64 * gcc: versions 4.5.x or later is recommended.
65 On some distributions, some specific compiler flags and linker flags are enabled by default and
66 affect performance (- fstack-protector, for example).
67 Please refer to the documentation of your distribution and to gcc -dumpspecs.
69 * libc headers (glibc-devel.i686 / libc6-dev-i386; glibc-devel.x86_64 for 64-bit compilation)
71 * Linux kernel headers or sources required to build kernel modules. (kernel- devel.x86_64)
73 * Additional packages required for 32-bit compilation on 64-bit systems are:
75 glibc.i686, libgcc.i686, libstdc++.i686 and glibc-devel.i686
77 * Python, version 2.6 or 2.7, to use various helper scripts included in the DPDK package
82 * Intel® C++ Compiler (icc). For installation, additional libraries may be required.
83 See the icc Installation Guide found in the Documentation directory under the compiler installation.
84 This release has been tested using version 12.1.
86 * libpcap headers and libraries (libpcap-devel) to compile and use the libpcap-based poll-mode driver.
87 This driver is disabled by default and can be enabled by setting CONFIG_RTE_LIBRTE_PMD_PCAP=y in the build time config file.
89 Running DPDK Applications
90 -------------------------
92 To run an DPDK application, some customization may be required on the target machine.
99 * Kernel version >= 2.6.33
101 The kernel version in use can be checked using the command:
103 .. code-block:: console
107 For details of the patches needed to use the DPDK with earlier kernel versions,
108 see the DPDK FAQ included in the *DPDK Release Notes*.
109 Note also that Redhat* Linux* 6.2 and 6.3 uses a 2.6.32 kernel that already has all the necessary patches applied.
111 * glibc >= 2.7 (for features related to cpuset)
113 The version can be checked using the ldd --version command. A sample output is shown below:
115 .. code-block:: console
119 ldd (GNU libc) 2.14.90
120 Copyright (C) 2011 Free Software Foundation, Inc.
121 This is free software; see the source for copying conditions. There is NO
122 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
123 Written by Roland McGrath and Ulrich Drepper.
125 * Kernel configuration
127 In the Fedora* OS and other common distributions, such as Ubuntu*, or RedHat Enterprise Linux*,
128 the vendor supplied kernel configurations can be used to run most DPDK applications.
130 For other kernel builds, options which should be enabled for DPDK include:
136 * PROC_PAGE_MONITOR support
138 * HPET and HPET_MMAP configuration options should also be enabled if HPET support is required.
139 See :ref:`Section 5.1 High Precision Event Timer (HPET) Functionality <High_Precision_Event_Timer>` for more details.
141 Use of Hugepages in the Linux* Environment
142 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
144 Hugepage support is required for the large memory pool allocation used for packet buffers
145 (the HUGETLBFS option must be enabled in the running kernel as indicated in Section 2.3).
146 By using hugepage allocations, performance is increased since fewer pages are needed,
147 and therefore less Translation Lookaside Buffers (TLBs, high speed translation caches),
148 which reduce the time it takes to translate a virtual page address to a physical page address.
149 Without hugepages, high TLB miss rates would occur with the standard 4k page size, slowing performance.
151 Reserving Hugepages for DPDK Use
152 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
154 The allocation of hugepages should be done at boot time or as soon as possible after system boot
155 to prevent memory from being fragmented in physical memory.
156 To reserve hugepages at boot time, a parameter is passed to the Linux* kernel on the kernel command line.
158 For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:
160 .. code-block:: console
164 For other hugepage sizes, for example 1G pages, the size must be specified explicitly and
165 can also be optionally set as the default hugepage size for the system.
166 For example, to reserve 4G of hugepage memory in the form of four 1G pages, the following options should be passed to the kernel:
168 .. code-block:: console
170 default_hugepagesz=1G hugepagesz=1G hugepages=4
174 The hugepage sizes that a CPU supports can be determined from the CPU flags.
175 If pse exists, 2M hugepages are supported; if pdpe1gb exists, 1G hugepages are supported.
179 For 64-bit applications, it is recommended to use 1 GB hugepages if the platform supports them.
181 In the case of a dual-socket NUMA system,
182 the number of hugepages reserved at boot time is generally divided equally between the two sockets
183 (on the assumption that sufficient memory is present on both sockets).
185 See the Documentation/kernel-parameters.txt file in your Linux* source tree for further details of these and other kernel options.
189 For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
190 This is done by echoing the number of hugepages required to a nr_hugepages file in the /sys/devices/ directory.
191 For a single-node system, the command to use is as follows (assuming that 1024 pages are required):
193 .. code-block:: console
195 echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
197 On a NUMA machine, pages should be allocated explicitly on separate nodes:
199 .. code-block:: console
201 echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
202 echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
206 For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
208 Using Hugepages with the DPDK
209 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
211 Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:
213 .. code-block:: console
216 mount -t hugetlbfs nodev /mnt/huge
218 The mount point can be made permanent across reboots, by adding the following line to the /etc/fstab file:
220 .. code-block:: console
222 nodev /mnt/huge hugetlbfs defaults 0 0
224 For 1GB pages, the page size must be specified as a mount option:
226 .. code-block:: console
228 nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
230 Xen Domain0 Support in the Linux* Environment
231 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
233 The existing memory management implementation is based on the Linux* kernel hugepage mechanism.
234 On the Xen hypervisor, hugepage support for DomainU (DomU) Guests means that DPDK applications work as normal for guests.
236 However, Domain0 (Dom0) does not support hugepages.
237 To work around this limitation, a new kernel module rte_dom0_mm is added to facilitate the allocation and mapping of memory via
238 **IOCTL** (allocation) and **MMAP** (mapping).
240 Enabling Xen Dom0 Mode in the DPDK
241 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
243 By default, Xen Dom0 mode is disabled in the DPDK build configuration files.
244 To support Xen Dom0, the CONFIG_RTE_LIBRTE_XEN_DOM0 setting should be changed to “y”, which enables the Xen Dom0 mode at compile time.
246 Furthermore, the CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID setting should also be changed to “y” in the case of the wrong socket ID being received.
248 Loading the DPDK rte_dom0_mm Module
249 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
251 To run any DPDK application on Xen Dom0, the rte_dom0_mm module must be loaded into the running kernel with rsv_memsize option.
252 The module is found in the kmod sub-directory of the DPDK target directory.
253 This module should be loaded using the insmod command as shown below (assuming that the current directory is the DPDK target directory):
255 .. code-block:: console
257 sudo insmod kmod/rte_dom0_mm.ko rsv_memsize=X
259 The value X cannot be greater than 4096(MB).
261 Configuring Memory for DPDK Use
262 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
264 After the rte_dom0_mm.ko kernel module has been loaded, the user must configure the memory size for DPDK usage.
265 This is done by echoing the memory size to a memsize file in the /sys/devices/ directory.
266 Use the following command (assuming that 2048 MB is required):
268 .. code-block:: console
270 echo 2048 > /sys/kernel/mm/dom0-mm/memsize-mB/memsize
272 The user can also check how much memory has already been used:
274 .. code-block:: console
276 cat /sys/kernel/mm/dom0-mm/memsize-mB/memsize_rsvd
278 Xen Domain0 does not support NUMA configuration, as a result the --socket-mem command line option is invalid for Xen Domain0.
282 The memsize value cannot be greater than the rsv_memsize value.
284 Running the DPDK Application on Xen Domain0
285 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
287 To run the DPDK application on Xen Domain0, an extra command line option --xen-dom0 is required.