2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 .. _building_from_source:
33 Compiling the DPDK Target from Source
34 =====================================
38 Testing has been performed using FreeBSD* 10.0-RELEASE (x86_64) and requires the
39 installation of the kernel sources, which should be included during the
40 installation of FreeBSD*. The DPDK also requires the use of FreeBSD*
41 ports to compile and function.
46 The DPDK and its applications require the GNU make system (gmake)
47 to build on FreeBSD*. Optionally, gcc may also be used in place of clang
48 to build the DPDK, in which case it too must be installed prior to
49 compiling the DPDK. The installation of these tools is covered in this
52 Compiling the DPDK requires the FreeBSD kernel sources, which should be
53 included during the installation of FreeBSD* on the development platform.
54 The DPDK also requires the use of FreeBSD* ports to compile and function.
56 To use the FreeBSD* ports system, it is required to update and extract the FreeBSD*
57 ports tree by issuing the following commands:
59 .. code-block:: console
61 root@host:~ # portsnap fetch
62 root@host:~ # portsnap extract
64 If the environment requires proxies for external communication, these can be set
67 .. code-block:: console
69 root@host:~ # setenv http_proxy <my_proxy_host>:<port>
70 root@host:~ # setenv ftp_proxy <my_proxy_host>:<port>
72 The FreeBSD* ports below need to be installed prior to building the DPDK.
73 In general these can be installed using the following set of commands:
75 #. cd /usr/ports/<port_location>
77 #. make config-recursive
83 Each port location can be found using:
85 .. code-block:: console
87 user@host:~ # whereis <port_name>
89 The ports required and their locations are as follows:
92 /usr/ports/ports-mgmt/dialog4ports
95 /usr/ports/devel/gmake
98 /usr/ports/sysutils/coreutils
100 For compiling and using the DPDK with gcc, it too must be installed
101 from the ports collection:
103 gcc: version 4.8 is recommended
104 /usr/ports/lang/gcc48
105 (Ensure that CPU_OPTS is selected (default is OFF))
107 When running the make config-recursive command, a dialog may be presented to the
108 user. For the installation of the DPDK, the default options were used.
112 To avoid multiple dialogs being presented to the user during make install,
113 it is advisable before running the make install command to re-run the
114 make config -recursive command until no more dialogs are seen.
117 Install the DPDK and Browse Sources
118 -----------------------------------
120 First, uncompress the archive and move to the DPDK source directory:
122 .. code-block:: console
124 user@host:~ # unzip DPDK-<version>zip
125 user@host:~ # cd DPDK-<version>
126 user@host:~/DPDK # ls
127 app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/
129 The DPDK is composed of several directories:
131 * lib: Source code of DPDK libraries
133 * app: Source code of DPDK applications (automatic tests)
135 * examples: Source code of DPDK applications
137 * config, tools, scripts, mk: Framework-related makefiles, scripts and configuration
139 Installation of the DPDK Target Environments
140 --------------------------------------------
142 The format of a DPDK target is:
144 ARCH-MACHINE-EXECENV-TOOLCHAIN
154 * TOOLCHAIN is: gcc | clang
156 The configuration files for the DPDK targets can be found in the DPDK/config
157 directory in the form of:
161 defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN
165 Configuration files are provided with the RTE_MACHINE optimization level set.
166 Within the configuration files, the RTE_MACHINE configuration value is set
167 to native, which means that the compiled software is tuned for the platform
168 on which it is built. For more information on this setting, and its
169 possible values, see the *DPDK Programmers Guide*.
171 To install and make the target, use "gmake install T=<target>".
173 For example to compile for FreeBSD* use:
175 .. code-block:: console
177 gmake install T=x86_64-native-bsdapp-clang
181 If the compiler binary to be used does not correspond to that given in the
182 TOOLCHAIN part of the target, the compiler command may need to be explicitly
183 specified. For example, if compiling for gcc, where the gcc binary is called
184 gcc4.8, the command would need to be "gmake install T=<target> CC=gcc4.8".
186 Browsing the Installed DPDK Environment Target
187 ----------------------------------------------
189 Once a target is created, it contains all the libraries and header files for the
190 DPDK environment that are required to build customer applications.
191 In addition, the test and testpmd applications are built under the build/app
192 directory, which may be used for testing. A kmod directory is also present that
193 contains the kernel modules to install:
195 .. code-block:: console
197 user@host:~/DPDK # ls x86_64-native-bsdapp-gcc
198 app build hostapp include kmod lib Makefile
201 .. _loading_contigmem:
203 Loading the DPDK contigmem Module
204 ---------------------------------
206 To run a DPDK application, physically contiguous memory is required.
207 In the absence of non-transparent superpages, the included sources for the
208 contigmem kernel module provides the ability to present contiguous blocks of
209 memory for the DPDK to use. The contigmem module must be loaded into the
210 running kernel before any DPDK is run. The module is found in the kmod
211 sub-directory of the DPDK target directory.
213 The amount of physically contiguous memory along with the number of physically
214 contiguous blocks to be reserved by the module can be set at runtime prior to
215 module loading using:
217 .. code-block:: console
219 root@host:~ # kenv hw.contigmem.num_buffers=n
220 root@host:~ # kenv hw.contigmem.buffer_size=m
222 The kernel environment variables can also be specified during boot by placing the
223 following in /boot/loader.conf:
227 hw.contigmem.num_buffers=n hw.contigmem.buffer_size=m
229 The variables can be inspected using the following command:
231 .. code-block:: console
233 root@host:~ # sysctl -a hw.contigmem
235 Where n is the number of blocks and m is the size in bytes of each area of
236 contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte)
237 each is set during module load if they are not specified in the environment.
239 The module can then be loaded using kldload (assuming that the current directory
240 is the DPDK target directory):
242 .. code-block:: console
244 kldload ./kmod/contigmem.ko
246 It is advisable to include the loading of the contigmem module during the boot
247 process to avoid issues with potential memory fragmentation during later system
248 up time. This can be achieved by copying the module to the /boot/kernel/
249 directory and placing the following into /boot/loader.conf:
257 The contigmem_load directive should be placed after any definitions of
258 hw.contigmem.num_buffers and hw.contigmem.buffer_size if the default values
263 .. code-block:: console
265 kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko: Exec format error
267 is generally attributed to not having enough contiguous memory
268 available and can be verified via dmesg or /var/log/messages:
270 .. code-block:: console
272 kernel: contigmalloc failed for buffer <n>
274 To avoid this error, reduce the number of buffers or the buffer size.
278 Loading the DPDK nic_uio Module
279 -------------------------------
281 After loading the contigmem module, the nic_uio must also be loaded into the
282 running kernel prior to running any DPDK application. This module must
283 be loaded using the kldload command as shown below (assuming that the current
284 directory is the DPDK target directory).
286 .. code-block:: console
288 kldload ./kmod/nic_uio.ko
292 If the ports to be used are currently bound to a existing kernel driver
293 then the hw.nic_uio.bdfs sysctl value will need to be set before loading the
294 module. Setting this value is described in the next section below.
296 Currently loaded modules can be seen by using the "kldstat" command and a module
297 can be removed from the running kernel by using "kldunload <module_name>".
299 To load the module during boot, copy the nic_uio module to /boot/kernel
300 and place the following into /boot/loader.conf:
308 nic_uio_load="YES" must appear after the contigmem_load directive, if it exists.
310 By default, the nic_uio module will take ownership of network ports if they are
311 recognized DPDK devices and are not owned by another module. However, since
312 the FreeBSD kernel includes support, either built-in, or via a separate driver
313 module, for most network card devices, it is likely that the ports to be used are
314 already bound to a driver other than nic_uio. The following sub-section describe
315 how to query and modify the device ownership of the ports to be used by
318 .. _binding_network_ports:
320 Binding Network Ports to the nic_uio Module
321 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
323 Device ownership can be viewed using the pciconf -l command. The example below shows
324 four IntelĀ® 82599 network ports under "if_ixgbe" module ownership.
326 .. code-block:: console
328 user@host:~ # pciconf -l
329 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
330 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
331 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
332 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
334 The first column constitutes three components:
340 #. Selector (Bus:Device:Function): 1:0:0
342 Where no driver is associated with a device, the device name will be none.
344 By default, the FreeBSD* kernel will include built-in drivers for the most common
345 devices; a kernel rebuild would normally be required to either remove the drivers
346 or configure them as loadable modules.
348 To avoid building a custom kernel, the nic_uio module can detach a network port
349 from its current device driver. This is achieved by setting the hw.nic_uio.bdfs
350 kernel environment variable prior to loading nic_uio, as follows:
354 hw.nic_uio.bdfs="b:d:f,b:d:f,..."
356 Where a comma separated list of selectors is set, the list must not contain any
359 For example to re-bind "ix2\@pci0:2:0:0" and "ix3\@pci0:2:0:1" to the nic_uio module
360 upon loading, use the following command:
362 .. code-block:: console
364 kenv hw.nic_uio.bdfs="2:0:0,2:0:1"
366 The variable can also be specified during boot by placing the following into
367 "/boot/loader.conf", before the previously-described "nic_uio_load" line - as
372 hw.nic_uio.bdfs="2:0:0,2:0:1"
375 Binding Network Ports Back to their Original Kernel Driver
376 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
378 If the original driver for a network port has been compiled into the kernel,
379 it is necessary to reboot FreeBSD* to restore the original device binding. Before
380 doing so, update or remove the "hw.nic_uio.bdfs" in "/boot/loader.conf".
382 If rebinding to a driver that is a loadable module, the network port binding can
383 be reset without rebooting. To do so, unload both the target kernel module and the
384 nic_uio module, modify or clear the "hw.nic_uio.bdfs" kernel environment (kenv)
385 value, and reload the two drivers - first the original kernel driver, and then
386 the nic_uio driver. [The latter does not need to be reloaded unless there are
387 ports that are still to be bound to it].
389 Example commands to perform these steps are shown below:
391 .. code-block:: console
394 kldunload <original_driver>
396 kenv -u hw.nic_uio.bdfs # to clear the value completely
398 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." # to update the list of ports to bind
400 kldload <original_driver>
402 kldload nic_uio # optional