2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 .. _building_from_source:
33 Compiling the DPDK Target from Source
34 =====================================
39 The DPDK and its applications require the GNU make system (gmake)
40 to build on FreeBSD. Optionally, gcc may also be used in place of clang
41 to build the DPDK, in which case it too must be installed prior to
42 compiling the DPDK. The installation of these tools is covered in this
45 Compiling the DPDK requires the FreeBSD kernel sources, which should be
46 included during the installation of FreeBSD on the development platform.
47 The DPDK also requires the use of FreeBSD ports to compile and function.
49 To use the FreeBSD ports system, it is required to update and extract the FreeBSD
50 ports tree by issuing the following commands:
52 .. code-block:: console
57 If the environment requires proxies for external communication, these can be set
60 .. code-block:: console
62 setenv http_proxy <my_proxy_host>:<port>
63 setenv ftp_proxy <my_proxy_host>:<port>
65 The FreeBSD ports below need to be installed prior to building the DPDK.
66 In general these can be installed using the following set of commands::
68 cd /usr/ports/<port_location>
76 Each port location can be found using::
80 The ports required and their locations are as follows:
82 * dialog4ports: ``/usr/ports/ports-mgmt/dialog4ports``
84 * GNU make(gmake): ``/usr/ports/devel/gmake``
86 * coreutils: ``/usr/ports/sysutils/coreutils``
88 For compiling and using the DPDK with gcc, the compiler must be installed
89 from the ports collection:
91 * gcc: version 4.8 is recommended ``/usr/ports/lang/gcc48``.
92 Ensure that ``CPU_OPTS`` is selected (default is OFF).
94 When running the make config-recursive command, a dialog may be presented to the
95 user. For the installation of the DPDK, the default options were used.
99 To avoid multiple dialogs being presented to the user during make install,
100 it is advisable before running the make install command to re-run the
101 make config-recursive command until no more dialogs are seen.
104 Install the DPDK and Browse Sources
105 -----------------------------------
107 First, uncompress the archive and move to the DPDK source directory:
109 .. code-block:: console
111 unzip DPDK-<version>.zip
115 app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile
118 The DPDK is composed of several directories:
120 * lib: Source code of DPDK libraries
122 * app: Source code of DPDK applications (automatic tests)
124 * examples: Source code of DPDK applications
126 * config, tools, scripts, mk: Framework-related makefiles, scripts and configuration
128 Installation of the DPDK Target Environments
129 --------------------------------------------
131 The format of a DPDK target is::
133 ARCH-MACHINE-EXECENV-TOOLCHAIN
137 * ``ARCH`` is: ``x86_64``
139 * ``MACHINE`` is: ``native``
141 * ``EXECENV`` is: ``bsdapp``
143 * ``TOOLCHAIN`` is: ``gcc`` | ``clang``
145 The configuration files for the DPDK targets can be found in the DPDK/config
146 directory in the form of::
148 defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN
152 Configuration files are provided with the ``RTE_MACHINE`` optimization level set.
153 Within the configuration files, the ``RTE_MACHINE`` configuration value is set
154 to native, which means that the compiled software is tuned for the platform
155 on which it is built. For more information on this setting, and its
156 possible values, see the *DPDK Programmers Guide*.
158 To make the target, use ``gmake install T=<target>``.
160 For example to compile for FreeBSD use:
162 .. code-block:: console
164 gmake install T=x86_64-native-bsdapp-clang
168 If the compiler binary to be used does not correspond to that given in the
169 TOOLCHAIN part of the target, the compiler command may need to be explicitly
170 specified. For example, if compiling for gcc, where the gcc binary is called
171 gcc4.8, the command would need to be ``gmake install T=<target> CC=gcc4.8``.
173 Browsing the Installed DPDK Environment Target
174 ----------------------------------------------
176 Once a target is created, it contains all the libraries and header files for the
177 DPDK environment that are required to build customer applications.
178 In addition, the test and testpmd applications are built under the build/app
179 directory, which may be used for testing. A kmod directory is also present that
180 contains the kernel modules to install:
182 .. code-block:: console
184 ls x86_64-native-bsdapp-gcc
186 app build hostapp include kmod lib Makefile
189 .. _loading_contigmem:
191 Loading the DPDK contigmem Module
192 ---------------------------------
194 To run a DPDK application, physically contiguous memory is required.
195 In the absence of non-transparent superpages, the included sources for the
196 contigmem kernel module provides the ability to present contiguous blocks of
197 memory for the DPDK to use. The contigmem module must be loaded into the
198 running kernel before any DPDK is run. The module is found in the kmod
199 sub-directory of the DPDK target directory.
201 The amount of physically contiguous memory along with the number of physically
202 contiguous blocks to be reserved by the module can be set at runtime prior to
203 module loading using:
205 .. code-block:: console
207 kenv hw.contigmem.num_buffers=n
208 kenv hw.contigmem.buffer_size=m
210 The kernel environment variables can also be specified during boot by placing the
211 following in ``/boot/loader.conf``::
213 hw.contigmem.num_buffers=n hw.contigmem.buffer_size=m
215 The variables can be inspected using the following command:
217 .. code-block:: console
219 sysctl -a hw.contigmem
221 Where n is the number of blocks and m is the size in bytes of each area of
222 contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte)
223 each is set during module load if they are not specified in the environment.
225 The module can then be loaded using kldload (assuming that the current directory
226 is the DPDK target directory):
228 .. code-block:: console
230 kldload ./kmod/contigmem.ko
232 It is advisable to include the loading of the contigmem module during the boot
233 process to avoid issues with potential memory fragmentation during later system
234 up time. This can be achieved by copying the module to the ``/boot/kernel/``
235 directory and placing the following into ``/boot/loader.conf``::
241 The contigmem_load directive should be placed after any definitions of
242 ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values
247 .. code-block:: console
249 kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko:
252 is generally attributed to not having enough contiguous memory
253 available and can be verified via dmesg or ``/var/log/messages``:
255 .. code-block:: console
257 kernel: contigmalloc failed for buffer <n>
259 To avoid this error, reduce the number of buffers or the buffer size.
263 Loading the DPDK nic_uio Module
264 -------------------------------
266 After loading the contigmem module, the ``nic_uio must`` also be loaded into the
267 running kernel prior to running any DPDK application. This module must
268 be loaded using the kldload command as shown below (assuming that the current
269 directory is the DPDK target directory).
271 .. code-block:: console
273 kldload ./kmod/nic_uio.ko
277 If the ports to be used are currently bound to a existing kernel driver
278 then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the
279 module. Setting this value is described in the next section below.
281 Currently loaded modules can be seen by using the ``kldstat`` command and a module
282 can be removed from the running kernel by using ``kldunload <module_name>``.
284 To load the module during boot, copy the ``nic_uio`` module to ``/boot/kernel``
285 and place the following into ``/boot/loader.conf``::
291 ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists.
293 By default, the ``nic_uio`` module will take ownership of network ports if they are
294 recognized DPDK devices and are not owned by another module. However, since
295 the FreeBSD kernel includes support, either built-in, or via a separate driver
296 module, for most network card devices, it is likely that the ports to be used are
297 already bound to a driver other than ``nic_uio``. The following sub-section describe
298 how to query and modify the device ownership of the ports to be used by
301 .. _binding_network_ports:
303 Binding Network Ports to the nic_uio Module
304 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
306 Device ownership can be viewed using the pciconf -l command. The example below shows
307 four IntelĀ® 82599 network ports under ``if_ixgbe`` module ownership.
309 .. code-block:: console
312 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
313 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
314 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
315 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
317 The first column constitutes three components:
319 #. Device name: ``ixN``
321 #. Unit name: ``pci0``
323 #. Selector (Bus:Device:Function): ``1:0:0``
325 Where no driver is associated with a device, the device name will be ``none``.
327 By default, the FreeBSD kernel will include built-in drivers for the most common
328 devices; a kernel rebuild would normally be required to either remove the drivers
329 or configure them as loadable modules.
331 To avoid building a custom kernel, the ``nic_uio`` module can detach a network port
332 from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs``
333 kernel environment variable prior to loading ``nic_uio``, as follows::
335 hw.nic_uio.bdfs="b:d:f,b:d:f,..."
337 Where a comma separated list of selectors is set, the list must not contain any
340 For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module
341 upon loading, use the following command::
343 kenv hw.nic_uio.bdfs="2:0:0,2:0:1"
345 The variable can also be specified during boot by placing the following into
346 ``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as
349 hw.nic_uio.bdfs="2:0:0,2:0:1"
352 Binding Network Ports Back to their Original Kernel Driver
353 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
355 If the original driver for a network port has been compiled into the kernel,
356 it is necessary to reboot FreeBSD to restore the original device binding. Before
357 doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``.
359 If rebinding to a driver that is a loadable module, the network port binding can
360 be reset without rebooting. To do so, unload both the target kernel module and the
361 ``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv)
362 value, and reload the two drivers - first the original kernel driver, and then
363 the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are
364 ports that are still to be bound to it.
366 Example commands to perform these steps are shown below:
368 .. code-block:: console
371 kldunload <original_driver>
373 # To clear the value completely:
374 kenv -u hw.nic_uio.bdfs
376 # To update the list of ports to bind:
377 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..."
379 kldload <original_driver>
381 kldload nic_uio # optional