1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2010-2014 Intel Corporation.
4 .. _building_from_source:
6 Compiling the DPDK Target from Source
7 =====================================
12 The DPDK and its applications require the GNU make system (gmake)
13 to build on FreeBSD. Optionally, gcc may also be used in place of clang
14 to build the DPDK, in which case it too must be installed prior to
15 compiling the DPDK. The installation of these tools is covered in this
18 Compiling the DPDK requires the FreeBSD kernel sources, which should be
19 included during the installation of FreeBSD on the development platform.
20 The DPDK also requires the use of FreeBSD ports to compile and function.
22 To use the FreeBSD ports system, it is required to update and extract the FreeBSD
23 ports tree by issuing the following commands:
25 .. code-block:: console
30 If the environment requires proxies for external communication, these can be set
33 .. code-block:: console
35 setenv http_proxy <my_proxy_host>:<port>
36 setenv ftp_proxy <my_proxy_host>:<port>
38 The FreeBSD ports below need to be installed prior to building the DPDK.
39 In general these can be installed using the following set of commands::
41 cd /usr/ports/<port_location>
49 Each port location can be found using::
53 The ports required and their locations are as follows:
55 * dialog4ports: ``/usr/ports/ports-mgmt/dialog4ports``
57 * GNU make(gmake): ``/usr/ports/devel/gmake``
59 * coreutils: ``/usr/ports/sysutils/coreutils``
61 For compiling and using the DPDK with gcc, the compiler must be installed
62 from the ports collection:
64 * gcc: version 4.9 is recommended ``/usr/ports/lang/gcc49``.
65 Ensure that ``CPU_OPTS`` is selected (default is OFF).
67 When running the make config-recursive command, a dialog may be presented to the
68 user. For the installation of the DPDK, the default options were used.
72 To avoid multiple dialogs being presented to the user during make install,
73 it is advisable before running the make install command to re-run the
74 make config-recursive command until no more dialogs are seen.
77 Install the DPDK and Browse Sources
78 -----------------------------------
80 First, uncompress the archive and move to the DPDK source directory:
82 .. code-block:: console
84 unzip DPDK-<version>.zip
87 The DPDK is composed of several directories:
89 * lib: Source code of DPDK libraries
91 * app: Source code of DPDK applications (automatic tests)
93 * examples: Source code of DPDK applications
95 * config, buildtools, mk: Framework-related makefiles, scripts and configuration
97 Installation of the DPDK Target Environments
98 --------------------------------------------
100 The format of a DPDK target is::
102 ARCH-MACHINE-EXECENV-TOOLCHAIN
106 * ``ARCH`` is: ``x86_64``
108 * ``MACHINE`` is: ``native``
110 * ``EXECENV`` is: ``bsdapp``
112 * ``TOOLCHAIN`` is: ``gcc`` | ``clang``
114 The configuration files for the DPDK targets can be found in the DPDK/config
115 directory in the form of::
117 defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN
121 Configuration files are provided with the ``RTE_MACHINE`` optimization level set.
122 Within the configuration files, the ``RTE_MACHINE`` configuration value is set
123 to native, which means that the compiled software is tuned for the platform
124 on which it is built. For more information on this setting, and its
125 possible values, see the *DPDK Programmers Guide*.
127 To make the target, use ``gmake install T=<target>``.
129 For example to compile for FreeBSD use:
131 .. code-block:: console
133 gmake install T=x86_64-native-bsdapp-clang
137 If the compiler binary to be used does not correspond to that given in the
138 TOOLCHAIN part of the target, the compiler command may need to be explicitly
139 specified. For example, if compiling for gcc, where the gcc binary is called
140 gcc4.9, the command would need to be ``gmake install T=<target> CC=gcc4.9``.
142 Browsing the Installed DPDK Environment Target
143 ----------------------------------------------
145 Once a target is created, it contains all the libraries and header files for the
146 DPDK environment that are required to build customer applications.
147 In addition, the test and testpmd applications are built under the build/app
148 directory, which may be used for testing. A kmod directory is also present that
149 contains the kernel modules to install.
151 .. _loading_contigmem:
153 Loading the DPDK contigmem Module
154 ---------------------------------
156 To run a DPDK application, physically contiguous memory is required.
157 In the absence of non-transparent superpages, the included sources for the
158 contigmem kernel module provides the ability to present contiguous blocks of
159 memory for the DPDK to use. The contigmem module must be loaded into the
160 running kernel before any DPDK is run. The module is found in the kmod
161 sub-directory of the DPDK target directory.
163 The amount of physically contiguous memory along with the number of physically
164 contiguous blocks to be reserved by the module can be set at runtime prior to
165 module loading using:
167 .. code-block:: console
169 kenv hw.contigmem.num_buffers=n
170 kenv hw.contigmem.buffer_size=m
172 The kernel environment variables can also be specified during boot by placing the
173 following in ``/boot/loader.conf``::
175 hw.contigmem.num_buffers=n hw.contigmem.buffer_size=m
177 The variables can be inspected using the following command:
179 .. code-block:: console
181 sysctl -a hw.contigmem
183 Where n is the number of blocks and m is the size in bytes of each area of
184 contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte)
185 each is set during module load if they are not specified in the environment.
187 The module can then be loaded using kldload (assuming that the current directory
188 is the DPDK target directory):
190 .. code-block:: console
192 kldload ./kmod/contigmem.ko
194 It is advisable to include the loading of the contigmem module during the boot
195 process to avoid issues with potential memory fragmentation during later system
196 up time. This can be achieved by copying the module to the ``/boot/kernel/``
197 directory and placing the following into ``/boot/loader.conf``::
203 The contigmem_load directive should be placed after any definitions of
204 ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values
209 .. code-block:: console
211 kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko:
214 is generally attributed to not having enough contiguous memory
215 available and can be verified via dmesg or ``/var/log/messages``:
217 .. code-block:: console
219 kernel: contigmalloc failed for buffer <n>
221 To avoid this error, reduce the number of buffers or the buffer size.
225 Loading the DPDK nic_uio Module
226 -------------------------------
228 After loading the contigmem module, the ``nic_uio`` module must also be loaded into the
229 running kernel prior to running any DPDK application. This module must
230 be loaded using the kldload command as shown below (assuming that the current
231 directory is the DPDK target directory).
233 .. code-block:: console
235 kldload ./kmod/nic_uio.ko
239 If the ports to be used are currently bound to a existing kernel driver
240 then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the
241 module. Setting this value is described in the next section below.
243 Currently loaded modules can be seen by using the ``kldstat`` command and a module
244 can be removed from the running kernel by using ``kldunload <module_name>``.
246 To load the module during boot, copy the ``nic_uio`` module to ``/boot/kernel``
247 and place the following into ``/boot/loader.conf``::
253 ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists.
255 By default, the ``nic_uio`` module will take ownership of network ports if they are
256 recognized DPDK devices and are not owned by another module. However, since
257 the FreeBSD kernel includes support, either built-in, or via a separate driver
258 module, for most network card devices, it is likely that the ports to be used are
259 already bound to a driver other than ``nic_uio``. The following sub-section describe
260 how to query and modify the device ownership of the ports to be used by
263 .. _binding_network_ports:
265 Binding Network Ports to the nic_uio Module
266 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
268 Device ownership can be viewed using the pciconf -l command. The example below shows
269 four IntelĀ® 82599 network ports under ``if_ixgbe`` module ownership.
271 .. code-block:: console
274 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
275 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
276 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
277 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
279 The first column constitutes three components:
281 #. Device name: ``ixN``
283 #. Unit name: ``pci0``
285 #. Selector (Bus:Device:Function): ``1:0:0``
287 Where no driver is associated with a device, the device name will be ``none``.
289 By default, the FreeBSD kernel will include built-in drivers for the most common
290 devices; a kernel rebuild would normally be required to either remove the drivers
291 or configure them as loadable modules.
293 To avoid building a custom kernel, the ``nic_uio`` module can detach a network port
294 from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs``
295 kernel environment variable prior to loading ``nic_uio``, as follows::
297 hw.nic_uio.bdfs="b:d:f,b:d:f,..."
299 Where a comma separated list of selectors is set, the list must not contain any
302 For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module
303 upon loading, use the following command::
305 kenv hw.nic_uio.bdfs="2:0:0,2:0:1"
307 The variable can also be specified during boot by placing the following into
308 ``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as
311 hw.nic_uio.bdfs="2:0:0,2:0:1"
314 Binding Network Ports Back to their Original Kernel Driver
315 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
317 If the original driver for a network port has been compiled into the kernel,
318 it is necessary to reboot FreeBSD to restore the original device binding. Before
319 doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``.
321 If rebinding to a driver that is a loadable module, the network port binding can
322 be reset without rebooting. To do so, unload both the target kernel module and the
323 ``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv)
324 value, and reload the two drivers - first the original kernel driver, and then
325 the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are
326 ports that are still to be bound to it.
328 Example commands to perform these steps are shown below:
330 .. code-block:: console
333 kldunload <original_driver>
335 # To clear the value completely:
336 kenv -u hw.nic_uio.bdfs
338 # To update the list of ports to bind:
339 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..."
341 kldload <original_driver>
343 kldload nic_uio # optional