X-Git-Url: http://git.droids-corp.org/?a=blobdiff_plain;f=doc%2Fguides%2Ffreebsd_gsg%2Fbuild_dpdk.rst;h=e3005a7f3cb0f499230012a40a7e6270b0b3ed79;hb=614af75489682e39bb005d80173434588e9490a6;hp=8f72a5ea3da3e18e2f737a6360a0b7c6dc113d02;hpb=f9e2411af05c407d177dd92b4d18343ca3b269b6;p=dpdk.git diff --git a/doc/guides/freebsd_gsg/build_dpdk.rst b/doc/guides/freebsd_gsg/build_dpdk.rst index 8f72a5ea3d..e3005a7f3c 100644 --- a/doc/guides/freebsd_gsg/build_dpdk.rst +++ b/doc/guides/freebsd_gsg/build_dpdk.rst @@ -1,207 +1,185 @@ -.. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - All rights reserved. +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions - are met: +.. _building_from_source: - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the - distribution. - * Neither the name of Intel Corporation nor the names of its - contributors may be used to endorse or promote products derived - from this software without specific prior written permission. +Compiling the DPDK Target from Source +===================================== - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +Prerequisites +------------- -Compiling the Intel® DPDK Target from Source -============================================ +The following FreeBSD packages are required to build DPDK: -Install the Intel® DPDK and Browse Sources ------------------------------------------- +* meson +* ninja +* pkgconf -First, uncompress the archive and move to the Intel® DPDK source directory: +These can be installed using (as root):: -.. code-block:: console + pkg install meson pkgconf - user@host:~ # unzip DPDK-zip - user@host:~ # cd DPDK- - user@host:~/DPDK # ls - app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/ +To compile the required kernel modules for memory management and working +with physical NIC devices, the kernel sources for FreeBSD also +need to be installed. If not already present on the system, these can be +installed via commands like the following, for FreeBSD 12.1 on x86_64:: -The Intel® DPDK is composed of several directories: + fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE/src.txz + tar -C / -xJvf src.txz -* lib: Source code of Intel® DPDK libraries +To enable the telemetry library in DPDK, the jansson library also needs to +be installed, and can be installed via:: -* app: Source code of Intel® DPDK applications (automatic tests) + pkg install jansson -* examples: Source code of Intel® DPDK applications +Individual drivers may have additional requirements. Consult the relevant +driver guide for any driver-specific requirements of interest. -* config, tools, scripts, mk: Framework-related makefiles, scripts and configuration +Building DPDK +------------- -Installation of the Intel® DPDK Target Environments ---------------------------------------------------- +The following commands can be used to build and install DPDK on a system. +The final, install, step generally needs to be run as root:: -The format of an Intel® DPDK target is: + meson build + cd build + ninja + ninja install -ARCH-MACHINE-EXECENV-TOOLCHAIN - -Where: - -* ARCH is: x86_64 - -* MACHINE is: native - -* EXECENV is: bsdapp - -* TOOLCHAIN is: gcc - -The configuration files for the Intel® DPDK targets can be found in the DPDK/config -directory in the form of: - -:: - - defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN +This will install the DPDK libraries and drivers to `/usr/local/lib` with a +pkg-config file `libdpdk.pc` installed to `/usr/local/lib/pkgconfig`. The +DPDK test applications, such as `dpdk-testpmd` are installed to +`/usr/local/bin`. To use these applications, it is recommended that the +`contigmem` and `nic_uio` kernel modules be loaded first, as described in +the next section. .. note:: - Configuration files are provided with the RTE_MACHINE optimization level set. - Within the configuration files, the RTE_MACHINE configuration value is set - to native, which means that the compiled software is tuned for the platform - on which it is built. For more information on this setting, and its - possible values, see the *Intel® DPDK Programmers Guide*. + It is recommended that pkg-config be used to query information + about the compiler and linker flags needed to build applications + against DPDK. In some cases, the path `/usr/local/lib/pkgconfig` + may not be in the default search paths for `.pc` files, which means + that queries for DPDK information may fail. This can be fixed by + setting the appropriate path in `PKG_CONFIG_PATH` environment + variable. -To install and make the target, use gmake install T= CC=gcc48. -For example to compile for FreeBSD* use: +.. _loading_contigmem: -.. code-block:: console +Loading the DPDK contigmem Module +--------------------------------- - gmake install T=x86_64-native-bsdapp-gcc CC=gcc48 +To run a DPDK application, physically contiguous memory is required. +In the absence of non-transparent superpages, the included sources for the +contigmem kernel module provides the ability to present contiguous blocks of +memory for the DPDK to use. The contigmem module must be loaded into the +running kernel before any DPDK is run. Once DPDK is installed on the +system, the module can be found in the `/boot/modules` directory. -To prepare a target without building it, for example, if the configuration -changes need to be made before compilation, use the gmake config T= command: +The amount of physically contiguous memory along with the number of physically +contiguous blocks to be reserved by the module can be set at runtime prior to +module loading using:: -.. code-block:: console + kenv hw.contigmem.num_buffers=n + kenv hw.contigmem.buffer_size=m - gmake config T=x86_64-native-bsdapp-gcc CC=gcc48 +The kernel environment variables can also be specified during boot by placing the +following in ``/boot/loader.conf``: -To build after configuration, change directory to ./x86_64-native-bsdapp-gcc and use: +.. code-block:: shell -.. code-block:: console + hw.contigmem.num_buffers=n + hw.contigmem.buffer_size=m - gmake CC=gcc48 +The variables can be inspected using the following command:: -Browsing the Installed Intel® DPDK Environment Target ------------------------------------------------------ + sysctl -a hw.contigmem -Once a target is created, it contains all the libraries and header files for the -Intel® DPDK environment that are required to build customer applications. -In addition, the test and testpmd applications are built under the build/app -directory, which may be used for testing. A kmod directory is also present that -contains the kernel modules to install: +Where n is the number of blocks and m is the size in bytes of each area of +contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) +each is set during module load if they are not specified in the environment. -.. code-block:: console +The module can then be loaded using kldload:: - user@host:~/DPDK # ls x86_64-native-bsdapp-gcc - app build hostapp include kmod lib Makefile - -Loading the Intel® DPDK contigmem Module ----------------------------------------- - -To run any Intel® DPDK application, the contigmem module must be loaded into the -running kernel. The module is found in the kmod sub-directory of the Intel® DPDK -target directory. The module can be loaded using kldload (assuming that the -current directory is the Intel® DPDK target directory): - -.. code-block:: console - - kldload ./kmod/contigmem.ko + kldload contigmem It is advisable to include the loading of the contigmem module during the boot process to avoid issues with potential memory fragmentation during later system -up time. This can be achieved by copying the module to the /boot/kernel/ -directory and placing the following into /boot/loader.conf: +up time. This can be achieved by placing lines similar to the following into +``/boot/loader.conf``: -:: +.. code-block:: shell + hw.contigmem.num_buffers=1 + hw.contigmem.buffer_size=1073741824 contigmem_load="YES" .. note:: The contigmem_load directive should be placed after any definitions of - hw.contigmem.num_buffers and hw.contigmem.buffer_size if the default values + ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values are not to be used. -An error such as kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko: -Exec format error, is generally attributed to not having enough contiguous memory -available and can be verified via dmesg or /var/log/messages: +An error such as:: -.. code-block:: console + kldload: can't load /kernel/freebsd/contigmem.ko: + Exec format error + +is generally attributed to not having enough contiguous memory +available and can be verified via dmesg or ``/var/log/messages``:: kernel: contigmalloc failed for buffer To avoid this error, reduce the number of buffers or the buffer size. -Loading the Intel® DPDK nic_uio Module --------------------------------------- +.. _loading_nic_uio: -After loading the contigmem module, the nic_uio must also be loaded into the -running kernel prior to running any Intel® DPDK application. This module must -be loaded using the kldload command as shown below (assuming that the current -directory is the Intel® DPDK target directory). +Loading the DPDK nic_uio Module +------------------------------- -.. code-block:: console +After loading the contigmem module, the ``nic_uio`` module must also be loaded into the +running kernel prior to running any DPDK application, e.g. using:: - kldload ./kmod/nic_uio.ko + kldload nic_uio .. note:: - Currently loaded modules can be seen by using the kldstat command. A module - can be removed from the running kernel by using kldunload . - While the nic_uio module can be loaded during boot, the module load order - cannot be guaranteed and in the case where only some ports are bound to - nic_uio and others remain in use by the original driver, it is necessary to - load nic_uio after booting into the kernel, specifically after the original - driver has been loaded. + If the ports to be used are currently bound to a existing kernel driver + then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the + module. Setting this value is described in the next section below. -To load the module during boot, copy the nic_uio module to /boot/kernel and place the following into /boot/loader.conf: +Currently loaded modules can be seen by using the ``kldstat`` command and a module +can be removed from the running kernel by using ``kldunload ``. -:: +To load the module during boot place the following into ``/boot/loader.conf``: + +.. code-block:: shell nic_uio_load="YES" .. note:: - nic_uio_load="YES" must appear after the contigmem_load directive, if it exists. + ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. -Binding Network Ports to the nic_uio Module -------------------------------------------- +By default, the ``nic_uio`` module will take ownership of network ports if they are +recognized DPDK devices and are not owned by another module. However, since +the FreeBSD kernel includes support, either built-in, or via a separate driver +module, for most network card devices, it is likely that the ports to be used are +already bound to a driver other than ``nic_uio``. The following sub-section describe +how to query and modify the device ownership of the ports to be used by +DPDK applications. -By default, the nic_uio module will take ownership of network ports if they are -recognized Intel® DPDK devices and are not owned by another module. +.. _binding_network_ports: -Device ownership can be viewed using the pciconf -l command. +Binding Network Ports to the nic_uio Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The example below shows four Intel® 82599 network ports under if_ixgbe module ownership. +Device ownership can be viewed using the pciconf -l command. The example below shows +four Intel® 82599 network ports under ``if_ixgbe`` module ownership. -.. code-block:: console +.. code-block:: none - user@host:~ # pciconf -l + pciconf -l ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 @@ -209,83 +187,66 @@ The example below shows four Intel® 82599 network ports under if_ixgbe module o The first column constitutes three components: -#. Device name: ixN +#. Device name: ``ixN`` -#. Unit name: pci0 +#. Unit name: ``pci0`` -#. Selector (Bus:Device:Function): 1:0:0 +#. Selector (Bus:Device:Function): ``1:0:0`` -Where no driver is associated with a device, the device name will be none. +Where no driver is associated with a device, the device name will be ``none``. -By default, the FreeBSD* kernel will include built-in drivers for the most common +By default, the FreeBSD kernel will include built-in drivers for the most common devices; a kernel rebuild would normally be required to either remove the drivers or configure them as loadable modules. -To avoid building a custom kernel, the nic_uio module can detach a network port -from its current device driver. This is achieved by setting the hw.nic_uio.bdfs -kernel environment variable prior to loading nic_uio, as follows: - -:: +To avoid building a custom kernel, the ``nic_uio`` module can detach a network port +from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` +kernel environment variable prior to loading ``nic_uio``, as follows:: - hw.nic_uio.bdfs="b:d:f,b:d:f,..." + kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." Where a comma separated list of selectors is set, the list must not contain any whitespace. -For example to re-bind ix2@pci0:2:0:0 and ix3@pci0:2:0: to the nic_uio module -upon loading, use the following command: - -.. code-block:: console +For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module +upon loading, use the following command:: kenv hw.nic_uio.bdfs="2:0:0,2:0:1" The variable can also be specified during boot by placing the following into -/boot/loader.conf: +``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as +shown: -:: +.. code-block:: shell hw.nic_uio.bdfs="2:0:0,2:0:1" + nic_uio_load="YES" -To restore the original device binding, it is necessary to reboot FreeBSD* if the -original driver has been compiled into the kernel. - -For example to rebind some or all ports to the original driver: - -Update or remove the hw.nic_uio.bdfs entry in /boot/loader.conf if specified there -for persistency, then; - -.. code-block:: console +Binding Network Ports Back to their Original Kernel Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - reboot +If the original driver for a network port has been compiled into the kernel, +it is necessary to reboot FreeBSD to restore the original device binding. Before +doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. If rebinding to a driver that is a loadable module, the network port binding can -be reset without rebooting. This requires the unloading of the nic_uio module -and the original driver. +be reset without rebooting. To do so, unload both the target kernel module and the +``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) +value, and reload the two drivers - first the original kernel driver, and then +the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are +ports that are still to be bound to it. -Update or remove the hw.nic_uio.bdfs entry from /boot/loader.conf if specified -there for persistency. - -.. code-block:: console +Example commands to perform these steps are shown below:: kldunload nic_uio + kldunload -kldunload - -.. code-block:: console - + # To clear the value completely: kenv -u hw.nic_uio.bdfs -to remove all network ports from nic_uio and undefined this system variable OR - -.. code-block:: console - + # To update the list of ports to bind: kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." -(to update nic_uio ports) - -.. code-block:: console - kldload - kldload nic_uio -(if updating the list of associated network ports) + kldload nic_uio # optional