X-Git-Url: http://git.droids-corp.org/?a=blobdiff_plain;f=doc%2Fguides%2Ffreebsd_gsg%2Fbuild_dpdk.rst;h=8bd9b6a1cea31943bb89fc12422015f72c0b7601;hb=4ec6960aec264172ac22f89c31ec4b43234bb264;hp=9b788408cc98126f4ccd5e2378f81cdf2b89f2f4;hpb=dacdbfa457525ef5cabdd1db9854310da8f37a6b;p=dpdk.git diff --git a/doc/guides/freebsd_gsg/build_dpdk.rst b/doc/guides/freebsd_gsg/build_dpdk.rst index 9b788408cc..8bd9b6a1ce 100644 --- a/doc/guides/freebsd_gsg/build_dpdk.rst +++ b/doc/guides/freebsd_gsg/build_dpdk.rst @@ -28,123 +28,218 @@ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Compiling the Intel® DPDK Target from Source -============================================ +.. _building_from_source: -Install the Intel® DPDK and Browse Sources ------------------------------------------- +Compiling the DPDK Target from Source +===================================== -First, uncompress the archive and move to the Intel® DPDK source directory: +System Requirements +------------------- + +The DPDK and its applications require the GNU make system (gmake) +to build on FreeBSD. Optionally, gcc may also be used in place of clang +to build the DPDK, in which case it too must be installed prior to +compiling the DPDK. The installation of these tools is covered in this +section. + +Compiling the DPDK requires the FreeBSD kernel sources, which should be +included during the installation of FreeBSD on the development platform. +The DPDK also requires the use of FreeBSD ports to compile and function. + +To use the FreeBSD ports system, it is required to update and extract the FreeBSD +ports tree by issuing the following commands: .. code-block:: console - user@host:~ # unzip DPDK-zip - user@host:~ # cd DPDK- - user@host:~/DPDK # ls - app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/ + portsnap fetch + portsnap extract + +If the environment requires proxies for external communication, these can be set +using: -The Intel® DPDK is composed of several directories: +.. code-block:: console -* lib: Source code of Intel® DPDK libraries + setenv http_proxy : + setenv ftp_proxy : -* app: Source code of Intel® DPDK applications (automatic tests) +The FreeBSD ports below need to be installed prior to building the DPDK. +In general these can be installed using the following set of commands:: -* examples: Source code of Intel® DPDK applications + cd /usr/ports/ -* config, tools, scripts, mk: Framework-related makefiles, scripts and configuration + make config-recursive -Installation of the Intel® DPDK Target Environments ---------------------------------------------------- + make install -The format of an Intel® DPDK target is: + make clean -ARCH-MACHINE-EXECENV-TOOLCHAIN +Each port location can be found using:: -Where: + whereis -* ARCH is: x86_64 +The ports required and their locations are as follows: -* MACHINE is: native +* dialog4ports: ``/usr/ports/ports-mgmt/dialog4ports`` -* EXECENV is: bsdapp +* GNU make(gmake): ``/usr/ports/devel/gmake`` -* TOOLCHAIN is: gcc +* coreutils: ``/usr/ports/sysutils/coreutils`` -The configuration files for the Intel® DPDK targets can be found in the DPDK/config directory in the form of: +For compiling and using the DPDK with gcc, the compiler must be installed +from the ports collection: -:: +* gcc: version 4.9 is recommended ``/usr/ports/lang/gcc49``. + Ensure that ``CPU_OPTS`` is selected (default is OFF). - defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN +When running the make config-recursive command, a dialog may be presented to the +user. For the installation of the DPDK, the default options were used. .. note:: - Configuration files are provided with the RTE_MACHINE optimization level set. - Within the configuration files, the RTE_MACHINE configuration value is set to native, - which means that the compiled software is tuned for the platform on which it is built. - For more information on this setting, and its possible values, - see the *Intel® DPDK Programmers Guide*. + To avoid multiple dialogs being presented to the user during make install, + it is advisable before running the make install command to re-run the + make config-recursive command until no more dialogs are seen. + -To install and make the target, use gmake install T= CC=gcc48. +Install the DPDK and Browse Sources +----------------------------------- -For example to compile for FreeBSD* use: +First, uncompress the archive and move to the DPDK source directory: .. code-block:: console - gmake install T=x86_64-native-bsdapp-gcc CC=gcc48 + unzip DPDK-.zip + cd DPDK- + +The DPDK is composed of several directories: + +* lib: Source code of DPDK libraries + +* app: Source code of DPDK applications (automatic tests) + +* examples: Source code of DPDK applications + +* config, buildtools, mk: Framework-related makefiles, scripts and configuration + +Installation of the DPDK Target Environments +-------------------------------------------- + +The format of a DPDK target is:: + + ARCH-MACHINE-EXECENV-TOOLCHAIN -To prepare a target without building it, for example, -if the configuration changes need to be made before compilation, -use the gmake config T= command: +Where: + +* ``ARCH`` is: ``x86_64`` + +* ``MACHINE`` is: ``native`` + +* ``EXECENV`` is: ``bsdapp`` + +* ``TOOLCHAIN`` is: ``gcc`` | ``clang`` + +The configuration files for the DPDK targets can be found in the DPDK/config +directory in the form of:: + + defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN + +.. note:: + + Configuration files are provided with the ``RTE_MACHINE`` optimization level set. + Within the configuration files, the ``RTE_MACHINE`` configuration value is set + to native, which means that the compiled software is tuned for the platform + on which it is built. For more information on this setting, and its + possible values, see the *DPDK Programmers Guide*. + +To make the target, use ``gmake install T=``. + +For example to compile for FreeBSD use: .. code-block:: console - gmake config T=x86_64-native-bsdapp-gcc CC=gcc48 + gmake install T=x86_64-native-bsdapp-clang + +.. note:: + + If the compiler binary to be used does not correspond to that given in the + TOOLCHAIN part of the target, the compiler command may need to be explicitly + specified. For example, if compiling for gcc, where the gcc binary is called + gcc4.9, the command would need to be ``gmake install T= CC=gcc4.9``. -To build after configuration, change directory to ./x86_64-native-bsdapp-gcc and use: +Browsing the Installed DPDK Environment Target +---------------------------------------------- + +Once a target is created, it contains all the libraries and header files for the +DPDK environment that are required to build customer applications. +In addition, the test and testpmd applications are built under the build/app +directory, which may be used for testing. A kmod directory is also present that +contains the kernel modules to install. + +.. _loading_contigmem: + +Loading the DPDK contigmem Module +--------------------------------- + +To run a DPDK application, physically contiguous memory is required. +In the absence of non-transparent superpages, the included sources for the +contigmem kernel module provides the ability to present contiguous blocks of +memory for the DPDK to use. The contigmem module must be loaded into the +running kernel before any DPDK is run. The module is found in the kmod +sub-directory of the DPDK target directory. + +The amount of physically contiguous memory along with the number of physically +contiguous blocks to be reserved by the module can be set at runtime prior to +module loading using: .. code-block:: console - gmake CC=gcc48 + kenv hw.contigmem.num_buffers=n + kenv hw.contigmem.buffer_size=m + +The kernel environment variables can also be specified during boot by placing the +following in ``/boot/loader.conf``:: -Browsing the Installed Intel®DPDK Environment Target ----------------------------------------------------- + hw.contigmem.num_buffers=n hw.contigmem.buffer_size=m -Once a target is created, it contains all the libraries -and header files for the Intel® DPDK environment that are required to build customer applications. -In addition, the test and testpmd applications are built under the build/app directory, which may be used for testing. -A kmod directory is also present that contains the kernel modules to install: +The variables can be inspected using the following command: .. code-block:: console - user@host:~/DPDK # ls x86_64-native-bsdapp-gcc - app build hostapp include kmod lib Makefile + sysctl -a hw.contigmem -Loading the Intel® DPDK contigmem Module ----------------------------------------- +Where n is the number of blocks and m is the size in bytes of each area of +contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) +each is set during module load if they are not specified in the environment. -To run any Intel® DPDK application, the contigmem module must be loaded into the running kernel. -The module is found in the kmod sub-directory of the Intel® DPDK target directory. -The module can be loaded using kldload (assuming that the current directory is the Intel® DPDK target directory): +The module can then be loaded using kldload (assuming that the current directory +is the DPDK target directory): .. code-block:: console kldload ./kmod/contigmem.ko -It is advisable to include the loading of the contigmem module during the boot process to avoid issues -with potential memory fragmentation during later system up time. -This can be achieved by copying the module to the /boot/kernel/ directory and placing the following into /boot/loader.conf: - -:: +It is advisable to include the loading of the contigmem module during the boot +process to avoid issues with potential memory fragmentation during later system +up time. This can be achieved by copying the module to the ``/boot/kernel/`` +directory and placing the following into ``/boot/loader.conf``:: contigmem_load="YES" .. note:: - The contigmem_load directive should be placed after any definitions of hw.contigmem.num_buffers - and hw.contigmem.buffer_size if the default values are not to be used. + The contigmem_load directive should be placed after any definitions of + ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values + are not to be used. + +An error such as: -An error such as kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko: Exec format error, -is generally attributed to not having enough contiguous memory available and can be verified via dmesg or /var/log/messages: +.. code-block:: console + + kldload: can't load ./x86_64-native-bsdapp-gcc/kmod/contigmem.ko: + Exec format error + +is generally attributed to not having enough contiguous memory +available and can be verified via dmesg or ``/var/log/messages``: .. code-block:: console @@ -152,11 +247,15 @@ is generally attributed to not having enough contiguous memory available and can To avoid this error, reduce the number of buffers or the buffer size. -Loading the Intel® DPDK nic_uio Module --------------------------------------- +.. _loading_nic_uio: + +Loading the DPDK nic_uio Module +------------------------------- -After loading the contigmem module, the nic_uio must also be loaded into the running kernel prior to running any Intel® DPDK application. -This module must be loaded using the kldload command as shown below (assuming that the current directory is the Intel® DPDK target directory). +After loading the contigmem module, the ``nic_uio`` module must also be loaded into the +running kernel prior to running any DPDK application. This module must +be loaded using the kldload command as shown below (assuming that the current +directory is the DPDK target directory). .. code-block:: console @@ -164,36 +263,41 @@ This module must be loaded using the kldload command as shown below (assuming th .. note:: - Currently loaded modules can be seen by using the kldstat command. - A module can be removed from the running kernel by using kldunload . - While the nic_uio module can be loaded during boot, - the module load order cannot be guaranteed and in the case where only some ports are bound to nic_uio - and others remain in use by the original driver, it is necessary to load nic_uio after booting into the kernel, - specifically after the original driver has been loaded. + If the ports to be used are currently bound to a existing kernel driver + then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the + module. Setting this value is described in the next section below. -To load the module during boot, copy the nic_uio module to /boot/kernel and place the following into /boot/loader.conf: +Currently loaded modules can be seen by using the ``kldstat`` command and a module +can be removed from the running kernel by using ``kldunload ``. -:: +To load the module during boot, copy the ``nic_uio`` module to ``/boot/kernel`` +and place the following into ``/boot/loader.conf``:: nic_uio_load="YES" .. note:: - nic_uio_load="YES" must appear after the contigmem_load directive, if it exists. + ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. -Binding Network Ports to the nic_uio Module -------------------------------------------- +By default, the ``nic_uio`` module will take ownership of network ports if they are +recognized DPDK devices and are not owned by another module. However, since +the FreeBSD kernel includes support, either built-in, or via a separate driver +module, for most network card devices, it is likely that the ports to be used are +already bound to a driver other than ``nic_uio``. The following sub-section describe +how to query and modify the device ownership of the ports to be used by +DPDK applications. -By default, the nic_uio module will take ownership of network ports if they are recognized Intel® DPDK devices -and are not owned by another module. +.. _binding_network_ports: -Device ownership can be viewed using the pciconf -l command. +Binding Network Ports to the nic_uio Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The example below shows four Intel® 82599 network ports under if_ixgbe module ownership. +Device ownership can be viewed using the pciconf -l command. The example below shows +four Intel® 82599 network ports under ``if_ixgbe`` module ownership. .. code-block:: console - user@host:~ # pciconf -l + pciconf -l ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 @@ -201,75 +305,66 @@ The example below shows four Intel® 82599 network ports under if_ixgbe module o The first column constitutes three components: -#. Device name: ixN +#. Device name: ``ixN`` -#. Unit name: pci0 +#. Unit name: ``pci0`` -#. Selector (Bus:Device:Function): 1:0:0 +#. Selector (Bus:Device:Function): ``1:0:0`` -Where no driver is associated with a device, the device name will be none. +Where no driver is associated with a device, the device name will be ``none``. -By default, the FreeBSD* kernel will include built-in drivers for the most common devices; -a kernel rebuild would normally be required to either remove the drivers or configure them as loadable modules. +By default, the FreeBSD kernel will include built-in drivers for the most common +devices; a kernel rebuild would normally be required to either remove the drivers +or configure them as loadable modules. -To avoid building a custom kernel, the nic_uio module can detach a network port from its current device driver. -This is achieved by setting the hw.nic_uio.bdfs kernel environment variable prior to loading nic_uio, as follows: - -:: +To avoid building a custom kernel, the ``nic_uio`` module can detach a network port +from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` +kernel environment variable prior to loading ``nic_uio``, as follows:: hw.nic_uio.bdfs="b:d:f,b:d:f,..." -Where a comma separated list of selectors is set, the list must not contain any whitespace. - -For example to re-bind ix2@pci0:2:0:0 and ix3@pci0:2:0: to the nic_uio module upon loading, use the following command: +Where a comma separated list of selectors is set, the list must not contain any +whitespace. -.. code-block:: console +For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module +upon loading, use the following command:: kenv hw.nic_uio.bdfs="2:0:0,2:0:1" -The variable can also be specified during boot by placing the following into /boot/ loader.conf: - -:: +The variable can also be specified during boot by placing the following into +``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as +shown:: hw.nic_uio.bdfs="2:0:0,2:0:1" + nic_uio_load="YES" -To restore the original device binding, -it is necessary to reboot FreeBSD* if the original driver has been compiled into the kernel. - -For example to rebind some or all ports to the original driver: - -Update or remove the hw.nic_uio.bdfs entry in /boot/loader.conf if specified there for persistency, then; - -.. code-block:: console +Binding Network Ports Back to their Original Kernel Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - reboot +If the original driver for a network port has been compiled into the kernel, +it is necessary to reboot FreeBSD to restore the original device binding. Before +doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. -If rebinding to a driver that is a loadable module, the network port binding can be reset without rebooting. -This requires the unloading of the nic_uio module and the original driver. +If rebinding to a driver that is a loadable module, the network port binding can +be reset without rebooting. To do so, unload both the target kernel module and the +``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) +value, and reload the two drivers - first the original kernel driver, and then +the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are +ports that are still to be bound to it. -Update or remove the hw.nic_uio.bdfs entry from /boot/loader.conf if specified there for persistency. +Example commands to perform these steps are shown below: .. code-block:: console kldunload nic_uio + kldunload -kldunload - -.. code-block:: console - + # To clear the value completely: kenv -u hw.nic_uio.bdfs -to remove all network ports from nic_uio and undefined this system variable OR - -.. code-block:: console - + # To update the list of ports to bind: kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." -(to update nic_uio ports) - -.. code-block:: console - kldload - kldload nic_uio -(if updating the list of associated network ports) + kldload nic_uio # optional