.. SPDX-License-Identifier: BSD-3-Clause
- Copyright(c) 2018 ARM Corporation.
+ Copyright(c) 2021 ARM Corporation.
-Cross compile DPDK for ARM64
-============================
-This chapter describes how to cross compile DPDK for ARM64 from x86 build hosts.
+Cross compiling DPDK for aarch64 and aarch32
+============================================
-.. note::
-
- Whilst it is recommended to natively build DPDK on ARM64 (just
- like with x86), it is also possible to cross-build DPDK for ARM64. An
- ARM64 cross compile GNU toolchain is used for this.
-
-Obtain the cross tool chain
----------------------------
-The latest cross compile tool chain can be downloaded from:
-https://developer.arm.com/open-source/gnu-toolchain/gnu-a/downloads.
-
-Following is the step to get the version 8.2, latest one at the time of this writing.
-
-.. code-block:: console
-
- wget https://developer.arm.com/-/media/Files/downloads/gnu-a/8.2-2019.01/gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu.tar.xz
-
-Unzip and add into the PATH
----------------------------
-
-.. code-block:: console
-
- tar -xvf gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu.tar.xz
- export PATH=$PATH:<cross_install_dir>/gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu/bin
+This chapter describes how to cross compile DPDK for aarch64 on x86 build
+machine and compile 32-bit aarch32 DPDK on aarch64 build machine.
.. note::
- For the host requirements and other info, refer to the release note section: https://releases.linaro.org/components/toolchain/binaries/
+ Whilst it is recommended to natively build DPDK on aarch64 (just
+ like with x86), it is also possible to cross compile DPDK for aarch64.
+ An aarch64 cross compiler GNU toolchain or an LLVM/clang toolchain
+ may be used for cross-compilation.
-.. _arm_cross_build_getting_the_prerequisite_library:
-Getting the prerequisite library
---------------------------------
+Prerequisites
+-------------
+
+NUMA library
+~~~~~~~~~~~~
NUMA is required by most modern machines, not needed for non-NUMA architectures.
git clone https://github.com/numactl/numactl.git
cd numactl
- git checkout v2.0.11 -b v2.0.11
+ git checkout v2.0.13 -b v2.0.13
./autogen.sh
autoconf -i
- ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc --prefix=<numa install dir>
+ ./configure --host=aarch64-linux-gnu CC=<compiler> --prefix=<numa install dir>
make install
-The numa header files and lib file is generated in the include and lib folder respectively under <numa install dir>.
+.. note::
+
+ The compiler above can be either aarch64-linux-gnu-gcc or clang.
+ See below for information on how to get specific compilers.
+
+The numa header files and lib file is generated in the include and lib folder
+respectively under ``<numa install dir>``.
+
+Meson prerequisites
+~~~~~~~~~~~~~~~~~~~
+
+Meson depends on pkgconfig to find the dependencies.
+The package ``pkg-config-aarch64-linux-gnu`` is required for aarch64.
+To install it in Ubuntu::
+
+ sudo apt install pkg-config-aarch64-linux-gnu
+
+For aarch32, install ``pkg-config-arm-linux-gnueabihf``::
+
+ sudo apt install pkg-config-arm-linux-gnueabihf
+
+
+GNU toolchain
+-------------
+
+.. _obtain_GNU_toolchain:
+
+Get the cross toolchain
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The latest GNU cross compiler toolchain can be downloaded from:
+https://developer.arm.com/open-source/gnu-toolchain/gnu-a/downloads.
-.. _augment_the_cross_toolchain_with_numa_support:
+It is always recommended to check and get the latest compiler tool
+from the page and use it to generate better code.
+As of this writing 9.2-2019.12 is the newest,
+the following description is an example of this version.
-Augment the cross toolchain with NUMA support
----------------------------------------------
+For aarch64::
+
+ wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
+ tar -xvf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
+ export PATH=$PATH:<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin
+
+For aarch32::
+
+ wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf.tar.xz
+ tar -xvf gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf.tar.xz
+ export PATH=$PATH:<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf/bin
+
+.. note::
+
+ For the host requirements and other info, refer to the release note section:
+ https://releases.linaro.org/components/toolchain/binaries/
+
+.. _augment_the_gnu_toolchain_with_numa_support:
+
+Augment the GNU toolchain with NUMA support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. note::
- This way is optional, an alternative is to use extra CFLAGS and LDFLAGS, depicted in :ref:`configure_and_cross_compile_dpdk_build` below.
+ This way is optional, an alternative is to use extra CFLAGS and LDFLAGS.
Copy the NUMA header files and lib to the cross compiler's directories:
.. code-block:: console
- cp <numa_install_dir>/include/numa*.h <cross_install_dir>/gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu/bin/../aarch64-linux-gnu/libc/usr/include/
- cp <numa_install_dir>/lib/libnuma.a <cross_install_dir>/gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu/lib/gcc/aarch64-linux-gnu/8.2/
- cp <numa_install_dir>/lib/libnuma.so <cross_install_dir>/gcc-arm-8.2-2019.01-x86_64-aarch64-linux-gnu/lib/gcc/aarch64-linux-gnu/8.2/
+ cp <numa_install_dir>/include/numa*.h <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc/usr/include/
+ cp <numa_install_dir>/lib/libnuma.a <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/lib/gcc/aarch64-none-linux-gnu/9.2.1/
+ cp <numa_install_dir>/lib/libnuma.so <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/lib/gcc/aarch64-none-linux-gnu/9.2.1/
-.. _configure_and_cross_compile_dpdk_build:
-
-Cross Compiling DPDK using Meson
---------------------------------
+Cross Compiling DPDK with GNU toolchain using Meson
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To cross-compile DPDK on a desired target machine we can use the following
command::
- meson cross-build --cross-file <target_machine_configuration>
- ninja -C cross-build
+ meson cross-build --cross-file <target_machine_configuration>
+ ninja -C cross-build
-For example if the target machine is arm64 we can use the following
+For example if the target machine is aarch64 we can use the following
command::
- meson arm64-build --cross-file config/arm/arm64_armv8_linux_gcc
- ninja -C arm64-build
+ meson aarch64-build-gcc --cross-file config/arm/arm64_armv8_linux_gcc
+ ninja -C aarch64-build-gcc
+
+If the target machine is aarch32 we can use the following command::
-Configure and Cross Compile DPDK using Make
--------------------------------------------
-To configure a build, choose one of the target configurations, like arm64-dpaa-linux-gcc and arm64-thunderx-linux-gcc.
+ meson aarch32-build --cross-file config/arm/arm32_armv8_linux_gcc
+ ninja -C aarch32-build
+
+LLVM/Clang toolchain
+--------------------
+
+Obtain the cross tool chain
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The latest LLVM/Clang cross compiler toolchain can be downloaded from:
+https://developer.arm.com/tools-and-software/open-source-software/developer-tools/llvm-toolchain.
.. code-block:: console
- make config T=arm64-armv8a-linux-gcc
+ # Ubuntu binaries
+ wget https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz
+
+The LLVM/Clang toolchain does not implement the standard c library.
+The GNU toolchain ships an implementation we can use.
+Refer to obtain_GNU_toolchain_ to get the GNU toolchain.
-To cross-compile, without compiling the kernel modules, use the following command:
+Unzip and add into the PATH
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: console
- make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
+ tar -xvf clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz
+ export PATH=$PATH:<cross_install_dir>/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04/bin
+
+Cross Compiling DPDK with LLVM/Clang toolchain using Meson
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+ To use the NUMA library follow the same steps as for
+ augment_the_gnu_toolchain_with_numa_support_.
-To cross-compile, including the kernel modules, the kernel source tree needs to be specified by setting
-RTE_KERNELDIR:
+The paths to GNU stdlib must be specified in a cross file.
+Augmenting the default cross-file's ``c_args`` and ``c_link_args``
+``config/arm/arm64_armv8_linux_clang_ubuntu1804`` would look like this:
.. code-block:: console
- make -j CROSS=aarch64-linux-gnu- RTE_KERNELDIR=<kernel_src_rootdir> CROSS_COMPILE=aarch64-linux-gnu-
+ ...
+ c_args = ['-target', 'aarch64-linux-gnu', '--sysroot', '<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc']
+ c_link_args = ['-target', 'aarch64-linux-gnu', '-fuse-ld=lld', '--sysroot', '<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc', '--gcc-toolchain=<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu']
+
+Assuming the file with augmented ``c_args`` and ``c_link_args``
+is named ``arm64_armv8_linux_clang``,
+use the following command to cross-compile DPDK for the target machine::
+
+ meson aarch64-build-clang --cross-file config/arm/arm64_armv8_linux_clang
+ ninja -C aarch64-build-clang
+
+Cross Compiling DPDK with LLVM/Clang toolchain using Meson on Ubuntu 18.04
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To compile for non-NUMA targets, without compiling the kernel modules, use the following command:
+On most popular Linux distribution it is not necessary to download
+the toolchains, but rather use the packages provided by said distributions.
+On Ubuntu 18.04, these packages are needed:
.. code-block:: console
- make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n CONFIG_RTE_LIBRTE_VHOST_NUMA=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
+ sudo apt-get install pkg-config-aarch64-linux-gnu clang llvm llvm-dev lld
+ libc6-dev-arm64-cross libatomic1-arm64-cross libgcc-8-dev-arm64-cross
-.. note::
+Use the following command to cross-compile DPDK for the target machine::
+
+ meson aarch64-build-clang --cross-file config/arm/arm64_armv8_linux_clang_ubuntu1804
+ ninja -C aarch64-build-clang
+
+Building for an aarch64 SoC on an aarch64 build machine
+-------------------------------------------------------
+
+If you wish to build on an aarch64 build machine for a different aarch64 SoC,
+you don't need a separate cross toolchain, just a different set of
+configuration options. To build for an aarch64 SoC, use the -Dplatform meson
+option::
- 1. EXTRA_CFLAGS and EXTRA_LDFLAGS should be added to include the NUMA headers and link the library respectively,
- if the above step :ref:`augment_the_cross_toolchain_with_numa_support` was skipped therefore the toolchain was not
- augmented with NUMA support.
+ meson soc_build -Dplatform=<target_soc>
- 2. "-isystem <numa_install_dir>/include" should be add to EXTRA_CFLAGS, otherwise the numa.h file will get a lot of compiling
- errors of Werror=cast-qual, Werror=strict-prototypes and Werror=old-style-definition.
+Substitute <target_soc> with one of the supported SoCs
- An example is given below:
+.. literalinclude:: ../../../config/arm/meson.build
+ :start-after: Start of SoCs list
+ :end-before: End of SoCs list
- .. code-block:: console
+These SoCs are also used in cross files, e.g.::
+
+ [properties]
+ # Generate binaries that are portable across all Armv8 machines
+ platform = 'generic'
+
+Supported SoC configuration
+---------------------------
- make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n EXTRA_CFLAGS="-isystem <numa_install_dir>/include" EXTRA_LDFLAGS="-L<numa_install_dir>/lib -lnuma"
+The SoC configuration is a combination of implementer and CPU part number
+configuration and SoC-specific configuration::
+
+ soc_<name> = {
+ 'description': 'SoC Description', # mandatory
+ 'implementer': <implementer_id>, # mandatory
+ 'part_number': <part_number>, # mandatory
+ 'numa': false, # optional, specify for non-NUMA SoCs
+ 'enable_drivers': 'common/*,bus/*', # optional, comma-separated list of
+ # drivers to build, wildcards are accepted
+ 'disable_drivers': 'crypto/*', # optional, comma-separated list of
+ # drivers to disable, wildcards are accepted
+ 'flags': [
+ ['RTE_MAX_LCORE', '16'],
+ ['RTE_MAX_NUMA_NODES', '1']
+ ] # optional, list of DPDK options that will be added
+ # or overwritten
+ }
+
+Where <implementer_id> is a key defined in the implementers dictionary
+in config/arm/meson.build (e.g. 0x41) and part_number is a key defined
+in implementers[<implementer_id>]['part_number_config'] dictionary
+(i.e. the part number must be defined for the implementer,
+e.g. for 0x41, a valid value is 0xd49, which is the neoverse-n2 SoC).