There is also a `section dedicated to this poll mode driver
<http://www.mellanox.com/page/products_dyn?product_family=209&mtag=pmd_for_dpdk>`_.
-.. note::
-
- Due to external dependencies, this driver is disabled by default. It must
- be enabled manually by setting ``CONFIG_RTE_LIBRTE_MLX4_PMD=y`` and
- recompiling DPDK.
Implementation details
----------------------
Compilation options
~~~~~~~~~~~~~~~~~~~
-These options can be modified in the ``.config`` file.
-
-- ``CONFIG_RTE_LIBRTE_MLX4_PMD`` (default **n**)
-
- Toggle compilation of librte_pmd_mlx4 itself.
-
-- ``CONFIG_RTE_IBVERBS_LINK_DLOPEN`` (default **n**)
+The ibverbs libraries can be linked with this PMD in a number of ways,
+configured by the ``ibverbs_link`` build option:
- Build PMD with additional code to make it loadable without hard
- dependencies on **libibverbs** nor **libmlx4**, which may not be installed
- on the target system.
+- ``shared`` (default): the PMD depends on some .so files.
- In this mode, their presence is still required for it to run properly,
- however their absence won't prevent a DPDK application from starting (with
- ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as
- missing with ``ldd(1)``.
+- ``dlopen``: Split the dependencies glue in a separate library
+ loaded when needed by dlopen.
+ It make dependencies on libibverbs and libmlx4 optional,
+ and has no performance impact.
- It works by moving these dependencies to a purpose-built rdma-core "glue"
- plug-in which must either be installed in a directory whose name is based
- on ``CONFIG_RTE_EAL_PMD_PATH`` suffixed with ``-glue`` if set, or in a
- standard location for the dynamic linker (e.g. ``/lib``) if left to the
- default empty string (``""``).
-
- This option has no performance impact.
-
-- ``CONFIG_RTE_IBVERBS_LINK_STATIC`` (default **n**)
-
- Embed static flavor of the dependencies **libibverbs** and **libmlx4**
+- ``static``: Embed static flavor of the dependencies libibverbs and libmlx4
in the PMD shared library or the executable static binary.
-- ``CONFIG_RTE_LIBRTE_MLX4_DEBUG`` (default **n**)
-
- Toggle debugging code and stricter compilation flags. Enabling this option
- adds additional run-time checks and debugging messages at the cost of
- lower performance.
Environment variables
~~~~~~~~~~~~~~~~~~~~~
A list of directories in which to search for the rdma-core "glue" plug-in,
separated by colons or semi-colons.
- Only matters when compiled with ``CONFIG_RTE_IBVERBS_LINK_DLOPEN``
- enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set,
- since ``LD_LIBRARY_PATH`` has no effect in this case.
Run-time configuration
~~~~~~~~~~~~~~~~~~~~~~
.. _`RDMA core installation documentation`: https://raw.githubusercontent.com/linux-rdma/rdma-core/master/README.md
-If rdma-core libraries are built but not installed, DPDK makefile can link them,
-thanks to these environment variables:
-
- - ``EXTRA_CFLAGS=-I/path/to/rdma-core/build/include``
- - ``EXTRA_LDFLAGS=-L/path/to/rdma-core/build/lib``
- - ``PKG_CONFIG_PATH=/path/to/rdma-core/build/lib/pkgconfig``
-
.. _Mellanox_OFED_as_a_fallback:
Mellanox OFED as a fallback
2. Install the required libraries and kernel modules either by installing
only the required set, or by installing the entire Mellanox OFED:
- For bare metal use:
-
- .. code-block:: console
+ For bare metal use::
./mlnxofedinstall --dpdk --upstream-libs
- For SR-IOV hypervisors use:
-
- .. code-block:: console
+ For SR-IOV hypervisors use::
./mlnxofedinstall --dpdk --upstream-libs --enable-sriov --hypervisor
- For SR-IOV virtual machine use:
-
- .. code-block:: console
+ For SR-IOV virtual machine use::
./mlnxofedinstall --dpdk --upstream-libs --guest
-3. Verify the firmware is the correct one:
-
- .. code-block:: console
+3. Verify the firmware is the correct one::
ibv_devinfo
-4. Set all ports links to Ethernet, follow instructions on the screen:
-
- .. code-block:: console
+4. Set all ports links to Ethernet, follow instructions on the screen::
connectx_port_config
5. Continue with :ref:`section 2 of the Quick Start Guide <QSG_2>`.
-Supported NICs
---------------
-
-* Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G)
-
.. _qsg:
Quick Start Guide
-----------------
-1. Set all ports links to Ethernet
-
- .. code-block:: console
+1. Set all ports links to Ethernet::
PCI=<NIC PCI address>
echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port0"
.. _QSG_2:
2. In case of bare metal or hypervisor, configure optimized steering mode
- by adding the following line to ``/etc/modprobe.d/mlx4_core.conf``:
-
- .. code-block:: console
+ by adding the following line to ``/etc/modprobe.d/mlx4_core.conf``::
options mlx4_core log_num_mgm_entry_size=-7
If VLAN filtering is used, set log_num_mgm_entry_size=-1.
Performance degradation can occur on this case.
-3. Restart the driver:
-
- .. code-block:: console
+3. Restart the driver::
/etc/init.d/openibd restart
- or:
-
- .. code-block:: console
+ or::
service openibd restart
-4. Compile DPDK and you are ready to go. See instructions on
- :ref:`Development Kit Build System <Development_Kit_Build_System>`
+4. Install DPDK and you are ready to go.
+ See :doc:`compilation instructions <../linux_gsg/build_dpdk>`.
Performance tuning
------------------
-1. Verify the optimized steering mode is configured:
-
- .. code-block:: console
+1. Verify the optimized steering mode is configured::
cat /sys/module/mlx4_core/parameters/log_num_mgm_entry_size
2. Use the CPU near local NUMA node to which the PCIe adapter is connected,
for better performance. For VMs, verify that the right CPU
- and NUMA node are pinned according to the above. Run:
-
- .. code-block:: console
+ and NUMA node are pinned according to the above. Run::
lstopo-no-graphics
This in order to forward packets from one to the other without
NUMA performance penalty.
-4. Disable pause frames:
-
- .. code-block:: console
+4. Disable pause frames::
ethtool -A <netdev> rx off tx off
to set the PCI max read request parameter to 1K. This can be
done in the following way:
- To query the read request size use:
-
- .. code-block:: console
+ To query the read request size use::
setpci -s <NIC PCI address> 68.w
- If the output is different than 3XXX, set it by:
-
- .. code-block:: console
+ If the output is different than 3XXX, set it by::
setpci -s <NIC PCI address> 68.w=3XXX
This section demonstrates how to launch **testpmd** with Mellanox ConnectX-3
devices managed by librte_pmd_mlx4.
-#. Load the kernel modules:
-
- .. code-block:: console
+#. Load the kernel modules::
modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
Alternatively if MLNX_OFED is fully installed, the following script can
- be run:
-
- .. code-block:: console
+ be run::
/etc/init.d/openibd restart
not have to be loaded.
#. Make sure Ethernet interfaces are in working order and linked to kernel
- verbs. Related sysfs entries should be present:
-
- .. code-block:: console
+ verbs. Related sysfs entries should be present::
ls -d /sys/class/net/*/device/infiniband_verbs/uverbs* | cut -d / -f 5
- Example output:
-
- .. code-block:: console
+ Example output::
eth2
eth3
eth4
eth5
-#. Optionally, retrieve their PCI bus addresses for whitelisting:
-
- .. code-block:: console
+#. Optionally, retrieve their PCI bus addresses for whitelisting::
{
for intf in eth2 eth3 eth4 eth5;
} |
sed -n 's,.*/\(.*\),-w \1,p'
- Example output:
-
- .. code-block:: console
+ Example output::
-w 0000:83:00.0
-w 0000:83:00.0
There are only two distinct PCI bus addresses because the Mellanox
ConnectX-3 adapters installed on this system are dual port.
-#. Request huge pages:
-
- .. code-block:: console
+#. Request huge pages::
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
-#. Start testpmd with basic parameters:
-
- .. code-block:: console
+#. Start testpmd with basic parameters::
testpmd -l 8-15 -n 4 -w 0000:83:00.0 -w 0000:84:00.0 -- --rxq=2 --txq=2 -i
- Example output:
-
- .. code-block:: console
+ Example output::
[...]
EAL: PCI device 0000:83:00.0 on NUMA socket 1
git.droids-corp.org / dpdk.git / blobdiff