-.. BSD LICENSE
-
- Copyright (c) 2015-2018 Amazon.com, Inc. or its affiliates.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
All rights reserved.
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Amazon.com, Inc. nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
ENA Poll Mode Driver
====================
Configuration information
-------------------------
-**DPDK Configuration Parameters**
-
- The following configuration options are available for the ENA PMD:
+**Runtime Configuration Parameters**
- * **CONFIG_RTE_LIBRTE_ENA_PMD** (default y): Enables or disables inclusion
- of the ENA PMD driver in the DPDK compilation.
+ * **large_llq_hdr** (default 0)
- * **CONFIG_RTE_LIBRTE_ENA_DEBUG_RX** (default n): Enables or disables debug
- logging of RX logic within the ENA PMD driver.
-
- * **CONFIG_RTE_LIBRTE_ENA_DEBUG_TX** (default n): Enables or disables debug
- logging of TX logic within the ENA PMD driver.
-
- * **CONFIG_RTE_LIBRTE_ENA_COM_DEBUG** (default n): Enables or disables debug
- logging of low level tx/rx logic in ena_com(base) within the ENA PMD driver.
+ Enables or disables usage of large LLQ headers. This option will have
+ effect only if the device also supports large LLQ headers. Otherwise, the
+ default value will be used.
**ENA Configuration Parameters**
Current ENA PMD supports the following ENA adapters including:
* ``1d0f:ec20`` - ENA VF
-* ``1d0f:ec21`` - ENA VF with LLQ support
+* ``1d0f:ec21`` - ENA VF RSERV0
Supported Operating Systems
---------------------------
* LSC event notification
* Watchdog (requires handling of timers in the application)
* Device reset upon failure
+* Rx interrupts
Prerequisites
-------------
(*) ENAv2 hardware supports Low Latency Queue v2 (LLQv2). This feature
reduces the latency of the packets by pushing the header directly through
the PCI to the device, before the DMA is even triggered. For proper work
- kernel PCI driver must support write combining (WC). In mainline version of
- ``igb_uio`` (in DPDK repo) it must be enabled by loading module with
+ kernel PCI driver must support write combining (WC).
+ In DPDK ``igb_uio`` it must be enabled by loading module with
``wc_activate=1`` flag (example below). However, mainline's vfio-pci
driver in kernel doesn't have WC support yet (planed to be added).
If vfio-pci used user should be either turn off ENAv2 (to avoid performance
echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-#. Bind the intended ENA device to ``vfio-pci`` or ``igb_uio`` module.
+ To use ``noiommu`` mode, the ``vfio-pci`` must be built with flag
+ ``CONFIG_VFIO_NOIOMMU``.
+#. Bind the intended ENA device to ``vfio-pci`` or ``igb_uio`` module.
At this point the system should be ready to run DPDK applications. Once the
-application runs to completion, the ENA can be detached from igb_uio if necessary.
+application runs to completion, the ENA can be detached from attached module if
+necessary.
+
+**Rx interrupts support**
+
+ENA PMD supports Rx interrupts, which can be used to wake up lcores waiting for
+input. Please note that it won't work with ``igb_uio``, so to use this feature,
+the ``vfio-pci`` should be used.
+
+ENA handles admin interrupts and AENQ notifications on separate interrupt.
+There is possibility that there won't be enough event file descriptors to
+handle both admin and Rx interrupts. In that situation the Rx interrupt request
+will fail.
+
+**Note about usage on \*.metal instances**
+
+On AWS, the metal instances are supporting IOMMU for both arm64 and x86_64
+hosts.
+
+* x86_64 (e.g. c5.metal, i3.metal):
+ IOMMU should be disabled by default. In that situation, the ``igb_uio`` can
+ be used as it is but ``vfio-pci`` should be working in no-IOMMU mode (please
+ see above).
+
+ When IOMMU is enabled, ``igb_uio`` cannot be used as it's not supporting this
+ feature, while ``vfio-pci`` should work without any changes.
+ To enable IOMMU on those hosts, please update ``GRUB_CMDLINE_LINUX`` in file
+ ``/etc/default/grub`` with the below extra boot arguments::
+
+ iommu=1 intel_iommu=on
+
+ Then, make the changes live by executing as a root::
+
+ # grub2-mkconfig > /boot/grub2/grub.cfg
+
+ Finally, reboot should result in IOMMU being enabled.
+
+* arm64 (a1.metal):
+ IOMMU should be enabled by default. Unfortunately, ``vfio-pci`` isn't
+ supporting SMMU, which is implementation of IOMMU for arm64 architecture and
+ ``igb_uio`` isn't supporting IOMMU at all, so to use DPDK with ENA on those
+ hosts, one must disable IOMMU. This can be done by updating
+ ``GRUB_CMDLINE_LINUX`` in file ``/etc/default/grub`` with the extra boot
+ argument::
+
+ iommu.passthrough=1
+
+ Then, make the changes live by executing as a root::
+
+ # grub2-mkconfig > /boot/grub2/grub.cfg
+
+ Finally, reboot should result in IOMMU being disabled.
+ Without IOMMU, ``igb_uio`` can be used as it is but ``vfio-pci`` should be
+ working in no-IOMMU mode (please see above).
Usage example
-------------
Follow instructions available in the document
:ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to launch
-**testpmd** with Amazon ENA devices managed by librte_pmd_ena.
+**testpmd** with Amazon ENA devices managed by librte_net_ena.
Example output:
git.droids-corp.org / dpdk.git / blobdiff