I40E Poll Mode Driver
======================
-The I40E PMD (librte_pmd_i40e) provides poll mode driver support
-for the Intel X710/XL710/X722 10/40 Gbps family of adapters.
+The i40e PMD (librte_pmd_i40e) provides poll mode driver support for
+10/25/40 Gbps Intel® Ethernet 700 Series Network Adapters based on
+the Intel Ethernet Controller X710/XL710/XXV710 and Intel Ethernet
+Connection X722 (only support part of features).
Features
--------
-Features of the I40E PMD are:
+Features of the i40e PMD are:
- Multiple queues for TX and RX
- Receiver Side Scaling (RSS)
- VF Daemon (VFD) - EXPERIMENTAL
- Dynamic Device Personalization (DDP)
- Queue region configuration
+- Virtual Function Port Representors
+- Malicious Device Drive event catch and notify
+- Generic flow API
Prerequisites
-------------
section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
- Upgrade the NVM/FW version following the `Intel® Ethernet NVM Update Tool Quick Usage Guide for Linux
- <https://www-ssl.intel.com/content/www/us/en/embedded/products/networking/nvm-update-tool-quick-linux-usage-guide.html>`_ if needed.
+ <https://www-ssl.intel.com/content/www/us/en/embedded/products/networking/nvm-update-tool-quick-linux-usage-guide.html>`_ and `Intel® Ethernet NVM Update Tool: Quick Usage Guide for EFI <https://www.intel.com/content/www/us/en/embedded/products/networking/nvm-update-tool-quick-efi-usage-guide.html>`_ if needed.
+
+- For information about supported media, please refer to this document: `Intel® Ethernet Controller X710/XXV710/XL710 Feature Support Matrix
+ <http://www.intel.com/content/dam/www/public/us/en/documents/release-notes/xl710-ethernet-controller-feature-matrix.pdf>`_.
+
+ .. Note::
+
+ * Some adapters based on the Intel(R) Ethernet Controller 700 Series only
+ support Intel Ethernet Optics modules. On these adapters, other modules are not
+ supported and will not function.
+
+ * For connections based on Intel(R) Ethernet Controller 700 Series,
+ support is dependent on your system board. Please see your vendor for details.
+
+ * In all cases Intel recommends using Intel Ethernet Optics; other modules
+ may function but are not validated by Intel. Contact Intel for supported media types.
+
+Recommended Matching List
+-------------------------
+
+It is highly recommended to upgrade the i40e kernel driver and firmware to
+avoid the compatibility issues with i40e PMD. Here is the suggested matching
+list which has been tested and verified. The detailed information can refer
+to chapter Tested Platforms/Tested NICs in release notes.
+
+For X710/XL710/XXV710,
+
+ +--------------+-----------------------+------------------+
+ | DPDK version | Kernel driver version | Firmware version |
+ +==============+=======================+==================+
+ | 20.05 | 2.11.27 | 7.30 |
+ +--------------+-----------------------+------------------+
+ | 20.02 | 2.10.19 | 7.20 |
+ +--------------+-----------------------+------------------+
+ | 19.11 | 2.9.21 | 7.00 |
+ +--------------+-----------------------+------------------+
+ | 19.08 | 2.8.43 | 7.00 |
+ +--------------+-----------------------+------------------+
+ | 19.05 | 2.7.29 | 6.80 |
+ +--------------+-----------------------+------------------+
+ | 19.02 | 2.7.26 | 6.80 |
+ +--------------+-----------------------+------------------+
+ | 18.11 | 2.4.6 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 18.08 | 2.4.6 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 18.05 | 2.4.6 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 18.02 | 2.4.3 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 17.11 | 2.1.26 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 17.08 | 2.0.19 | 6.01 |
+ +--------------+-----------------------+------------------+
+ | 17.05 | 1.5.23 | 5.05 |
+ +--------------+-----------------------+------------------+
+ | 17.02 | 1.5.23 | 5.05 |
+ +--------------+-----------------------+------------------+
+ | 16.11 | 1.5.23 | 5.05 |
+ +--------------+-----------------------+------------------+
+ | 16.07 | 1.4.25 | 5.04 |
+ +--------------+-----------------------+------------------+
+ | 16.04 | 1.4.25 | 5.02 |
+ +--------------+-----------------------+------------------+
+
+
+For X722,
+
+ +--------------+-----------------------+------------------+
+ | DPDK version | Kernel driver version | Firmware version |
+ +==============+=======================+==================+
+ | 20.05 | 2.11.27 | 4.11 |
+ +--------------+-----------------------+------------------+
+ | 20.02 | 2.10.19 | 4.11 |
+ +--------------+-----------------------+------------------+
+ | 19.11 | 2.9.21 | 4.10 |
+ +--------------+-----------------------+------------------+
+ | 19.08 | 2.9.21 | 4.10 |
+ +--------------+-----------------------+------------------+
+ | 19.05 | 2.7.29 | 3.33 |
+ +--------------+-----------------------+------------------+
+ | 19.02 | 2.7.26 | 3.33 |
+ +--------------+-----------------------+------------------+
+ | 18.11 | 2.4.6 | 3.33 |
+ +--------------+-----------------------+------------------+
+
Pre-Installation Configuration
------------------------------
Number of queues reserved for each VMDQ Pool.
-- ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` (default ``-1``)
-
- Interrupt Throttling interval.
-
-
Runtime Config Options
~~~~~~~~~~~~~~~~~~~~~~
-- ``Number of Queues per VF`` (default ``4``)
+- ``Reserved number of Queues per VF`` (default ``4``)
+
+ The number of reserved queue per VF is determined by its host PF. If the
+ PCI address of an i40e PF is aaaa:bb.cc, the number of reserved queues per
+ VF can be configured with EAL parameter like -w aaaa:bb.cc,queue-num-per-vf=n.
+ The value n can be 1, 2, 4, 8 or 16. If no such parameter is configured, the
+ number of reserved queues per VF is 4 by default. If VF request more than
+ reserved queues per VF, PF will able to allocate max to 16 queues after a VF
+ reset.
- The number of queue per VF is determined by its host PF. If the PCI address
- of an i40e PF is aaaa:bb.cc, the number of queues per VF can be configured
- with EAL parameter like -w aaaa:bb.cc,queue-num-per-vf=n. The value n can be
- 1, 2, 4, 8 or 16. If no such parameter is configured, the number of queues
- per VF is 4 by default.
- ``Support multiple driver`` (default ``disable``)
will switch PF interrupt from IntN to Int0 to avoid interrupt conflict between
DPDK and Linux Kernel.
+- ``Support VF Port Representor`` (default ``not enabled``)
+
+ The i40e PF PMD supports the creation of VF port representors for the control
+ and monitoring of i40e virtual function devices. Each port representor
+ corresponds to a single virtual function of that device. Using the ``devargs``
+ option ``representor`` the user can specify which virtual functions to create
+ port representors for on initialization of the PF PMD by passing the VF IDs of
+ the VFs which are required.::
+
+ -w DBDF,representor=[0,1,4]
+
+ Currently hot-plugging of representor ports is not supported so all required
+ representors must be specified on the creation of the PF.
+
+- ``Use latest supported vector`` (default ``disable``)
+
+ Latest supported vector path may not always get the best perf so vector path was
+ recommended to use only on later platform. But users may want the latest vector path
+ since it can get better perf in some real work loading cases. So ``devargs`` param
+ ``use-latest-supported-vec`` is introduced, for example::
+
+ -w 84:00.0,use-latest-supported-vec=1
+
+- ``Enable validation for VF message`` (default ``not enabled``)
+
+ The PF counts messages from each VF. If in any period of seconds the message
+ statistic from a VF exceeds maximal limitation, the PF will ignore any new message
+ from that VF for some seconds.
+ Format -- "maximal-message@period-seconds:ignore-seconds"
+ For example::
+
+ -w 84:00.0,vf_msg_cfg=80@120:180
+
+Vector RX Pre-conditions
+~~~~~~~~~~~~~~~~~~~~~~~~
+For Vector RX it is assumed that the number of descriptor rings will be a power
+of 2. With this pre-condition, the ring pointer can easily scroll back to the
+head after hitting the tail without a conditional check. In addition Vector RX
+can use this assumption to do a bit mask using ``ring_size - 1``.
+
Driver compilation and testing
------------------------------
Floating VEB
~~~~~~~~~~~~~
-The Intel® Ethernet Controller X710 and XL710 Family support a feature called
+The Intel® Ethernet 700 Series support a feature called
"Floating VEB".
A Virtual Ethernet Bridge (VEB) is an IEEE Edge Virtual Bridging (EVB) term
Dynamic Device Personalization (DDP)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The Intel® Ethernet Controller X*710 support a feature called "Dynamic Device
-Personalization (DDP)", which is used to configure hardware by downloading
-a profile to support protocols/filters which are not supported by default.
-The DDP functionality requires a NIC firmware version of 6.0 or greater.
+The Intel® Ethernet 700 Series except for the Intel Ethernet Connection
+X722 support a feature called "Dynamic Device Personalization (DDP)",
+which is used to configure hardware by downloading a profile to support
+protocols/filters which are not supported by default. The DDP
+functionality requires a NIC firmware version of 6.0 or greater.
-Current implementation supports MPLSoUDP/MPLSoGRE/GTP-C/GTP-U/PPPoE/PPPoL2TP,
+Current implementation supports GTP-C/GTP-U/PPPoE/PPPoL2TP/ESP,
steering can be used with rte_flow API.
-Load a profile which supports MPLSoUDP/MPLSoGRE and store backup profile:
+GTPv1 package is released, and it can be downloaded from
+https://downloadcenter.intel.com/download/27587.
+
+PPPoE package is released, and it can be downloaded from
+https://downloadcenter.intel.com/download/28040.
+
+ESP-AH package is released, and it can be downloaded from
+https://downloadcenter.intel.com/download/29446.
+
+Load a profile which supports GTP and store backup profile:
.. code-block:: console
- testpmd> ddp add 0 ./mpls.pkgo,./backup.pkgo
+ testpmd> ddp add 0 ./gtp.pkgo,./backup.pkgo
-Delete a MPLS profile and restore backup profile:
+Delete a GTP profile and restore backup profile:
.. code-block:: console
testpmd> ddp get list 0
-Display information about a MPLS profile:
+Display information about a GTP profile:
.. code-block:: console
- testpmd> ddp get info ./mpls.pkgo
+ testpmd> ddp get info ./gtp.pkgo
Input set configuration
~~~~~~~~~~~~~~~~~~~~~~~
Queue region configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The Ethernet Controller X710/XL710 supports a feature of queue regions
+The Intel® Ethernet 700 Series supports a feature of queue regions
configuration for RSS in the PF, so that different traffic classes or
different packet classification types can be separated to different
queues in different queue regions. There is an API for configuration
testpmd> set port (port_id) queue-region flush (on|off)
testpmd> show port (port_id) queue-region
+Generic flow API
+~~~~~~~~~~~~~~~~~~~
+
+- ``RSS Flow``
+
+ RSS Flow supports to set hash input set, hash function, enable hash
+ and configure queue region.
+ For example:
+ Configure queue region as queue 0, 1, 2, 3.
+
+ .. code-block:: console
+
+ testpmd> flow create 0 ingress pattern end actions rss types end \
+ queues 0 1 2 3 end / end
+
+ Enable hash and set input set for ipv4-tcp.
+
+ .. code-block:: console
+
+ testpmd> flow create 0 ingress pattern eth / ipv4 / tcp / end \
+ actions rss types ipv4-tcp l3-src-only end queues end / end
+
+ Set symmetric hash enable for flow type ipv4-tcp.
+
+ .. code-block:: console
+
+ testpmd> flow create 0 ingress pattern eth / ipv4 / tcp / end \
+ actions rss types ipv4-tcp end queues end func symmetric_toeplitz / end
+
+ Set hash function as simple xor.
+
+ .. code-block:: console
+
+ testpmd> flow create 0 ingress pattern end actions rss types end \
+ queues end func simple_xor / end
+
Limitations or Known issues
---------------------------
-MPLS packet classification on X710/XL710
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+MPLS packet classification
+~~~~~~~~~~~~~~~~~~~~~~~~~~
For firmware versions prior to 5.0, MPLS packets are not recognized by the NIC.
The L2 Payload flow type in flow director can be used to classify MPLS packet
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When a packet is over maximum frame size, the packet is dropped.
-However the Rx statistics, when calling `rte_eth_stats_get` incorrectly
+However, the Rx statistics, when calling `rte_eth_stats_get` incorrectly
shows it as received.
VF & TC max bandwidth setting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The per VF max bandwidth and per TC max bandwidth cannot be enabled in parallel.
-The dehavior is different when handling per VF and per TC max bandwidth setting.
+The behavior is different when handling per VF and per TC max bandwidth setting.
When enabling per VF max bandwidth, SW will check if per TC max bandwidth is
enabled. If so, return failure.
When enabling per TC max bandwidth, SW will check if per VF max bandwidth
TC TX scheduling mode setting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-There're 2 TX scheduling modes for TCs, round robin and strict priority mode.
+There are 2 TX scheduling modes for TCs, round robin and strict priority mode.
If a TC is set to strict priority mode, it can consume unlimited bandwidth.
It means if APP has set the max bandwidth for that TC, it comes to no
effect.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To reach maximum NIC performance in the VF the PCI extended tag must be
-enabled. The DPDK I40E PF driver will set this feature during initialization,
+enabled. The DPDK i40e PF driver will set this feature during initialization,
but the kernel PF driver does not. So when running traffic on a VF which is
managed by the kernel PF driver, a significant NIC performance downgrade has
-been observed (for 64 byte packets, there is about 25% linerate downgrade for
-a 25G device and about 35% for a 40G device).
+been observed (for 64 byte packets, there is about 25% line-rate downgrade for
+a 25GbE device and about 35% for a 40GbE device).
For kernel version >= 4.11, the kernel's PCI driver will enable the extended
tag if it detects that the device supports it. So by default, this is not an
kernel driver, which don't want to use the TPID.
So PMD reports warning to clarify what is changed by writing global register.
-High Performance of Small Packets on 40G NIC
---------------------------------------------
+High Performance of Small Packets on 40GbE NIC
+----------------------------------------------
As there might be firmware fixes for performance enhancement in latest version
of firmware image, the firmware update might be needed for getting high performance.
-Check with the local Intel's Network Division application engineers for firmware updates.
+Check the Intel support website for the latest firmware updates.
Users should consult the release notes specific to a DPDK release to identify
the validated firmware version for a NIC using the i40e driver.
As i40e PMD supports both 16 and 32 bytes RX descriptor sizes, and 16 bytes size can provide helps to high performance of small packets.
Configuration of ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` in config files can be changed to use 16 bytes size RX descriptors.
-High Performance and per Packet Latency Tradeoff
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Input set requirement of each pctype for FDIR
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Due to the hardware design, the interrupt signal inside NIC is needed for per
-packet descriptor write-back. The minimum interval of interrupts could be set
-at compile time by ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` in configuration files.
-Though there is a default configuration, the interval could be tuned by the
-users with that configuration item depends on what the user cares about more,
-performance or per packet latency.
+Each PCTYPE can only have one specific FDIR input set at one time.
+For example, if creating 2 rte_flow rules with different input set for one PCTYPE,
+it will fail and return the info "Conflict with the first rule's input set",
+which means the current rule's input set conflicts with the first rule's.
+Remove the first rule if want to change the input set of the PCTYPE.
Example of getting best performance with l3fwd example
------------------------------------------------------
-The following is an example of running the DPDK ``l3fwd`` sample application to get high performance with an
-Intel server platform and Intel XL710 NICs.
+The following is an example of running the DPDK ``l3fwd`` sample application to get high performance with a
+server with Intel Xeon processors and Intel Ethernet CNA XL710.
-The example scenario is to get best performance with two Intel XL710 40GbE ports.
+The example scenario is to get best performance with two Intel Ethernet CNA XL710 40GbE ports.
See :numref:`figure_intel_perf_test_setup` for the performance test setup.
.. _figure_intel_perf_test_setup:
Performance Test Setup
-1. Add two Intel XL710 NICs to the platform, and use one port per card to get best performance.
- The reason for using two NICs is to overcome a PCIe Gen3's limitation since it cannot provide 80G bandwidth
- for two 40G ports, but two different PCIe Gen3 x8 slot can.
+1. Add two Intel Ethernet CNA XL710 to the platform, and use one port per card to get best performance.
+ The reason for using two NICs is to overcome a PCIe v3.0 limitation since it cannot provide 80GbE bandwidth
+ for two 40GbE ports, but two different PCIe v3.0 x8 slot can.
Refer to the sample NICs output above, then we can select ``82:00.0`` and ``85:00.0`` as test ports::
82:00.0 Ethernet [0200]: Intel XL710 for 40GbE QSFP+ [8086:1583]
4. Bind these two ports to igb_uio.
-5. As to XL710 40G port, we need at least two queue pairs to achieve best performance, then two queues per port
+5. As to Intel Ethernet CNA XL710 40GbE port, we need at least two queue pairs to achieve best performance, then two queues per port
will be required, and each queue pair will need a dedicated CPU core for receiving/transmitting packets.
6. The DPDK sample application ``l3fwd`` will be used for performance testing, with using two ports for bi-directional forwarding.
* Start creating a stream on packet generator.
* Set the Ethernet II type to 0x0800.
+
+Tx bytes affected by the link status change
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For firmware versions prior to 6.01 for X710 series and 3.33 for X722 series, the tx_bytes statistics data is affected by
+the link down event. Each time the link status changes to down, the tx_bytes decreases 110 bytes.