-.. BSD LICENSE
- Copyright 2015-2017 Chelsio Communications.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Chelsio Communications nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2014-2018 Chelsio Communications.
+ All rights reserved.
CXGBE Poll Mode Driver
======================
for **Chelsio Terminator** 10/25/40/100 Gbps family of adapters. CXGBE PMD
has support for the latest Linux and FreeBSD operating systems.
+CXGBEVF PMD provides poll mode driver support for SR-IOV Virtual functions
+and has support for the latest Linux operating systems.
+
More information can be found at `Chelsio Communications Official Website
<http://www.chelsio.com>`_.
Features
--------
-CXGBE PMD has support for:
+CXGBE and CXGBEVF PMD has support for:
- Multiple queues for TX and RX
- Receiver Side Steering (RSS)
- All multicast mode
- Port hardware statistics
- Jumbo frames
+- Flow API - Support for both Wildcard (LE-TCAM) and Exact (HASH) match filters.
Limitations
-----------
For this reason, one cannot whitelist/blacklist a single port without
whitelisting/blacklisting the other ports on the same device.
+.. _t5-nics:
+
Supported Chelsio T5 NICs
-------------------------
- 40G NICs: T580-CR, T580-LP-CR, T580-SO-CR
- Other T5 NICs: T522-CR
+.. _t6-nics:
+
Supported Chelsio T6 NICs
-------------------------
- 25G NICs: T6425-CR, T6225-CR, T6225-LL-CR, T6225-SO-CR
- 100G NICs: T62100-CR, T62100-LP-CR, T62100-SO-CR
+Supported SR-IOV Chelsio NICs
+-----------------------------
+
+SR-IOV virtual functions are supported on all the Chelsio NICs listed
+in :ref:`t5-nics` and :ref:`t6-nics`.
+
Prerequisites
-------------
-- Requires firmware version **1.17.14.0** and higher. Visit
+- Requires firmware version **1.24.11.0** and higher. Visit
`Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware
bundled with the latest Chelsio Unified Wire package.
Toggle compilation of librte_pmd_cxgbe driver.
-- ``CONFIG_RTE_LIBRTE_CXGBE_DEBUG`` (default **n**)
+ .. note::
+
+ This controls compilation of both CXGBE and CXGBEVF PMD.
+
+Runtime Options
+~~~~~~~~~~~~~~~
+
+The following ``devargs`` options can be enabled at runtime. They must
+be passed as part of EAL arguments. For example,
+
+.. code-block:: console
+
+ testpmd -w 02:00.4,keep_ovlan=1 -- -i
+
+Common Runtime Options
+^^^^^^^^^^^^^^^^^^^^^^
+
+- ``keep_ovlan`` (default **0**)
+
+ Toggle behavior to keep/strip outer VLAN in Q-in-Q packets. If
+ enabled, the outer VLAN tag is preserved in Q-in-Q packets. Otherwise,
+ the outer VLAN tag is stripped in Q-in-Q packets.
+
+- ``tx_mode_latency`` (default **0**)
+
+ When set to 1, Tx doesn't wait for max number of packets to get
+ coalesced and sends the packets immediately at the end of the
+ current Tx burst. When set to 0, Tx waits across multiple Tx bursts
+ until the max number of packets have been coalesced. In this case,
+ Tx only sends the coalesced packets to hardware once the max
+ coalesce limit has been reached.
+
+CXGBE VF Only Runtime Options
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- ``force_link_up`` (default **0**)
+
+ When set to 1, CXGBEVF PMD always forces link as up for all VFs on
+ underlying Chelsio NICs. This enables multiple VFs on the same NIC
+ to send traffic to each other even when the physical link is down.
+
+CXGBE PF Only Runtime Options
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- ``filtermode`` (default **0**)
+
+ Apart from the 4-tuple (IP src/dst addresses and TCP/UDP src/dst port
+ addresses), there are only 40-bits available to match other fields in
+ packet headers. So, ``filtermode`` devarg allows user to dynamically
+ select a 40-bit supported match field combination for LETCAM (wildcard)
+ filters.
+
+ Default value of **0** makes driver pick the combination configured in
+ the firmware configuration file on the adapter.
+
+ The supported flags and their corresponding values are shown in table below.
+ These flags can be OR'd to create 1 of the multiple supported combinations
+ for LETCAM filters.
+
+ ================== ======
+ FLAG VALUE
+ ================== ======
+ Physical Port 0x1
+ PFVF 0x2
+ Destination MAC 0x4
+ Ethertype 0x8
+ Inner VLAN 0x10
+ Outer VLAN 0x20
+ IP TOS 0x40
+ IP Protocol 0x80
+ ================== ======
+
+ The supported ``filtermode`` combinations and their corresponding OR'd
+ values are shown in table below.
+
+ +-----------------------------------+-----------+
+ | FILTERMODE COMBINATIONS | VALUE |
+ +===================================+===========+
+ | Protocol, TOS, Outer VLAN, Port | 0xE1 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, Outer VLAN | 0xE0 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, Inner VLAN, Port | 0xD1 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, Inner VLAN | 0xD0 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, PFVF, Port | 0xC3 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, PFVF | 0xC2 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS, Port | 0xC1 |
+ +-----------------------------------+-----------+
+ | Protocol, TOS | 0xC0 |
+ +-----------------------------------+-----------+
+ | Protocol, Outer VLAN, Port | 0xA1 |
+ +-----------------------------------+-----------+
+ | Protocol, Outer VLAN | 0xA0 |
+ +-----------------------------------+-----------+
+ | Protocol, Inner VLAN, Port | 0x91 |
+ +-----------------------------------+-----------+
+ | Protocol, Inner VLAN | 0x90 |
+ +-----------------------------------+-----------+
+ | Protocol, Ethertype, DstMAC, Port | 0x8D |
+ +-----------------------------------+-----------+
+ | Protocol, Ethertype, DstMAC | 0x8C |
+ +-----------------------------------+-----------+
+ | Protocol, Ethertype, Port | 0x89 |
+ +-----------------------------------+-----------+
+ | Protocol, Ethertype | 0x88 |
+ +-----------------------------------+-----------+
+ | Protocol, DstMAC, PFVF, Port | 0x87 |
+ +-----------------------------------+-----------+
+ | Protocol, DstMAC, PFVF | 0x86 |
+ +-----------------------------------+-----------+
+ | Protocol, DstMAC, Port | 0x85 |
+ +-----------------------------------+-----------+
+ | Protocol, DstMAC | 0x84 |
+ +-----------------------------------+-----------+
+ | Protocol, PFVF, Port | 0x83 |
+ +-----------------------------------+-----------+
+ | Protocol, PFVF | 0x82 |
+ +-----------------------------------+-----------+
+ | Protocol, Port | 0x81 |
+ +-----------------------------------+-----------+
+ | Protocol | 0x80 |
+ +-----------------------------------+-----------+
+ | TOS, Outer VLAN, Port | 0x61 |
+ +-----------------------------------+-----------+
+ | TOS, Outer VLAN | 0x60 |
+ +-----------------------------------+-----------+
+ | TOS, Inner VLAN, Port | 0x51 |
+ +-----------------------------------+-----------+
+ | TOS, Inner VLAN | 0x50 |
+ +-----------------------------------+-----------+
+ | TOS, Ethertype, DstMAC, Port | 0x4D |
+ +-----------------------------------+-----------+
+ | TOS, Ethertype, DstMAC | 0x4C |
+ +-----------------------------------+-----------+
+ | TOS, Ethertype, Port | 0x49 |
+ +-----------------------------------+-----------+
+ | TOS, Ethertype | 0x48 |
+ +-----------------------------------+-----------+
+ | TOS, DstMAC, PFVF, Port | 0x47 |
+ +-----------------------------------+-----------+
+ | TOS, DstMAC, PFVF | 0x46 |
+ +-----------------------------------+-----------+
+ | TOS, DstMAC, Port | 0x45 |
+ +-----------------------------------+-----------+
+ | TOS, DstMAC | 0x44 |
+ +-----------------------------------+-----------+
+ | TOS, PFVF, Port | 0x43 |
+ +-----------------------------------+-----------+
+ | TOS, PFVF | 0x42 |
+ +-----------------------------------+-----------+
+ | TOS, Port | 0x41 |
+ +-----------------------------------+-----------+
+ | TOS | 0x40 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, Inner VLAN, Port | 0x31 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, Ethertype, Port | 0x29 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, Ethertype | 0x28 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, DstMAC, Port | 0x25 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, DstMAC | 0x24 |
+ +-----------------------------------+-----------+
+ | Outer VLAN, Port | 0x21 |
+ +-----------------------------------+-----------+
+ | Outer VLAN | 0x20 |
+ +-----------------------------------+-----------+
+ | Inner VLAN, Ethertype, Port | 0x19 |
+ +-----------------------------------+-----------+
+ | Inner VLAN, Ethertype | 0x18 |
+ +-----------------------------------+-----------+
+ | Inner VLAN, DstMAC, Port | 0x15 |
+ +-----------------------------------+-----------+
+ | Inner VLAN, DstMAC | 0x14 |
+ +-----------------------------------+-----------+
+ | Inner VLAN, Port | 0x11 |
+ +-----------------------------------+-----------+
+ | Inner VLAN | 0x10 |
+ +-----------------------------------+-----------+
+ | Ethertype, DstMAC, Port | 0xD |
+ +-----------------------------------+-----------+
+ | Ethertype, DstMAC | 0xC |
+ +-----------------------------------+-----------+
+ | Ethertype, PFVF, Port | 0xB |
+ +-----------------------------------+-----------+
+ | Ethertype, PFVF | 0xA |
+ +-----------------------------------+-----------+
+ | Ethertype, Port | 0x9 |
+ +-----------------------------------+-----------+
+ | Ethertype | 0x8 |
+ +-----------------------------------+-----------+
+ | DstMAC, PFVF, Port | 0x7 |
+ +-----------------------------------+-----------+
+ | DstMAC, PFVF | 0x6 |
+ +-----------------------------------+-----------+
+ | DstMAC, Port | 0x5 |
+ +-----------------------------------+-----------+
+ | Destination MAC | 0x4 |
+ +-----------------------------------+-----------+
+ | PFVF, Port | 0x3 |
+ +-----------------------------------+-----------+
+ | PFVF | 0x2 |
+ +-----------------------------------+-----------+
+ | Physical Port | 0x1 +
+ +-----------------------------------+-----------+
+
+ For example, to enable matching ``ethertype`` field in Ethernet
+ header, and ``protocol`` field in IPv4 header, the ``filtermode``
+ combination must be given as:
+
+ .. code-block:: console
- Toggle display of generic debugging messages.
+ testpmd -w 02:00.4,filtermode=0x88 -- -i
-- ``CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG`` (default **n**)
+- ``filtermask`` (default **0**)
- Toggle display of registers related run-time check messages.
+ ``filtermask`` devarg works similar to ``filtermode``, but is used
+ to configure a filter mode combination for HASH (exact-match) filters.
-- ``CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX`` (default **n**)
+ .. note::
- Toggle display of firmware mailbox related run-time check messages.
+ The combination chosen for ``filtermask`` devarg **must be a subset** of
+ the combination chosen for ``filtermode`` devarg.
-- ``CONFIG_RTE_LIBRTE_CXGBE_DEBUG_TX`` (default **n**)
+ Default value of **0** makes driver pick the combination configured in
+ the firmware configuration file on the adapter.
- Toggle display of transmission data path run-time check messages.
+ Note that the filter rule will only be inserted in HASH region, if the
+ rule contains **all** the fields specified in the ``filtermask`` combination.
+ Otherwise, the filter rule will get inserted in LETCAM region.
-- ``CONFIG_RTE_LIBRTE_CXGBE_DEBUG_RX`` (default **n**)
+ The same combination list explained in the tables in ``filtermode`` devarg
+ section earlier applies for ``filtermask`` devarg, as well.
- Toggle display of receiving data path run-time check messages.
+ For example, to enable matching only protocol field in IPv4 header, the
+ ``filtermask`` combination must be given as:
-- ``CONFIG_RTE_LIBRTE_CXGBE_TPUT`` (default **y**)
+ .. code-block:: console
- Toggle behaviour to prefer Throughput or Latency.
+ testpmd -w 02:00.4,filtermode=0x88,filtermask=0x80 -- -i
.. _driver-compilation:
.. code-block:: console
- firmware-version: 1.17.14.0, TP 0.1.4.9
+ firmware-version: 1.24.11.0, TP 0.1.23.2
Running testpmd
~~~~~~~~~~~~~~~
EAL: PCI memory mapped at 0x7fd7c0200000
EAL: PCI memory mapped at 0x7fd77cdfd000
EAL: PCI memory mapped at 0x7fd7c10b7000
- PMD: rte_cxgbe_pmd: fw: 1.17.14.0, TP: 0.1.4.9
+ PMD: rte_cxgbe_pmd: fw: 1.24.11.0, TP: 0.1.23.2
PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
Interactive-mode selected
Configuring Port 0 (socket 0)
Flow control pause TX/RX is disabled by default and can be enabled via
testpmd. Refer section :ref:`flow-control` for more details.
+Configuring SR-IOV Virtual Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This section demonstrates how to enable SR-IOV virtual functions
+on Chelsio NICs and demonstrates how to run testpmd with SR-IOV
+virtual functions.
+
+#. Load the kernel module:
+
+ .. code-block:: console
+
+ modprobe cxgb4
+
+#. Get the PCI bus addresses of the interfaces bound to cxgb4 driver:
+
+ .. code-block:: console
+
+ dmesg | tail -2
+
+ Example output:
+
+ .. code-block:: console
+
+ cxgb4 0000:02:00.4 p1p1: renamed from eth0
+ cxgb4 0000:02:00.4 p1p2: renamed from eth1
+
+ .. note::
+
+ Both the interfaces of a Chelsio 2-port adapter are bound to the
+ same PCI bus address.
+
+#. Use ifconfig to get the interface name assigned to Chelsio card:
+
+ .. code-block:: console
+
+ ifconfig -a | grep "00:07:43"
+
+ Example output:
+
+ .. code-block:: console
+
+ p1p1 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C0
+ p1p2 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C8
+
+#. Bring up the interfaces:
+
+ .. code-block:: console
+
+ ifconfig p1p1 up
+ ifconfig p1p2 up
+
+#. Instantiate SR-IOV Virtual Functions. PF0..3 can be used for
+ SR-IOV VFs. Multiple VFs can be instantiated on each of PF0..3.
+ To instantiate one SR-IOV VF on each PF0 and PF1:
+
+ .. code-block:: console
+
+ echo 1 > /sys/bus/pci/devices/0000\:02\:00.0/sriov_numvfs
+ echo 1 > /sys/bus/pci/devices/0000\:02\:00.1/sriov_numvfs
+
+#. Get the PCI bus addresses of the virtual functions:
+
+ .. code-block:: console
+
+ lspci | grep -i "Chelsio" | grep -i "VF"
+
+ Example output:
+
+ .. code-block:: console
+
+ 02:01.0 Ethernet controller: Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller [VF]
+ 02:01.1 Ethernet controller: Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller [VF]
+
+#. Running testpmd
+
+ Follow instructions available in the document
+ :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+ to bind virtual functions and run testpmd.
+
+ Example output:
+
+ .. code-block:: console
+
+ [...]
+ EAL: PCI device 0000:02:01.0 on NUMA socket 0
+ EAL: probe driver: 1425:5803 net_cxgbevf
+ PMD: rte_cxgbe_pmd: Firmware version: 1.24.11.0
+ PMD: rte_cxgbe_pmd: TP Microcode version: 0.1.23.2
+ PMD: rte_cxgbe_pmd: Chelsio rev 0
+ PMD: rte_cxgbe_pmd: No bootstrap loaded
+ PMD: rte_cxgbe_pmd: No Expansion ROM loaded
+ PMD: rte_cxgbe_pmd: 0000:02:01.0 Chelsio rev 0 1G/10GBASE-SFP
+ EAL: PCI device 0000:02:01.1 on NUMA socket 0
+ EAL: probe driver: 1425:5803 net_cxgbevf
+ PMD: rte_cxgbe_pmd: Firmware version: 1.24.11.0
+ PMD: rte_cxgbe_pmd: TP Microcode version: 0.1.23.2
+ PMD: rte_cxgbe_pmd: Chelsio rev 0
+ PMD: rte_cxgbe_pmd: No bootstrap loaded
+ PMD: rte_cxgbe_pmd: No Expansion ROM loaded
+ PMD: rte_cxgbe_pmd: 0000:02:01.1 Chelsio rev 0 1G/10GBASE-SFP
+ Configuring Port 0 (socket 0)
+ Port 0: 06:44:29:44:40:00
+ Configuring Port 1 (socket 0)
+ Port 1: 06:44:29:44:40:10
+ Checking link statuses...
+ Done
+ testpmd>
+
FreeBSD
-------
.. code-block:: console
- dev.t5nex.0.firmware_version: 1.17.14.0
+ dev.t5nex.0.firmware_version: 1.24.11.0
Running testpmd
~~~~~~~~~~~~~~~
.. code-block:: console
- cp x86_64-native-bsdapp-clang/kmod/contigmem.ko /boot/kernel/
+ cp x86_64-native-freebsd-clang/kmod/contigmem.ko /boot/kernel/
#. Add the following lines to /boot/loader.conf:
.. code-block:: console
- kldload ./x86_64-native-bsdapp-clang/kmod/nic_uio.ko
+ kldload ./x86_64-native-freebsd-clang/kmod/nic_uio.ko
#. Start testpmd with basic parameters:
.. code-block:: console
- ./x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4 -w 0000:02:00.4 -- -i
+ ./x86_64-native-freebsd-clang/app/testpmd -l 0-3 -n 4 -w 0000:02:00.4 -- -i
Example output:
EAL: PCI memory mapped at 0x8007ec000
EAL: PCI memory mapped at 0x842800000
EAL: PCI memory mapped at 0x80086c000
- PMD: rte_cxgbe_pmd: fw: 1.17.14.0, TP: 0.1.4.9
+ PMD: rte_cxgbe_pmd: fw: 1.24.11.0, TP: 0.1.23.2
PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
Interactive-mode selected
Configuring Port 0 (socket 0)
git.droids-corp.org / dpdk.git / blobdiff