1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(C) 2021 Marvell.
7 The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver
8 support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K**
9 SoC family as well as for their virtual functions (VF) in SR-IOV context.
11 More information can be found at `Marvell Official Website
12 <https://www.marvell.com/embedded-processors/infrastructure-processors>`_.
17 Features of the CNXK Ethdev PMD are:
19 - Packet type information
24 - Multiple queues for TX and RX
25 - Receiver Side Scaling (RSS)
28 - Inner and Outer Checksum offload
29 - Port hardware statistics
30 - Link state information
33 - Scatter-Gather IO support
34 - Vector Poll mode driver
35 - Debug utilities - Context dump and error interrupt support
36 - Support Rx interrupt
37 - Inline IPsec processing support
42 See :doc:`../platform/cnxk` for setup information.
45 Driver compilation and testing
46 ------------------------------
48 Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
53 Follow instructions available in the document
54 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
59 .. code-block:: console
61 ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1
62 EAL: Detected 4 lcore(s)
63 EAL: Detected 1 NUMA nodes
64 EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
65 EAL: Selected IOVA mode 'VA'
66 EAL: No available hugepages reported in hugepages-16777216kB
67 EAL: No available hugepages reported in hugepages-2048kB
68 EAL: Probing VFIO support...
69 EAL: VFIO support initialized
70 EAL: using IOMMU type 1 (Type 1)
71 [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98
72 EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0)
74 EAL: No legacy callbacks, legacy socket not created
75 testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
76 testpmd: preferred mempool ops selected: cn10k_mempool_ops
77 Configuring Port 0 (socket 0)
78 PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex
80 Port 0: link state change event
81 Port 0: 96:D4:99:72:A5:BF
82 Checking link statuses...
84 No commandline core given, start packet forwarding
85 io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native
86 Logical Core 3 (socket 0) forwards packets on 1 streams:
87 RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
89 io packet forwarding packets/burst=32
90 nb forwarding cores=1 - nb forwarding ports=1
91 port 0: RX queue number: 1 Tx queue number: 1
92 Rx offloads=0x0 Tx offloads=0x10000
94 RX desc=4096 - RX free threshold=0
95 RX threshold registers: pthresh=0 hthresh=0 wthresh=0
98 TX desc=512 - TX free threshold=0
99 TX threshold registers: pthresh=0 hthresh=0 wthresh=0
100 TX offloads=0x0 - TX RS bit threshold=0
103 Runtime Config Options
104 ----------------------
106 - ``Rx&Tx scalar mode enable`` (default ``0``)
108 PMD supports both scalar and vector mode, it may be selected at runtime
109 using ``scalar_enable`` ``devargs`` parameter.
111 - ``RSS reta size`` (default ``64``)
113 RSS redirection table size may be configured during runtime using ``reta_size``
114 ``devargs`` parameter.
118 -a 0002:02:00.0,reta_size=256
120 With the above configuration, reta table of size 256 is populated.
122 - ``Flow priority levels`` (default ``3``)
124 RTE Flow priority levels can be configured during runtime using
125 ``flow_max_priority`` ``devargs`` parameter.
129 -a 0002:02:00.0,flow_max_priority=10
131 With the above configuration, priority level was set to 10 (0-9). Max
132 priority level supported is 32.
134 - ``Reserve Flow entries`` (default ``8``)
136 RTE flow entries can be pre allocated and the size of pre allocation can be
137 selected runtime using ``flow_prealloc_size`` ``devargs`` parameter.
141 -a 0002:02:00.0,flow_prealloc_size=4
143 With the above configuration, pre alloc size was set to 4. Max pre alloc
144 size supported is 32.
146 - ``Max SQB buffer count`` (default ``512``)
148 Send queue descriptor buffer count may be limited during runtime using
149 ``max_sqb_count`` ``devargs`` parameter.
153 -a 0002:02:00.0,max_sqb_count=64
155 With the above configuration, each send queue's descriptor buffer count is
156 limited to a maximum of 64 buffers.
158 - ``Switch header enable`` (default ``none``)
160 A port can be configured to a specific switch header type by using
161 ``switch_header`` ``devargs`` parameter.
165 -a 0002:02:00.0,switch_header="higig2"
167 With the above configuration, higig2 will be enabled on that port and the
168 traffic on this port should be higig2 traffic only. Supported switch header
169 types are "chlen24b", "chlen90b", "dsa", "exdsa", "higig2" and "vlan_exdsa".
171 - ``RSS tag as XOR`` (default ``0``)
173 The HW gives two options to configure the RSS adder i.e
175 * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>``
177 * ``rss_adder<7:0> = flow_tag<7:0>``
179 Latter one aligns with standard NIC behavior vs former one is a legacy
180 RSS adder scheme used in OCTEON TX2 products.
182 By default, the driver runs in the latter mode.
183 Setting this flag to 1 to select the legacy mode.
185 For example to select the legacy mode(RSS tag adder as XOR)::
187 -a 0002:02:00.0,tag_as_xor=1
189 - ``Max SPI for inbound inline IPsec`` (default ``255``)
191 Max SPI supported for inbound inline IPsec processing can be specified by
192 ``ipsec_in_max_spi`` ``devargs`` parameter.
196 -a 0002:02:00.0,ipsec_in_max_spi=128
198 With the above configuration, application can enable inline IPsec processing
199 for 128 inbound SAs (SPI 0-127).
201 - ``Max SA's for outbound inline IPsec`` (default ``4096``)
203 Max number of SA's supported for outbound inline IPsec processing can be
204 specified by ``ipsec_out_max_sa`` ``devargs`` parameter.
208 -a 0002:02:00.0,ipsec_out_max_sa=128
210 With the above configuration, application can enable inline IPsec processing
211 for 128 outbound SAs.
213 - ``Outbound CPT LF queue size`` (default ``8200``)
215 Size of Outbound CPT LF queue in number of descriptors can be specified by
216 ``outb_nb_desc`` ``devargs`` parameter.
220 -a 0002:02:00.0,outb_nb_desc=16384
222 With the above configuration, Outbound CPT LF will be created to accommodate
223 at max 16384 descriptors at any given time.
225 - ``Outbound CPT LF count`` (default ``1``)
227 Number of CPT LF's to attach for Outbound processing can be specified by
228 ``outb_nb_crypto_qs`` ``devargs`` parameter.
232 -a 0002:02:00.0,outb_nb_crypto_qs=2
234 With the above configuration, two CPT LF's are setup and distributed among
235 all the Tx queues for outbound processing.
237 - ``Force using inline ipsec device for inbound`` (default ``0``)
239 In CN10K, in event mode, driver can work in two modes,
241 1. Inbound encrypted traffic received by probed ipsec inline device while
242 plain traffic post decryption is received by ethdev.
244 2. Both Inbound encrypted traffic and plain traffic post decryption are
247 By default event mode works without using inline device i.e mode ``2``.
248 This behaviour can be changed to pick mode ``1`` by using
249 ``force_inb_inl_dev`` ``devargs`` parameter.
253 -a 0002:02:00.0,force_inb_inl_dev=1 -a 0002:03:00.0,force_inb_inl_dev=1
255 With the above configuration, inbound encrypted traffic from both the ports
256 is received by ipsec inline device.
258 - ``Inline IPsec device channel and mask`` (default ``none``)
260 Set channel and channel mask configuration for the inline IPSec device. This
261 will be used when creating flow rules with RTE_FLOW_ACTION_TYPE_SECURITY
264 By default, RTE Flow API sets the channel number of the port on which the
265 rule is created in the MCAM entry and matches it exactly. This behaviour can
266 be modified using the ``inl_cpt_channel`` ``devargs`` parameter.
270 -a 0002:1d:00.0,inl_cpt_channel=0x100/0xf00
272 With the above configuration, RTE Flow rules API will set the channel
273 and channel mask as 0x100 and 0xF00 in the MCAM entries of the flow rules
274 created with RTE_FLOW_ACTION_TYPE_SECURITY action. Since channel number is
275 set with this custom mask, inbound encrypted traffic from all ports with
276 matching channel number pattern will be directed to the inline IPSec device.
280 Above devarg parameters are configurable per device, user needs to pass the
281 parameters to all the PCIe devices if application requires to configure on
282 all the ethdev ports.
287 ``mempool_cnxk`` external mempool handler dependency
288 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
290 The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager.
291 ``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler
292 as it is performance wise most effective way for packet allocation and Tx buffer
293 recycling on OCTEON TX2 SoC platform.
298 The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by
299 the host interface irrespective of the offload configuration.
304 - ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing
305 bits in the GRE header are equal to 0.
307 Custom protocols supported in RTE Flow
308 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
310 The ``RTE_FLOW_ITEM_TYPE_RAW`` can be used to parse the below custom protocols.
312 * ``vlan_exdsa`` and ``exdsa`` can be parsed at L2 level.
313 * ``NGIO`` can be parsed at L3 level.
315 For ``vlan_exdsa`` and ``exdsa``, the port has to be configured with the
316 respective switch header.
320 -a 0002:02:00.0,switch_header="vlan_exdsa"
322 The below fields of ``struct rte_flow_item_raw`` shall be used to specify the
325 - ``relative`` Selects the layer at which parsing is done.
327 - 0 for ``exdsa`` and ``vlan_exdsa``.
331 - ``offset`` The offset in the header where the pattern should be matched.
332 - ``length`` Length of the pattern.
333 - ``pattern`` Pattern as a byte string.
335 Example usage in testpmd::
337 ./dpdk-testpmd -c 3 -w 0002:02:00.0,switch_header=exdsa -- -i \
338 --rx-offloads=0x00080000 --rxq 8 --txq 8
339 testpmd> flow create 0 ingress pattern eth / raw relative is 0 pattern \
340 spec ab pattern mask ab offset is 4 / end actions queue index 1 / end
342 Inline device support for CN10K
343 -------------------------------
345 CN10K HW provides a misc device Inline device that supports ethernet devices in
346 providing following features.
348 - Aggregate all the inline IPsec inbound traffic from all the CN10K ethernet
349 devices to be processed by the single inline IPSec device. This allows
350 single rte security session to accept traffic from multiple ports.
352 - Support for event generation on outbound inline IPsec processing errors.
354 - Support CN106xx poll mode of operation for inline IPSec inbound processing.
356 Inline IPsec device is identified by PCI PF vendid:devid ``177D:A0F0`` or
359 Runtime Config Options for inline device
360 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
362 - ``Max SPI for inbound inline IPsec`` (default ``255``)
364 Max SPI supported for inbound inline IPsec processing can be specified by
365 ``ipsec_in_max_spi`` ``devargs`` parameter.
369 -a 0002:1d:00.0,ipsec_in_max_spi=128
371 With the above configuration, application can enable inline IPsec processing
372 for 128 inbound SAs (SPI 0-127) for traffic aggregated on inline device.
378 .. _table_cnxk_ethdev_debug_options:
380 .. table:: cnxk ethdev debug options
382 +---+------------+-------------------------------------------------------+
383 | # | Component | EAL log command |
384 +===+============+=======================================================+
385 | 1 | NIX | --log-level='pmd\.net.cnxk,8' |
386 +---+------------+-------------------------------------------------------+
387 | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' |
388 +---+------------+-------------------------------------------------------+