1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(C) 2021 Marvell.
7 The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver
8 support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K**
9 SoC family as well as for their virtual functions (VF) in SR-IOV context.
11 More information can be found at `Marvell Official Website
12 <https://www.marvell.com/embedded-processors/infrastructure-processors>`_.
17 Features of the CNXK Ethdev PMD are:
19 - Packet type information
24 - Multiple queues for TX and RX
25 - Receiver Side Scaling (RSS)
28 - Inner and Outer Checksum offload
29 - Port hardware statistics
30 - Link state information
33 - Scatter-Gather IO support
34 - Vector Poll mode driver
35 - Debug utilities - Context dump and error interrupt support
36 - Support Rx interrupt
37 - Inline IPsec processing support
38 - Ingress meter support
43 See :doc:`../platform/cnxk` for setup information.
46 Driver compilation and testing
47 ------------------------------
49 Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
54 Follow instructions available in the document
55 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
60 .. code-block:: console
62 ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1
63 EAL: Detected 4 lcore(s)
64 EAL: Detected 1 NUMA nodes
65 EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
66 EAL: Selected IOVA mode 'VA'
67 EAL: No available hugepages reported in hugepages-16777216kB
68 EAL: No available hugepages reported in hugepages-2048kB
69 EAL: Probing VFIO support...
70 EAL: VFIO support initialized
71 EAL: using IOMMU type 1 (Type 1)
72 [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98
73 EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0)
75 EAL: No legacy callbacks, legacy socket not created
76 testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
77 testpmd: preferred mempool ops selected: cn10k_mempool_ops
78 Configuring Port 0 (socket 0)
79 PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex
81 Port 0: link state change event
82 Port 0: 96:D4:99:72:A5:BF
83 Checking link statuses...
85 No commandline core given, start packet forwarding
86 io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native
87 Logical Core 3 (socket 0) forwards packets on 1 streams:
88 RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
90 io packet forwarding packets/burst=32
91 nb forwarding cores=1 - nb forwarding ports=1
92 port 0: RX queue number: 1 Tx queue number: 1
93 Rx offloads=0x0 Tx offloads=0x10000
95 RX desc=4096 - RX free threshold=0
96 RX threshold registers: pthresh=0 hthresh=0 wthresh=0
99 TX desc=512 - TX free threshold=0
100 TX threshold registers: pthresh=0 hthresh=0 wthresh=0
101 TX offloads=0x0 - TX RS bit threshold=0
104 Runtime Config Options
105 ----------------------
107 - ``Rx&Tx scalar mode enable`` (default ``0``)
109 PMD supports both scalar and vector mode, it may be selected at runtime
110 using ``scalar_enable`` ``devargs`` parameter.
112 - ``RSS reta size`` (default ``64``)
114 RSS redirection table size may be configured during runtime using ``reta_size``
115 ``devargs`` parameter.
119 -a 0002:02:00.0,reta_size=256
121 With the above configuration, reta table of size 256 is populated.
123 - ``Flow priority levels`` (default ``3``)
125 RTE Flow priority levels can be configured during runtime using
126 ``flow_max_priority`` ``devargs`` parameter.
130 -a 0002:02:00.0,flow_max_priority=10
132 With the above configuration, priority level was set to 10 (0-9). Max
133 priority level supported is 32.
135 - ``Reserve Flow entries`` (default ``8``)
137 RTE flow entries can be pre allocated and the size of pre allocation can be
138 selected runtime using ``flow_prealloc_size`` ``devargs`` parameter.
142 -a 0002:02:00.0,flow_prealloc_size=4
144 With the above configuration, pre alloc size was set to 4. Max pre alloc
145 size supported is 32.
147 - ``Max SQB buffer count`` (default ``512``)
149 Send queue descriptor buffer count may be limited during runtime using
150 ``max_sqb_count`` ``devargs`` parameter.
154 -a 0002:02:00.0,max_sqb_count=64
156 With the above configuration, each send queue's descriptor buffer count is
157 limited to a maximum of 64 buffers.
159 - ``Switch header enable`` (default ``none``)
161 A port can be configured to a specific switch header type by using
162 ``switch_header`` ``devargs`` parameter.
166 -a 0002:02:00.0,switch_header="higig2"
168 With the above configuration, higig2 will be enabled on that port and the
169 traffic on this port should be higig2 traffic only. Supported switch header
170 types are "chlen24b", "chlen90b", "dsa", "exdsa", "higig2" and "vlan_exdsa".
172 - ``RSS tag as XOR`` (default ``0``)
174 The HW gives two options to configure the RSS adder i.e
176 * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>``
178 * ``rss_adder<7:0> = flow_tag<7:0>``
180 Latter one aligns with standard NIC behavior vs former one is a legacy
181 RSS adder scheme used in OCTEON TX2 products.
183 By default, the driver runs in the latter mode.
184 Setting this flag to 1 to select the legacy mode.
186 For example to select the legacy mode(RSS tag adder as XOR)::
188 -a 0002:02:00.0,tag_as_xor=1
190 - ``Max SPI for inbound inline IPsec`` (default ``255``)
192 Max SPI supported for inbound inline IPsec processing can be specified by
193 ``ipsec_in_max_spi`` ``devargs`` parameter.
197 -a 0002:02:00.0,ipsec_in_max_spi=128
199 With the above configuration, application can enable inline IPsec processing
200 for 128 inbound SAs (SPI 0-127).
202 - ``Max SA's for outbound inline IPsec`` (default ``4096``)
204 Max number of SA's supported for outbound inline IPsec processing can be
205 specified by ``ipsec_out_max_sa`` ``devargs`` parameter.
209 -a 0002:02:00.0,ipsec_out_max_sa=128
211 With the above configuration, application can enable inline IPsec processing
212 for 128 outbound SAs.
214 - ``Outbound CPT LF queue size`` (default ``8200``)
216 Size of Outbound CPT LF queue in number of descriptors can be specified by
217 ``outb_nb_desc`` ``devargs`` parameter.
221 -a 0002:02:00.0,outb_nb_desc=16384
223 With the above configuration, Outbound CPT LF will be created to accommodate
224 at max 16384 descriptors at any given time.
226 - ``Outbound CPT LF count`` (default ``1``)
228 Number of CPT LF's to attach for Outbound processing can be specified by
229 ``outb_nb_crypto_qs`` ``devargs`` parameter.
233 -a 0002:02:00.0,outb_nb_crypto_qs=2
235 With the above configuration, two CPT LF's are setup and distributed among
236 all the Tx queues for outbound processing.
238 - ``Force using inline ipsec device for inbound`` (default ``0``)
240 In CN10K, in event mode, driver can work in two modes,
242 1. Inbound encrypted traffic received by probed ipsec inline device while
243 plain traffic post decryption is received by ethdev.
245 2. Both Inbound encrypted traffic and plain traffic post decryption are
248 By default event mode works without using inline device i.e mode ``2``.
249 This behaviour can be changed to pick mode ``1`` by using
250 ``force_inb_inl_dev`` ``devargs`` parameter.
254 -a 0002:02:00.0,force_inb_inl_dev=1 -a 0002:03:00.0,force_inb_inl_dev=1
256 With the above configuration, inbound encrypted traffic from both the ports
257 is received by ipsec inline device.
259 - ``Inline IPsec device channel and mask`` (default ``none``)
261 Set channel and channel mask configuration for the inline IPSec device. This
262 will be used when creating flow rules with RTE_FLOW_ACTION_TYPE_SECURITY
265 By default, RTE Flow API sets the channel number of the port on which the
266 rule is created in the MCAM entry and matches it exactly. This behaviour can
267 be modified using the ``inl_cpt_channel`` ``devargs`` parameter.
271 -a 0002:1d:00.0,inl_cpt_channel=0x100/0xf00
273 With the above configuration, RTE Flow rules API will set the channel
274 and channel mask as 0x100 and 0xF00 in the MCAM entries of the flow rules
275 created with RTE_FLOW_ACTION_TYPE_SECURITY action. Since channel number is
276 set with this custom mask, inbound encrypted traffic from all ports with
277 matching channel number pattern will be directed to the inline IPSec device.
281 Above devarg parameters are configurable per device, user needs to pass the
282 parameters to all the PCIe devices if application requires to configure on
283 all the ethdev ports.
288 ``mempool_cnxk`` external mempool handler dependency
289 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
291 The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager.
292 ``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler
293 as it is performance wise most effective way for packet allocation and Tx buffer
294 recycling on OCTEON TX2 SoC platform.
299 The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by
300 the host interface irrespective of the offload configuration.
305 - ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing
306 bits in the GRE header are equal to 0.
308 RTE flow action port_id support
309 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
311 - ``RTE_FLOW_ACTION_TYPE_PORT_ID`` is only supported between PF and its VFs.
313 Custom protocols supported in RTE Flow
314 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
316 The ``RTE_FLOW_ITEM_TYPE_RAW`` can be used to parse the below custom protocols.
318 * ``vlan_exdsa`` and ``exdsa`` can be parsed at L2 level.
319 * ``NGIO`` can be parsed at L3 level.
321 For ``vlan_exdsa`` and ``exdsa``, the port has to be configured with the
322 respective switch header.
326 -a 0002:02:00.0,switch_header="vlan_exdsa"
328 The below fields of ``struct rte_flow_item_raw`` shall be used to specify the
331 - ``relative`` Selects the layer at which parsing is done.
333 - 0 for ``exdsa`` and ``vlan_exdsa``.
337 - ``offset`` The offset in the header where the pattern should be matched.
338 - ``length`` Length of the pattern.
339 - ``pattern`` Pattern as a byte string.
341 Example usage in testpmd::
343 ./dpdk-testpmd -c 3 -w 0002:02:00.0,switch_header=exdsa -- -i \
344 --rx-offloads=0x00080000 --rxq 8 --txq 8
345 testpmd> flow create 0 ingress pattern eth / raw relative is 0 pattern \
346 spec ab pattern mask ab offset is 4 / end actions queue index 1 / end
348 Inline device support for CN10K
349 -------------------------------
351 CN10K HW provides a misc device Inline device that supports ethernet devices in
352 providing following features.
354 - Aggregate all the inline IPsec inbound traffic from all the CN10K ethernet
355 devices to be processed by the single inline IPSec device. This allows
356 single rte security session to accept traffic from multiple ports.
358 - Support for event generation on outbound inline IPsec processing errors.
360 - Support CN106xx poll mode of operation for inline IPSec inbound processing.
362 Inline IPsec device is identified by PCI PF vendid:devid ``177D:A0F0`` or
365 Runtime Config Options for inline device
366 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
368 - ``Max SPI for inbound inline IPsec`` (default ``255``)
370 Max SPI supported for inbound inline IPsec processing can be specified by
371 ``ipsec_in_max_spi`` ``devargs`` parameter.
375 -a 0002:1d:00.0,ipsec_in_max_spi=128
377 With the above configuration, application can enable inline IPsec processing
378 for 128 inbound SAs (SPI 0-127) for traffic aggregated on inline device.
384 .. _table_cnxk_ethdev_debug_options:
386 .. table:: cnxk ethdev debug options
388 +---+------------+-------------------------------------------------------+
389 | # | Component | EAL log command |
390 +===+============+=======================================================+
391 | 1 | NIX | --log-level='pmd\.net.cnxk,8' |
392 +---+------------+-------------------------------------------------------+
393 | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' |
394 +---+------------+-------------------------------------------------------+