1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(C) 2021 Marvell.
7 The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver
8 support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K**
9 SoC family as well as for their virtual functions (VF) in SR-IOV context.
11 More information can be found at `Marvell Official Website
12 <https://www.marvell.com/embedded-processors/infrastructure-processors>`_.
17 Features of the CNXK Ethdev PMD are:
19 - Packet type information
24 - Multiple queues for TX and RX
25 - Receiver Side Scaling (RSS)
28 - Inner and Outer Checksum offload
29 - Port hardware statistics
30 - Link state information
33 - Scatter-Gather IO support
34 - Vector Poll mode driver
35 - Debug utilities - Context dump and error interrupt support
36 - Support Rx interrupt
37 - Inline IPsec processing support
38 - Ingress meter support
43 See :doc:`../platform/cnxk` for setup information.
46 Driver compilation and testing
47 ------------------------------
49 Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
54 Follow instructions available in the document
55 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
60 .. code-block:: console
62 ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1
63 EAL: Detected 4 lcore(s)
64 EAL: Detected 1 NUMA nodes
65 EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
66 EAL: Selected IOVA mode 'VA'
67 EAL: No available hugepages reported in hugepages-16777216kB
68 EAL: No available hugepages reported in hugepages-2048kB
69 EAL: Probing VFIO support...
70 EAL: VFIO support initialized
71 EAL: using IOMMU type 1 (Type 1)
72 [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98
73 EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0)
75 EAL: No legacy callbacks, legacy socket not created
76 testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
77 testpmd: preferred mempool ops selected: cn10k_mempool_ops
78 Configuring Port 0 (socket 0)
79 PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex
81 Port 0: link state change event
82 Port 0: 96:D4:99:72:A5:BF
83 Checking link statuses...
85 No commandline core given, start packet forwarding
86 io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native
87 Logical Core 3 (socket 0) forwards packets on 1 streams:
88 RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
90 io packet forwarding packets/burst=32
91 nb forwarding cores=1 - nb forwarding ports=1
92 port 0: RX queue number: 1 Tx queue number: 1
93 Rx offloads=0x0 Tx offloads=0x10000
95 RX desc=4096 - RX free threshold=0
96 RX threshold registers: pthresh=0 hthresh=0 wthresh=0
99 TX desc=512 - TX free threshold=0
100 TX threshold registers: pthresh=0 hthresh=0 wthresh=0
101 TX offloads=0x0 - TX RS bit threshold=0
104 Runtime Config Options
105 ----------------------
107 - ``Rx&Tx scalar mode enable`` (default ``0``)
109 PMD supports both scalar and vector mode, it may be selected at runtime
110 using ``scalar_enable`` ``devargs`` parameter.
112 - ``RSS reta size`` (default ``64``)
114 RSS redirection table size may be configured during runtime using ``reta_size``
115 ``devargs`` parameter.
119 -a 0002:02:00.0,reta_size=256
121 With the above configuration, reta table of size 256 is populated.
123 - ``Flow priority levels`` (default ``3``)
125 RTE Flow priority levels can be configured during runtime using
126 ``flow_max_priority`` ``devargs`` parameter.
130 -a 0002:02:00.0,flow_max_priority=10
132 With the above configuration, priority level was set to 10 (0-9). Max
133 priority level supported is 32.
135 - ``Reserve Flow entries`` (default ``8``)
137 RTE flow entries can be pre allocated and the size of pre allocation can be
138 selected runtime using ``flow_prealloc_size`` ``devargs`` parameter.
142 -a 0002:02:00.0,flow_prealloc_size=4
144 With the above configuration, pre alloc size was set to 4. Max pre alloc
145 size supported is 32.
147 - ``Max SQB buffer count`` (default ``512``)
149 Send queue descriptor buffer count may be limited during runtime using
150 ``max_sqb_count`` ``devargs`` parameter.
154 -a 0002:02:00.0,max_sqb_count=64
156 With the above configuration, each send queue's descriptor buffer count is
157 limited to a maximum of 64 buffers.
159 - ``Switch header enable`` (default ``none``)
161 A port can be configured to a specific switch header type by using
162 ``switch_header`` ``devargs`` parameter.
166 -a 0002:02:00.0,switch_header="higig2"
168 With the above configuration, higig2 will be enabled on that port and the
169 traffic on this port should be higig2 traffic only. Supported switch header
170 types are "chlen24b", "chlen90b", "dsa", "exdsa", "higig2", "vlan_exdsa" and
173 - ``Flow pre_l2 info`` (default ``0x0/0x0/0x0``)
175 pre_l2 headers are custom headers placed before the ethernet header. For
176 parsing custom pre_l2 headers, an offset, mask within the offset and shift
177 direction has to be provided within the custom header that holds the size of
178 the custom header. This is valid only with switch header pre_l2. Maximum
179 supported offset range is 0 to 255 and mask range is 1 to 255 and
180 shift direction, 0: left shift, 1: right shift.
181 Info format will be "offset/mask/shift direction". All parameters has to be
182 in hexadecimal format and mask should be contiguous. Info can be configured
183 using ``flow_pre_l2_info`` ``devargs`` parameter.
187 -a 0002:02:00.0,switch_header="pre_l2",flow_pre_l2_info=0x2/0x7e/0x1
189 With the above configuration, custom pre_l2 header will be enabled on that
190 port and size of the header is placed at byte offset 0x2 in the packet with
191 mask 0x7e and right shift will be used to get the size. That is, size will be
192 (pkt[0x2] & 0x7e) >> shift count. Shift count will be calculated based on
193 mask and shift direction. For example, if mask is 0x7c and shift direction is
194 1 (i.e., right shift) then the shift count will be 2, that is, absolute
195 position of the rightmost set bit. If the mask is 0x7c and shift direction
196 is 0 (i.e., left shift) then the shift count will be 1, that is, (8 - n),
197 where n is the absolute position of leftmost set bit.
199 - ``RSS tag as XOR`` (default ``0``)
201 The HW gives two options to configure the RSS adder i.e
203 * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>``
205 * ``rss_adder<7:0> = flow_tag<7:0>``
207 Latter one aligns with standard NIC behavior vs former one is a legacy
208 RSS adder scheme used in OCTEON 9 products.
210 By default, the driver runs in the latter mode.
211 Setting this flag to 1 to select the legacy mode.
213 For example to select the legacy mode(RSS tag adder as XOR)::
215 -a 0002:02:00.0,tag_as_xor=1
217 - ``Max SPI for inbound inline IPsec`` (default ``255``)
219 Max SPI supported for inbound inline IPsec processing can be specified by
220 ``ipsec_in_max_spi`` ``devargs`` parameter.
224 -a 0002:02:00.0,ipsec_in_max_spi=128
226 With the above configuration, application can enable inline IPsec processing
227 for 128 inbound SAs (SPI 0-127).
229 - ``Max SA's for outbound inline IPsec`` (default ``4096``)
231 Max number of SA's supported for outbound inline IPsec processing can be
232 specified by ``ipsec_out_max_sa`` ``devargs`` parameter.
236 -a 0002:02:00.0,ipsec_out_max_sa=128
238 With the above configuration, application can enable inline IPsec processing
239 for 128 outbound SAs.
241 - ``Outbound CPT LF queue size`` (default ``8200``)
243 Size of Outbound CPT LF queue in number of descriptors can be specified by
244 ``outb_nb_desc`` ``devargs`` parameter.
248 -a 0002:02:00.0,outb_nb_desc=16384
250 With the above configuration, Outbound CPT LF will be created to accommodate
251 at max 16384 descriptors at any given time.
253 - ``Outbound CPT LF count`` (default ``1``)
255 Number of CPT LF's to attach for Outbound processing can be specified by
256 ``outb_nb_crypto_qs`` ``devargs`` parameter.
260 -a 0002:02:00.0,outb_nb_crypto_qs=2
262 With the above configuration, two CPT LF's are setup and distributed among
263 all the Tx queues for outbound processing.
265 - ``Force using inline ipsec device for inbound`` (default ``0``)
267 In CN10K, in event mode, driver can work in two modes,
269 1. Inbound encrypted traffic received by probed ipsec inline device while
270 plain traffic post decryption is received by ethdev.
272 2. Both Inbound encrypted traffic and plain traffic post decryption are
275 By default event mode works without using inline device i.e mode ``2``.
276 This behaviour can be changed to pick mode ``1`` by using
277 ``force_inb_inl_dev`` ``devargs`` parameter.
281 -a 0002:02:00.0,force_inb_inl_dev=1 -a 0002:03:00.0,force_inb_inl_dev=1
283 With the above configuration, inbound encrypted traffic from both the ports
284 is received by ipsec inline device.
286 - ``Inline IPsec device channel and mask`` (default ``none``)
288 Set channel and channel mask configuration for the inline IPSec device. This
289 will be used when creating flow rules with RTE_FLOW_ACTION_TYPE_SECURITY
292 By default, RTE Flow API sets the channel number of the port on which the
293 rule is created in the MCAM entry and matches it exactly. This behaviour can
294 be modified using the ``inl_cpt_channel`` ``devargs`` parameter.
298 -a 0002:1d:00.0,inl_cpt_channel=0x100/0xf00
300 With the above configuration, RTE Flow rules API will set the channel
301 and channel mask as 0x100 and 0xF00 in the MCAM entries of the flow rules
302 created with RTE_FLOW_ACTION_TYPE_SECURITY action. Since channel number is
303 set with this custom mask, inbound encrypted traffic from all ports with
304 matching channel number pattern will be directed to the inline IPSec device.
306 - ``SDP device channel and mask`` (default ``none``)
307 Set channel and channel mask configuration for the SDP device. This
308 will be used when creating flow rules on the SDP device.
310 By default, for rules created on the SDP device, the RTE Flow API sets the
311 channel number and mask to cover the entire SDP channel range in the channel
312 field of the MCAM entry. This behaviour can be modified using the
313 ``sdp_channel_mask`` ``devargs`` parameter.
317 -a 0002:1d:00.0,sdp_channel_mask=0x700/0xf00
319 With the above configuration, RTE Flow rules API will set the channel
320 and channel mask as 0x700 and 0xF00 in the MCAM entries of the flow rules
321 created on the SDP device. This option needs to be used when more than one
322 SDP interface is in use and RTE Flow rules created need to distinguish
323 between traffic from each SDP interface. The channel and mask combination
324 specified should match all the channels(or rings) configured on the SDP
329 Above devarg parameters are configurable per device, user needs to pass the
330 parameters to all the PCIe devices if application requires to configure on
331 all the ethdev ports.
336 ``mempool_cnxk`` external mempool handler dependency
337 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
339 The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager.
340 ``net_cnxk`` PMD only works with ``mempool_cnxk`` mempool handler
341 as it is performance wise most effective way for packet allocation and Tx buffer
342 recycling on OCTEON 9 SoC platform.
347 The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by
348 the host interface irrespective of the offload configuration.
353 - ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing
354 bits in the GRE header are equal to 0.
356 RTE flow action port_id support
357 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
359 - ``RTE_FLOW_ACTION_TYPE_PORT_ID`` is only supported between PF and its VFs.
361 Custom protocols supported in RTE Flow
362 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
364 The ``RTE_FLOW_ITEM_TYPE_RAW`` can be used to parse the below custom protocols.
366 * ``vlan_exdsa`` and ``exdsa`` can be parsed at L2 level.
367 * ``NGIO`` can be parsed at L3 level.
369 For ``vlan_exdsa`` and ``exdsa``, the port has to be configured with the
370 respective switch header.
374 -a 0002:02:00.0,switch_header="vlan_exdsa"
376 The below fields of ``struct rte_flow_item_raw`` shall be used to specify the
379 - ``relative`` Selects the layer at which parsing is done.
381 - 0 for ``exdsa`` and ``vlan_exdsa``.
385 - ``offset`` The offset in the header where the pattern should be matched.
386 - ``length`` Length of the pattern.
387 - ``pattern`` Pattern as a byte string.
389 Example usage in testpmd::
391 ./dpdk-testpmd -c 3 -w 0002:02:00.0,switch_header=exdsa -- -i \
392 --rx-offloads=0x00080000 --rxq 8 --txq 8
393 testpmd> flow create 0 ingress pattern eth / raw relative is 0 pattern \
394 spec ab pattern mask ab offset is 4 / end actions queue index 1 / end
396 Inline device support for CN10K
397 -------------------------------
399 CN10K HW provides a misc device Inline device that supports ethernet devices in
400 providing following features.
402 - Aggregate all the inline IPsec inbound traffic from all the CN10K ethernet
403 devices to be processed by the single inline IPSec device. This allows
404 single rte security session to accept traffic from multiple ports.
406 - Support for event generation on outbound inline IPsec processing errors.
408 - Support CN106xx poll mode of operation for inline IPSec inbound processing.
410 Inline IPsec device is identified by PCI PF vendid:devid ``177D:A0F0`` or
413 Runtime Config Options for inline device
414 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
416 - ``Max SPI for inbound inline IPsec`` (default ``255``)
418 Max SPI supported for inbound inline IPsec processing can be specified by
419 ``ipsec_in_max_spi`` ``devargs`` parameter.
423 -a 0002:1d:00.0,ipsec_in_max_spi=128
425 With the above configuration, application can enable inline IPsec processing
426 for 128 inbound SAs (SPI 0-127) for traffic aggregated on inline device.
432 .. _table_cnxk_ethdev_debug_options:
434 .. table:: cnxk ethdev debug options
436 +---+------------+-------------------------------------------------------+
437 | # | Component | EAL log command |
438 +===+============+=======================================================+
439 | 1 | NIX | --log-level='pmd\.net.cnxk,8' |
440 +---+------------+-------------------------------------------------------+
441 | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' |
442 +---+------------+-------------------------------------------------------+