1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Marvell International Ltd.
4 Marvell OCTEON TX2 Platform Guide
5 =================================
7 This document gives an overview of **Marvell OCTEON TX2** RVU H/W block,
8 packet flow and procedure to build DPDK on OCTEON TX2 platform.
10 More information about OCTEON TX2 SoC can be found at `Marvell Official Website
11 <https://www.marvell.com/embedded-processors/infrastructure-processors/>`_.
13 Supported OCTEON TX2 SoCs
14 -------------------------
20 OCTEON TX2 Resource Virtualization Unit architecture
21 ----------------------------------------------------
23 The :numref:`figure_octeontx2_resource_virtualization` diagram depicts the
24 RVU architecture and a resource provisioning example.
26 .. _figure_octeontx2_resource_virtualization:
28 .. figure:: img/octeontx2_resource_virtualization.*
30 OCTEON TX2 Resource virtualization architecture and provisioning example
33 Resource Virtualization Unit (RVU) on Marvell's OCTEON TX2 SoC maps HW
34 resources belonging to the network, crypto and other functional blocks onto
35 PCI-compatible physical and virtual functions.
37 Each functional block has multiple local functions (LFs) for
38 provisioning to different PCIe devices. RVU supports multiple PCIe SRIOV
39 physical functions (PFs) and virtual functions (VFs).
41 The :numref:`table_octeontx2_rvu_dpdk_mapping` shows the various local
42 functions (LFs) provided by the RVU and its functional mapping to
45 .. _table_octeontx2_rvu_dpdk_mapping:
47 .. table:: RVU managed functional blocks and its mapping to DPDK subsystem
49 +---+-----+--------------------------------------------------------------+
50 | # | LF | DPDK subsystem mapping |
51 +===+=====+==============================================================+
52 | 1 | NIX | rte_ethdev, rte_tm, rte_event_eth_[rt]x_adapter, rte_security|
53 +---+-----+--------------------------------------------------------------+
54 | 2 | NPA | rte_mempool |
55 +---+-----+--------------------------------------------------------------+
56 | 3 | NPC | rte_flow |
57 +---+-----+--------------------------------------------------------------+
58 | 4 | CPT | rte_cryptodev, rte_event_crypto_adapter |
59 +---+-----+--------------------------------------------------------------+
60 | 5 | SSO | rte_eventdev |
61 +---+-----+--------------------------------------------------------------+
62 | 6 | TIM | rte_event_timer_adapter |
63 +---+-----+--------------------------------------------------------------+
65 PF0 is called the administrative / admin function (AF) and has exclusive
66 privileges to provision RVU functional block's LFs to each of the PF/VF.
68 PF/VFs communicates with AF via a shared memory region (mailbox).Upon receiving
69 requests from PF/VF, AF does resource provisioning and other HW configuration.
71 AF is always attached to host, but PF/VFs may be used by host kernel itself,
72 or attached to VMs or to userspace applications like DPDK, etc. So, AF has to
73 handle provisioning/configuration requests sent by any device from any domain.
75 The AF driver does not receive or process any data.
76 It is only a configuration driver used in control path.
78 The :numref:`figure_octeontx2_resource_virtualization` diagram also shows a
79 resource provisioning example where,
81 1. PFx and PFx-VF0 bound to Linux netdev driver.
82 2. PFx-VF1 ethdev driver bound to the first DPDK application.
83 3. PFy ethdev driver, PFy-VF0 ethdev driver, PFz eventdev driver, PFm-VF0 cryptodev driver bound to the second DPDK application.
85 OCTEON TX2 packet flow
86 ----------------------
88 The :numref:`figure_octeontx2_packet_flow_hw_accelerators` diagram depicts
89 the packet flow on OCTEON TX2 SoC in conjunction with use of various HW accelerators.
91 .. _figure_octeontx2_packet_flow_hw_accelerators:
93 .. figure:: img/octeontx2_packet_flow_hw_accelerators.*
95 OCTEON TX2 packet flow in conjunction with use of HW accelerators
100 This section lists dataplane H/W block(s) available in OCTEON TX2 SoC.
103 See :doc:`../nics/octeontx2` for NIX Ethdev driver information.
105 #. **Mempool Driver**
106 See :doc:`../mempool/octeontx2` for NPA mempool driver information.
108 #. **Event Device Driver**
109 See :doc:`../eventdevs/octeontx2` for SSO event device driver information.
111 #. **DMA Rawdev Driver**
112 See :doc:`../rawdevs/octeontx2_dma` for DMA driver information.
114 Procedure to Setup Platform
115 ---------------------------
117 There are three main prerequisites for setting up DPDK on OCTEON TX2
120 1. **OCTEON TX2 Linux kernel driver**
122 The dependent kernel drivers can be obtained from the
123 `kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/marvell/octeontx2>`_.
125 Alternatively, the Marvell SDK also provides the required kernel drivers.
127 Linux kernel should be configured with the following features enabled:
129 .. code-block:: console
131 # 64K pages enabled for better performance
132 CONFIG_ARM64_64K_PAGES=y
133 CONFIG_ARM64_VA_BITS_48=y
134 # huge pages support enabled
136 CONFIG_HUGETLB_PAGE=y
137 # VFIO enabled with TYPE1 IOMMU at minimum
138 CONFIG_VFIO_IOMMU_TYPE1=y
141 CONFIG_VFIO_NOIOMMU=y
143 CONFIG_VFIO_PCI_MMAP=y
146 # ARMv8.1 LSE atomics
147 CONFIG_ARM64_LSE_ATOMICS=y
149 CONFIG_OCTEONTX2_MBOX=y
150 CONFIG_OCTEONTX2_AF=y
151 # Enable if netdev PF driver required
152 CONFIG_OCTEONTX2_PF=y
153 # Enable if netdev VF driver required
154 CONFIG_OCTEONTX2_VF=y
155 CONFIG_CRYPTO_DEV_OCTEONTX2_CPT=y
156 # Enable if OCTEONTX2 DMA PF driver required
157 CONFIG_OCTEONTX2_DPI_PF=n
159 2. **ARM64 Linux Tool Chain**
161 For example, the *aarch64* Linaro Toolchain, which can be obtained from
162 `here <https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/>`_.
164 Alternatively, the Marvell SDK also provides GNU GCC toolchain, which is
165 optimized for OCTEON TX2 CPU.
167 3. **Rootfile system**
169 Any *aarch64* supporting filesystem may be used. For example,
170 Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained
171 from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_.
173 Alternatively, the Marvell SDK provides the buildroot based root filesystem.
174 The SDK includes all the above prerequisites necessary to bring up the OCTEON TX2 board.
176 - Follow the DPDK :doc:`../linux_gsg/index` to setup the basic DPDK environment.
182 .. _table_octeontx2_common_debug_options:
184 .. table:: OCTEON TX2 common debug options
186 +---+------------+-------------------------------------------------------+
187 | # | Component | EAL log command |
188 +===+============+=======================================================+
189 | 1 | Common | --log-level='pmd\.octeontx2\.base,8' |
190 +---+------------+-------------------------------------------------------+
191 | 2 | Mailbox | --log-level='pmd\.octeontx2\.mbox,8' |
192 +---+------------+-------------------------------------------------------+
197 The **OCTEON TX2 Linux kernel driver** provides support to dump RVU blocks
198 context or stats using debugfs.
200 Enable ``debugfs`` by:
202 1. Compile kernel with debugfs enabled, i.e ``CONFIG_DEBUGFS=y``.
203 2. Boot OCTEON TX2 with debugfs supported kernel.
204 3. Verify ``debugfs`` mounted by default "mount | grep -i debugfs" or mount it manually by using.
206 .. code-block:: console
208 # mount -t debugfs none /sys/kernel/debug
210 Currently ``debugfs`` supports the following RVU blocks NIX, NPA, NPC, NDC,
213 The file structure under ``/sys/kernel/debug`` is as follows
215 .. code-block:: console
231 | |-- cpt_engines_info
232 | |-- cpt_engines_sts
239 | |-- ndc_rx_hits_miss
241 | |-- ndc_tx_hits_miss
245 | '-- tx_stall_hwissue
254 | '-- rx_miss_act_stats
260 |-- sso_hwgrp_aq_thresh
261 |-- sso_hwgrp_iaq_walk
263 |-- sso_hwgrp_free_list_walk
264 |-- sso_hwgrp_ient_walk
265 '-- sso_hwgrp_taq_walk
267 RVU block LF allocation:
269 .. code-block:: console
271 cat /sys/kernel/debug/octeontx2/rsrc_alloc
273 pcifunc NPA NIX SSO GROUP SSOWS TIM CPT
280 .. code-block:: console
282 cat /sys/kernel/debug/octeontx2/cgx/cgx2/lmac0/stats
284 =======Link Status======
285 Link is UP 40000 Mbps
286 =======RX_STATS======
288 Octets of received packets: 0
289 Received PAUSE packets: 0
290 Received PAUSE and control packets: 0
291 Filtered DMAC0 (NIX-bound) packets: 0
292 Filtered DMAC0 (NIX-bound) octets: 0
293 Packets dropped due to RX FIFO full: 0
294 Octets dropped due to RX FIFO full: 0
296 Filtered DMAC1 (NCSI-bound) packets: 0
297 Filtered DMAC1 (NCSI-bound) octets: 0
298 NCSI-bound packets dropped: 0
299 NCSI-bound octets dropped: 0
300 =======TX_STATS======
301 Packets dropped due to excessive collisions: 0
302 Packets dropped due to excessive deferral: 0
303 Multiple collisions before successful transmission: 0
304 Single collisions before successful transmission: 0
305 Total octets sent on the interface: 0
306 Total frames sent on the interface: 0
307 Packets sent with an octet count < 64: 0
308 Packets sent with an octet count == 64: 0
309 Packets sent with an octet count of 65127: 0
310 Packets sent with an octet count of 128-255: 0
311 Packets sent with an octet count of 256-511: 0
312 Packets sent with an octet count of 512-1023: 0
313 Packets sent with an octet count of 1024-1518: 0
314 Packets sent with an octet count of > 1518: 0
315 Packets sent to a broadcast DMAC: 0
316 Packets sent to the multicast DMAC: 0
317 Transmit underflow and were truncated: 0
318 Control/PAUSE packets sent: 0
322 .. code-block:: console
324 cat /sys/kernel/debug/octeontx2/cpt/cpt_pc
326 CPT instruction requests 0
327 CPT instruction latency 0
328 CPT NCB read requests 0
329 CPT NCB read latency 0
330 CPT read requests caused by UC fills 0
331 CPT active cycles pc 1395642
332 CPT clock count pc 5579867595493
336 .. code-block:: console
338 Usage: echo <nixlf> [cq number/all] > /sys/kernel/debug/octeontx2/nix/cq_ctx
339 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
340 echo 0 0 > /sys/kernel/debug/octeontx2/nix/cq_ctx
341 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
343 =====cq_ctx for nixlf:0 and qidx:0 is=====
354 W2: update_time 31043
371 .. code-block:: console
373 Usage: echo <npalf> [pool number/all] > /sys/kernel/debug/octeontx2/npa/pool_ctx
374 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
375 echo 0 0 > /sys/kernel/debug/octeontx2/npa/pool_ctx
376 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
378 ======POOL : 0=======
379 W0: Stack base 1375bff00
386 W2: stack_max_pages 24315
387 W2: stack_pages 24314
397 W4: update_time 62993
399 W6: ptr_start 1593adf00
400 W7: ptr_end 180000000
406 W8: thresh_qint_idx 0
411 .. code-block:: console
413 cat /sys/kernel/debug/octeontx2/npc/mcam_info
416 RX keywidth : 224bits
417 TX keywidth : 224bits
429 .. code-block:: console
431 Usage: echo [<hws>/all] > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
432 echo 0 > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
434 ==================================================
435 SSOW HWS[0] Arbitration State 0x0
436 SSOW HWS[0] Guest Machine Control 0x0
437 SSOW HWS[0] SET[0] Group Mask[0] 0xffffffffffffffff
438 SSOW HWS[0] SET[0] Group Mask[1] 0xffffffffffffffff
439 SSOW HWS[0] SET[0] Group Mask[2] 0xffffffffffffffff
440 SSOW HWS[0] SET[0] Group Mask[3] 0xffffffffffffffff
441 SSOW HWS[0] SET[1] Group Mask[0] 0xffffffffffffffff
442 SSOW HWS[0] SET[1] Group Mask[1] 0xffffffffffffffff
443 SSOW HWS[0] SET[1] Group Mask[2] 0xffffffffffffffff
444 SSOW HWS[0] SET[1] Group Mask[3] 0xffffffffffffffff
445 ==================================================
450 DPDK may be compiled either natively on OCTEON TX2 platform or cross-compiled on
451 an x86 based platform.
459 .. code-block:: console
461 make config T=arm64-octeontx2-linux-gcc
464 The example applications can be compiled using the following:
466 .. code-block:: console
470 export RTE_TARGET=build
471 cd examples/<application>
477 .. code-block:: console
485 Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` for generic arm64 details.
490 .. code-block:: console
492 make config T=arm64-octeontx2-linux-gcc
493 make -j CROSS=aarch64-marvell-linux-gnu- CONFIG_RTE_KNI_KMOD=n
498 .. code-block:: console
500 meson build --cross-file config/arm/arm64_octeontx2_linux_gcc
505 By default, meson cross compilation uses ``aarch64-linux-gnu-gcc`` toolchain,
506 if Marvell toolchain is available then it can be used by overriding the
507 c, cpp, ar, strip ``binaries`` attributes to respective Marvell
508 toolchain binaries in ``config/arm/arm64_octeontx2_linux_gcc`` file.