1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Marvell International Ltd.
4 Marvell OCTEON TX2 Platform Guide
5 =================================
7 This document gives an overview of **Marvell OCTEON TX2** RVU H/W block,
8 packet flow and procedure to build DPDK on OCTEON TX2 platform.
10 More information about OCTEON TX2 SoC can be found at `Marvell Official Website
11 <https://www.marvell.com/embedded-processors/infrastructure-processors/>`_.
13 Supported OCTEON TX2 SoCs
14 -------------------------
19 OCTEON TX2 Resource Virtualization Unit architecture
20 ----------------------------------------------------
22 The :numref:`figure_octeontx2_resource_virtualization` diagram depicts the
23 RVU architecture and a resource provisioning example.
25 .. _figure_octeontx2_resource_virtualization:
27 .. figure:: img/octeontx2_resource_virtualization.*
29 OCTEON TX2 Resource virtualization architecture and provisioning example
32 Resource Virtualization Unit (RVU) on Marvell's OCTEON TX2 SoC maps HW
33 resources belonging to the network, crypto and other functional blocks onto
34 PCI-compatible physical and virtual functions.
36 Each functional block has multiple local functions (LFs) for
37 provisioning to different PCIe devices. RVU supports multiple PCIe SRIOV
38 physical functions (PFs) and virtual functions (VFs).
40 The :numref:`table_octeontx2_rvu_dpdk_mapping` shows the various local
41 functions (LFs) provided by the RVU and its functional mapping to
44 .. _table_octeontx2_rvu_dpdk_mapping:
46 .. table:: RVU managed functional blocks and its mapping to DPDK subsystem
48 +---+-----+--------------------------------------------------------------+
49 | # | LF | DPDK subsystem mapping |
50 +===+=====+==============================================================+
51 | 1 | NIX | rte_ethdev, rte_tm, rte_event_eth_[rt]x_adapter, rte_security|
52 +---+-----+--------------------------------------------------------------+
53 | 2 | NPA | rte_mempool |
54 +---+-----+--------------------------------------------------------------+
55 | 3 | NPC | rte_flow |
56 +---+-----+--------------------------------------------------------------+
57 | 4 | CPT | rte_cryptodev, rte_event_crypto_adapter |
58 +---+-----+--------------------------------------------------------------+
59 | 5 | SSO | rte_eventdev |
60 +---+-----+--------------------------------------------------------------+
61 | 6 | TIM | rte_event_timer_adapter |
62 +---+-----+--------------------------------------------------------------+
64 PF0 is called the administrative / admin function (AF) and has exclusive
65 privileges to provision RVU functional block's LFs to each of the PF/VF.
67 PF/VFs communicates with AF via a shared memory region (mailbox).Upon receiving
68 requests from PF/VF, AF does resource provisioning and other HW configuration.
70 AF is always attached to host, but PF/VFs may be used by host kernel itself,
71 or attached to VMs or to userspace applications like DPDK, etc. So, AF has to
72 handle provisioning/configuration requests sent by any device from any domain.
74 The AF driver does not receive or process any data.
75 It is only a configuration driver used in control path.
77 The :numref:`figure_octeontx2_resource_virtualization` diagram also shows a
78 resource provisioning example where,
80 1. PFx and PFx-VF0 bound to Linux netdev driver.
81 2. PFx-VF1 ethdev driver bound to the first DPDK application.
82 3. PFy ethdev driver, PFy-VF0 ethdev driver, PFz eventdev driver, PFm-VF0 cryptodev driver bound to the second DPDK application.
84 OCTEON TX2 packet flow
85 ----------------------
87 The :numref:`figure_octeontx2_packet_flow_hw_accelerators` diagram depicts
88 the packet flow on OCTEON TX2 SoC in conjunction with use of various HW accelerators.
90 .. _figure_octeontx2_packet_flow_hw_accelerators:
92 .. figure:: img/octeontx2_packet_flow_hw_accelerators.*
94 OCTEON TX2 packet flow in conjunction with use of HW accelerators
99 This section lists dataplane H/W block(s) available in OCTEON TX2 SoC.
102 Procedure to Setup Platform
103 ---------------------------
105 There are three main prerequisites for setting up DPDK on OCTEON TX2
108 1. **OCTEON TX2 Linux kernel driver**
110 The dependent kernel drivers can be obtained from the
111 `kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/marvell/octeontx2>`_.
113 Alternatively, the Marvell SDK also provides the required kernel drivers.
115 Linux kernel should be configured with the following features enabled:
117 .. code-block:: console
119 # 64K pages enabled for better performance
120 CONFIG_ARM64_64K_PAGES=y
121 CONFIG_ARM64_VA_BITS_48=y
122 # huge pages support enabled
124 CONFIG_HUGETLB_PAGE=y
125 # VFIO enabled with TYPE1 IOMMU at minimum
126 CONFIG_VFIO_IOMMU_TYPE1=y
129 CONFIG_VFIO_NOIOMMU=y
131 CONFIG_VFIO_PCI_MMAP=y
134 # ARMv8.1 LSE atomics
135 CONFIG_ARM64_LSE_ATOMICS=y
137 CONFIG_OCTEONTX2_MBOX=y
138 CONFIG_OCTEONTX2_AF=y
139 # Enable if netdev PF driver required
140 CONFIG_OCTEONTX2_PF=y
141 # Enable if netdev VF driver required
142 CONFIG_OCTEONTX2_VF=y
143 CONFIG_CRYPTO_DEV_OCTEONTX2_CPT=y
145 2. **ARM64 Linux Tool Chain**
147 For example, the *aarch64* Linaro Toolchain, which can be obtained from
148 `here <https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/>`_.
150 Alternatively, the Marvell SDK also provides GNU GCC toolchain, which is
151 optimized for OCTEON TX2 CPU.
153 3. **Rootfile system**
155 Any *aarch64* supporting filesystem may be used. For example,
156 Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained
157 from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_.
159 Alternatively, the Marvell SDK provides the buildroot based root filesystem.
160 The SDK includes all the above prerequisites necessary to bring up the OCTEON TX2 board.
162 - Follow the DPDK :doc:`../linux_gsg/index` to setup the basic DPDK environment.
168 .. _table_octeontx2_common_debug_options:
170 .. table:: OCTEON TX2 common debug options
172 +---+------------+-------------------------------------------------------+
173 | # | Component | EAL log command |
174 +===+============+=======================================================+
175 | 1 | Common | --log-level='pmd\.octeontx2\.base,8' |
176 +---+------------+-------------------------------------------------------+
177 | 2 | Mailbox | --log-level='pmd\.octeontx2\.mbox,8' |
178 +---+------------+-------------------------------------------------------+
183 The **OCTEON TX2 Linux kernel driver** provides support to dump RVU blocks
184 context or stats using debugfs.
186 Enable ``debugfs`` by:
188 1. Compile kernel with debugfs enabled, i.e ``CONFIG_DEBUGFS=y``.
189 2. Boot OCTEON TX2 with debugfs supported kernel.
190 3. Verify ``debugfs`` mounted by default "mount | grep -i debugfs" or mount it manually by using.
192 .. code-block:: console
194 # mount -t debugfs none /sys/kernel/debug
196 Currently ``debugfs`` supports the following RVU blocks NIX, NPA, NPC, NDC,
199 The file structure under ``/sys/kernel/debug`` is as follows
201 .. code-block:: console
217 │ ├── cpt_engines_info
218 │ ├── cpt_engines_sts
225 │ ├── ndc_rx_hits_miss
227 │ ├── ndc_tx_hits_miss
231 │ └── tx_stall_hwissue
240 │ └── rx_miss_act_stats
246 ├── sso_hwgrp_aq_thresh
247 ├── sso_hwgrp_iaq_walk
249 ├── sso_hwgrp_free_list_walk
250 ├── sso_hwgrp_ient_walk
251 └── sso_hwgrp_taq_walk
253 RVU block LF allocation:
255 .. code-block:: console
257 cat /sys/kernel/debug/octeontx2/rsrc_alloc
259 pcifunc NPA NIX SSO GROUP SSOWS TIM CPT
266 .. code-block:: console
268 cat /sys/kernel/debug/octeontx2/cgx/cgx2/lmac0/stats
270 =======Link Status======
271 Link is UP 40000 Mbps
272 =======RX_STATS======
274 Octets of received packets: 0
275 Received PAUSE packets: 0
276 Received PAUSE and control packets: 0
277 Filtered DMAC0 (NIX-bound) packets: 0
278 Filtered DMAC0 (NIX-bound) octets: 0
279 Packets dropped due to RX FIFO full: 0
280 Octets dropped due to RX FIFO full: 0
282 Filtered DMAC1 (NCSI-bound) packets: 0
283 Filtered DMAC1 (NCSI-bound) octets: 0
284 NCSI-bound packets dropped: 0
285 NCSI-bound octets dropped: 0
286 =======TX_STATS======
287 Packets dropped due to excessive collisions: 0
288 Packets dropped due to excessive deferral: 0
289 Multiple collisions before successful transmission: 0
290 Single collisions before successful transmission: 0
291 Total octets sent on the interface: 0
292 Total frames sent on the interface: 0
293 Packets sent with an octet count < 64: 0
294 Packets sent with an octet count == 64: 0
295 Packets sent with an octet count of 65127: 0
296 Packets sent with an octet count of 128-255: 0
297 Packets sent with an octet count of 256-511: 0
298 Packets sent with an octet count of 512-1023: 0
299 Packets sent with an octet count of 1024-1518: 0
300 Packets sent with an octet count of > 1518: 0
301 Packets sent to a broadcast DMAC: 0
302 Packets sent to the multicast DMAC: 0
303 Transmit underflow and were truncated: 0
304 Control/PAUSE packets sent: 0
308 .. code-block:: console
310 cat /sys/kernel/debug/octeontx2/cpt/cpt_pc
312 CPT instruction requests 0
313 CPT instruction latency 0
314 CPT NCB read requests 0
315 CPT NCB read latency 0
316 CPT read requests caused by UC fills 0
317 CPT active cycles pc 1395642
318 CPT clock count pc 5579867595493
322 .. code-block:: console
324 Usage: echo <nixlf> [cq number/all] > /sys/kernel/debug/octeontx2/nix/cq_ctx
325 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
326 echo 0 0 > /sys/kernel/debug/octeontx2/nix/cq_ctx
327 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
329 =====cq_ctx for nixlf:0 and qidx:0 is=====
340 W2: update_time 31043
357 .. code-block:: console
359 Usage: echo <npalf> [pool number/all] > /sys/kernel/debug/octeontx2/npa/pool_ctx
360 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
361 echo 0 0 > /sys/kernel/debug/octeontx2/npa/pool_ctx
362 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
364 ======POOL : 0=======
365 W0: Stack base 1375bff00
372 W2: stack_max_pages 24315
373 W2: stack_pages 24314
383 W4: update_time 62993
385 W6: ptr_start 1593adf00
386 W7: ptr_end 180000000
392 W8: thresh_qint_idx 0
397 .. code-block:: console
399 cat /sys/kernel/debug/octeontx2/npc/mcam_info
402 RX keywidth : 224bits
403 TX keywidth : 224bits
415 .. code-block:: console
417 Usage: echo [<hws>/all] > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
418 echo 0 > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
420 ==================================================
421 SSOW HWS[0] Arbitration State 0x0
422 SSOW HWS[0] Guest Machine Control 0x0
423 SSOW HWS[0] SET[0] Group Mask[0] 0xffffffffffffffff
424 SSOW HWS[0] SET[0] Group Mask[1] 0xffffffffffffffff
425 SSOW HWS[0] SET[0] Group Mask[2] 0xffffffffffffffff
426 SSOW HWS[0] SET[0] Group Mask[3] 0xffffffffffffffff
427 SSOW HWS[0] SET[1] Group Mask[0] 0xffffffffffffffff
428 SSOW HWS[0] SET[1] Group Mask[1] 0xffffffffffffffff
429 SSOW HWS[0] SET[1] Group Mask[2] 0xffffffffffffffff
430 SSOW HWS[0] SET[1] Group Mask[3] 0xffffffffffffffff
431 ==================================================
436 DPDK may be compiled either natively on OCTEON TX2 platform or cross-compiled on
437 an x86 based platform.
445 .. code-block:: console
447 make config T=arm64-octeontx2-linux-gcc
450 The example applications can be compiled using the following:
452 .. code-block:: console
456 export RTE_TARGET=build
457 cd examples/<application>
463 .. code-block:: console
471 Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` for generic arm64 details.
476 .. code-block:: console
478 make config T=arm64-octeontx2-linux-gcc
479 make -j CROSS=aarch64-marvell-linux-gnu- CONFIG_RTE_KNI_KMOD=n
484 .. code-block:: console
486 meson build --cross-file config/arm/arm64_octeontx2_linux_gcc
491 By default, meson cross compilation uses ``aarch64-linux-gnu-gcc`` toolchain,
492 if Marvell toolchain is available then it can be used by overriding the
493 c, cpp, ar, strip ``binaries`` attributes to respective Marvell
494 toolchain binaries in ``config/arm/arm64_octeontx2_linux_gcc`` file.