1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Marvell International Ltd.
4 Marvell OCTEON TX2 Platform Guide
5 =================================
7 This document gives an overview of **Marvell OCTEON TX2** RVU H/W block,
8 packet flow and procedure to build DPDK on OCTEON TX2 platform.
10 More information about OCTEON TX2 SoC can be found at `Marvell Official Website
11 <https://www.marvell.com/embedded-processors/infrastructure-processors/>`_.
13 Supported OCTEON TX2 SoCs
14 -------------------------
19 OCTEON TX2 Resource Virtualization Unit architecture
20 ----------------------------------------------------
22 The :numref:`figure_octeontx2_resource_virtualization` diagram depicts the
23 RVU architecture and a resource provisioning example.
25 .. _figure_octeontx2_resource_virtualization:
27 .. figure:: img/octeontx2_resource_virtualization.*
29 OCTEON TX2 Resource virtualization architecture and provisioning example
32 Resource Virtualization Unit (RVU) on Marvell's OCTEON TX2 SoC maps HW
33 resources belonging to the network, crypto and other functional blocks onto
34 PCI-compatible physical and virtual functions.
36 Each functional block has multiple local functions (LFs) for
37 provisioning to different PCIe devices. RVU supports multiple PCIe SRIOV
38 physical functions (PFs) and virtual functions (VFs).
40 The :numref:`table_octeontx2_rvu_dpdk_mapping` shows the various local
41 functions (LFs) provided by the RVU and its functional mapping to
44 .. _table_octeontx2_rvu_dpdk_mapping:
46 .. table:: RVU managed functional blocks and its mapping to DPDK subsystem
48 +---+-----+--------------------------------------------------------------+
49 | # | LF | DPDK subsystem mapping |
50 +===+=====+==============================================================+
51 | 1 | NIX | rte_ethdev, rte_tm, rte_event_eth_[rt]x_adapter, rte_security|
52 +---+-----+--------------------------------------------------------------+
53 | 2 | NPA | rte_mempool |
54 +---+-----+--------------------------------------------------------------+
55 | 3 | NPC | rte_flow |
56 +---+-----+--------------------------------------------------------------+
57 | 4 | CPT | rte_cryptodev, rte_event_crypto_adapter |
58 +---+-----+--------------------------------------------------------------+
59 | 5 | SSO | rte_eventdev |
60 +---+-----+--------------------------------------------------------------+
61 | 6 | TIM | rte_event_timer_adapter |
62 +---+-----+--------------------------------------------------------------+
64 PF0 is called the administrative / admin function (AF) and has exclusive
65 privileges to provision RVU functional block's LFs to each of the PF/VF.
67 PF/VFs communicates with AF via a shared memory region (mailbox).Upon receiving
68 requests from PF/VF, AF does resource provisioning and other HW configuration.
70 AF is always attached to host, but PF/VFs may be used by host kernel itself,
71 or attached to VMs or to userspace applications like DPDK, etc. So, AF has to
72 handle provisioning/configuration requests sent by any device from any domain.
74 The AF driver does not receive or process any data.
75 It is only a configuration driver used in control path.
77 The :numref:`figure_octeontx2_resource_virtualization` diagram also shows a
78 resource provisioning example where,
80 1. PFx and PFx-VF0 bound to Linux netdev driver.
81 2. PFx-VF1 ethdev driver bound to the first DPDK application.
82 3. PFy ethdev driver, PFy-VF0 ethdev driver, PFz eventdev driver, PFm-VF0 cryptodev driver bound to the second DPDK application.
84 OCTEON TX2 packet flow
85 ----------------------
87 The :numref:`figure_octeontx2_packet_flow_hw_accelerators` diagram depicts
88 the packet flow on OCTEON TX2 SoC in conjunction with use of various HW accelerators.
90 .. _figure_octeontx2_packet_flow_hw_accelerators:
92 .. figure:: img/octeontx2_packet_flow_hw_accelerators.*
94 OCTEON TX2 packet flow in conjunction with use of HW accelerators
99 This section lists dataplane H/W block(s) available in OCTEON TX2 SoC.
101 #. **Mempool Driver**
102 See :doc:`../mempool/octeontx2` for NPA mempool driver information.
104 Procedure to Setup Platform
105 ---------------------------
107 There are three main prerequisites for setting up DPDK on OCTEON TX2
110 1. **OCTEON TX2 Linux kernel driver**
112 The dependent kernel drivers can be obtained from the
113 `kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/marvell/octeontx2>`_.
115 Alternatively, the Marvell SDK also provides the required kernel drivers.
117 Linux kernel should be configured with the following features enabled:
119 .. code-block:: console
121 # 64K pages enabled for better performance
122 CONFIG_ARM64_64K_PAGES=y
123 CONFIG_ARM64_VA_BITS_48=y
124 # huge pages support enabled
126 CONFIG_HUGETLB_PAGE=y
127 # VFIO enabled with TYPE1 IOMMU at minimum
128 CONFIG_VFIO_IOMMU_TYPE1=y
131 CONFIG_VFIO_NOIOMMU=y
133 CONFIG_VFIO_PCI_MMAP=y
136 # ARMv8.1 LSE atomics
137 CONFIG_ARM64_LSE_ATOMICS=y
139 CONFIG_OCTEONTX2_MBOX=y
140 CONFIG_OCTEONTX2_AF=y
141 # Enable if netdev PF driver required
142 CONFIG_OCTEONTX2_PF=y
143 # Enable if netdev VF driver required
144 CONFIG_OCTEONTX2_VF=y
145 CONFIG_CRYPTO_DEV_OCTEONTX2_CPT=y
147 2. **ARM64 Linux Tool Chain**
149 For example, the *aarch64* Linaro Toolchain, which can be obtained from
150 `here <https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/>`_.
152 Alternatively, the Marvell SDK also provides GNU GCC toolchain, which is
153 optimized for OCTEON TX2 CPU.
155 3. **Rootfile system**
157 Any *aarch64* supporting filesystem may be used. For example,
158 Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained
159 from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_.
161 Alternatively, the Marvell SDK provides the buildroot based root filesystem.
162 The SDK includes all the above prerequisites necessary to bring up the OCTEON TX2 board.
164 - Follow the DPDK :doc:`../linux_gsg/index` to setup the basic DPDK environment.
170 .. _table_octeontx2_common_debug_options:
172 .. table:: OCTEON TX2 common debug options
174 +---+------------+-------------------------------------------------------+
175 | # | Component | EAL log command |
176 +===+============+=======================================================+
177 | 1 | Common | --log-level='pmd\.octeontx2\.base,8' |
178 +---+------------+-------------------------------------------------------+
179 | 2 | Mailbox | --log-level='pmd\.octeontx2\.mbox,8' |
180 +---+------------+-------------------------------------------------------+
185 The **OCTEON TX2 Linux kernel driver** provides support to dump RVU blocks
186 context or stats using debugfs.
188 Enable ``debugfs`` by:
190 1. Compile kernel with debugfs enabled, i.e ``CONFIG_DEBUGFS=y``.
191 2. Boot OCTEON TX2 with debugfs supported kernel.
192 3. Verify ``debugfs`` mounted by default "mount | grep -i debugfs" or mount it manually by using.
194 .. code-block:: console
196 # mount -t debugfs none /sys/kernel/debug
198 Currently ``debugfs`` supports the following RVU blocks NIX, NPA, NPC, NDC,
201 The file structure under ``/sys/kernel/debug`` is as follows
203 .. code-block:: console
219 │ ├── cpt_engines_info
220 │ ├── cpt_engines_sts
227 │ ├── ndc_rx_hits_miss
229 │ ├── ndc_tx_hits_miss
233 │ └── tx_stall_hwissue
242 │ └── rx_miss_act_stats
248 ├── sso_hwgrp_aq_thresh
249 ├── sso_hwgrp_iaq_walk
251 ├── sso_hwgrp_free_list_walk
252 ├── sso_hwgrp_ient_walk
253 └── sso_hwgrp_taq_walk
255 RVU block LF allocation:
257 .. code-block:: console
259 cat /sys/kernel/debug/octeontx2/rsrc_alloc
261 pcifunc NPA NIX SSO GROUP SSOWS TIM CPT
268 .. code-block:: console
270 cat /sys/kernel/debug/octeontx2/cgx/cgx2/lmac0/stats
272 =======Link Status======
273 Link is UP 40000 Mbps
274 =======RX_STATS======
276 Octets of received packets: 0
277 Received PAUSE packets: 0
278 Received PAUSE and control packets: 0
279 Filtered DMAC0 (NIX-bound) packets: 0
280 Filtered DMAC0 (NIX-bound) octets: 0
281 Packets dropped due to RX FIFO full: 0
282 Octets dropped due to RX FIFO full: 0
284 Filtered DMAC1 (NCSI-bound) packets: 0
285 Filtered DMAC1 (NCSI-bound) octets: 0
286 NCSI-bound packets dropped: 0
287 NCSI-bound octets dropped: 0
288 =======TX_STATS======
289 Packets dropped due to excessive collisions: 0
290 Packets dropped due to excessive deferral: 0
291 Multiple collisions before successful transmission: 0
292 Single collisions before successful transmission: 0
293 Total octets sent on the interface: 0
294 Total frames sent on the interface: 0
295 Packets sent with an octet count < 64: 0
296 Packets sent with an octet count == 64: 0
297 Packets sent with an octet count of 65127: 0
298 Packets sent with an octet count of 128-255: 0
299 Packets sent with an octet count of 256-511: 0
300 Packets sent with an octet count of 512-1023: 0
301 Packets sent with an octet count of 1024-1518: 0
302 Packets sent with an octet count of > 1518: 0
303 Packets sent to a broadcast DMAC: 0
304 Packets sent to the multicast DMAC: 0
305 Transmit underflow and were truncated: 0
306 Control/PAUSE packets sent: 0
310 .. code-block:: console
312 cat /sys/kernel/debug/octeontx2/cpt/cpt_pc
314 CPT instruction requests 0
315 CPT instruction latency 0
316 CPT NCB read requests 0
317 CPT NCB read latency 0
318 CPT read requests caused by UC fills 0
319 CPT active cycles pc 1395642
320 CPT clock count pc 5579867595493
324 .. code-block:: console
326 Usage: echo <nixlf> [cq number/all] > /sys/kernel/debug/octeontx2/nix/cq_ctx
327 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
328 echo 0 0 > /sys/kernel/debug/octeontx2/nix/cq_ctx
329 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
331 =====cq_ctx for nixlf:0 and qidx:0 is=====
342 W2: update_time 31043
359 .. code-block:: console
361 Usage: echo <npalf> [pool number/all] > /sys/kernel/debug/octeontx2/npa/pool_ctx
362 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
363 echo 0 0 > /sys/kernel/debug/octeontx2/npa/pool_ctx
364 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
366 ======POOL : 0=======
367 W0: Stack base 1375bff00
374 W2: stack_max_pages 24315
375 W2: stack_pages 24314
385 W4: update_time 62993
387 W6: ptr_start 1593adf00
388 W7: ptr_end 180000000
394 W8: thresh_qint_idx 0
399 .. code-block:: console
401 cat /sys/kernel/debug/octeontx2/npc/mcam_info
404 RX keywidth : 224bits
405 TX keywidth : 224bits
417 .. code-block:: console
419 Usage: echo [<hws>/all] > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
420 echo 0 > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
422 ==================================================
423 SSOW HWS[0] Arbitration State 0x0
424 SSOW HWS[0] Guest Machine Control 0x0
425 SSOW HWS[0] SET[0] Group Mask[0] 0xffffffffffffffff
426 SSOW HWS[0] SET[0] Group Mask[1] 0xffffffffffffffff
427 SSOW HWS[0] SET[0] Group Mask[2] 0xffffffffffffffff
428 SSOW HWS[0] SET[0] Group Mask[3] 0xffffffffffffffff
429 SSOW HWS[0] SET[1] Group Mask[0] 0xffffffffffffffff
430 SSOW HWS[0] SET[1] Group Mask[1] 0xffffffffffffffff
431 SSOW HWS[0] SET[1] Group Mask[2] 0xffffffffffffffff
432 SSOW HWS[0] SET[1] Group Mask[3] 0xffffffffffffffff
433 ==================================================
438 DPDK may be compiled either natively on OCTEON TX2 platform or cross-compiled on
439 an x86 based platform.
447 .. code-block:: console
449 make config T=arm64-octeontx2-linux-gcc
452 The example applications can be compiled using the following:
454 .. code-block:: console
458 export RTE_TARGET=build
459 cd examples/<application>
465 .. code-block:: console
473 Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` for generic arm64 details.
478 .. code-block:: console
480 make config T=arm64-octeontx2-linux-gcc
481 make -j CROSS=aarch64-marvell-linux-gnu- CONFIG_RTE_KNI_KMOD=n
486 .. code-block:: console
488 meson build --cross-file config/arm/arm64_octeontx2_linux_gcc
493 By default, meson cross compilation uses ``aarch64-linux-gnu-gcc`` toolchain,
494 if Marvell toolchain is available then it can be used by overriding the
495 c, cpp, ar, strip ``binaries`` attributes to respective Marvell
496 toolchain binaries in ``config/arm/arm64_octeontx2_linux_gcc`` file.