1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Marvell International Ltd.
4 Marvell OCTEON TX2 Platform Guide
5 =================================
7 This document gives an overview of **Marvell OCTEON TX2** RVU H/W block,
8 packet flow and procedure to build DPDK on OCTEON TX2 platform.
10 More information about OCTEON TX2 SoC can be found at `Marvell Official Website
11 <https://www.marvell.com/embedded-processors/infrastructure-processors/>`_.
13 Supported OCTEON TX2 SoCs
14 -------------------------
19 OCTEON TX2 Resource Virtualization Unit architecture
20 ----------------------------------------------------
22 The :numref:`figure_octeontx2_resource_virtualization` diagram depicts the
23 RVU architecture and a resource provisioning example.
25 .. _figure_octeontx2_resource_virtualization:
27 .. figure:: img/octeontx2_resource_virtualization.*
29 OCTEON TX2 Resource virtualization architecture and provisioning example
32 Resource Virtualization Unit (RVU) on Marvell's OCTEON TX2 SoC maps HW
33 resources belonging to the network, crypto and other functional blocks onto
34 PCI-compatible physical and virtual functions.
36 Each functional block has multiple local functions (LFs) for
37 provisioning to different PCIe devices. RVU supports multiple PCIe SRIOV
38 physical functions (PFs) and virtual functions (VFs).
40 The :numref:`table_octeontx2_rvu_dpdk_mapping` shows the various local
41 functions (LFs) provided by the RVU and its functional mapping to
44 .. _table_octeontx2_rvu_dpdk_mapping:
46 .. table:: RVU managed functional blocks and its mapping to DPDK subsystem
48 +---+-----+--------------------------------------------------------------+
49 | # | LF | DPDK subsystem mapping |
50 +===+=====+==============================================================+
51 | 1 | NIX | rte_ethdev, rte_tm, rte_event_eth_[rt]x_adapter, rte_security|
52 +---+-----+--------------------------------------------------------------+
53 | 2 | NPA | rte_mempool |
54 +---+-----+--------------------------------------------------------------+
55 | 3 | NPC | rte_flow |
56 +---+-----+--------------------------------------------------------------+
57 | 4 | CPT | rte_cryptodev, rte_event_crypto_adapter |
58 +---+-----+--------------------------------------------------------------+
59 | 5 | SSO | rte_eventdev |
60 +---+-----+--------------------------------------------------------------+
61 | 6 | TIM | rte_event_timer_adapter |
62 +---+-----+--------------------------------------------------------------+
64 PF0 is called the administrative / admin function (AF) and has exclusive
65 privileges to provision RVU functional block's LFs to each of the PF/VF.
67 PF/VFs communicates with AF via a shared memory region (mailbox).Upon receiving
68 requests from PF/VF, AF does resource provisioning and other HW configuration.
70 AF is always attached to host, but PF/VFs may be used by host kernel itself,
71 or attached to VMs or to userspace applications like DPDK, etc. So, AF has to
72 handle provisioning/configuration requests sent by any device from any domain.
74 The AF driver does not receive or process any data.
75 It is only a configuration driver used in control path.
77 The :numref:`figure_octeontx2_resource_virtualization` diagram also shows a
78 resource provisioning example where,
80 1. PFx and PFx-VF0 bound to Linux netdev driver.
81 2. PFx-VF1 ethdev driver bound to the first DPDK application.
82 3. PFy ethdev driver, PFy-VF0 ethdev driver, PFz eventdev driver, PFm-VF0 cryptodev driver bound to the second DPDK application.
84 OCTEON TX2 packet flow
85 ----------------------
87 The :numref:`figure_octeontx2_packet_flow_hw_accelerators` diagram depicts
88 the packet flow on OCTEON TX2 SoC in conjunction with use of various HW accelerators.
90 .. _figure_octeontx2_packet_flow_hw_accelerators:
92 .. figure:: img/octeontx2_packet_flow_hw_accelerators.*
94 OCTEON TX2 packet flow in conjunction with use of HW accelerators
99 This section lists dataplane H/W block(s) available in OCTEON TX2 SoC.
101 #. **Mempool Driver**
102 See :doc:`../mempool/octeontx2` for NPA mempool driver information.
104 #. **Event Device Driver**
105 See :doc:`../eventdevs/octeontx2` for SSO event device driver information.
107 #. **DMA Rawdev Driver**
108 See :doc:`../rawdevs/octeontx2_dma` for DMA driver information.
110 Procedure to Setup Platform
111 ---------------------------
113 There are three main prerequisites for setting up DPDK on OCTEON TX2
116 1. **OCTEON TX2 Linux kernel driver**
118 The dependent kernel drivers can be obtained from the
119 `kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/marvell/octeontx2>`_.
121 Alternatively, the Marvell SDK also provides the required kernel drivers.
123 Linux kernel should be configured with the following features enabled:
125 .. code-block:: console
127 # 64K pages enabled for better performance
128 CONFIG_ARM64_64K_PAGES=y
129 CONFIG_ARM64_VA_BITS_48=y
130 # huge pages support enabled
132 CONFIG_HUGETLB_PAGE=y
133 # VFIO enabled with TYPE1 IOMMU at minimum
134 CONFIG_VFIO_IOMMU_TYPE1=y
137 CONFIG_VFIO_NOIOMMU=y
139 CONFIG_VFIO_PCI_MMAP=y
142 # ARMv8.1 LSE atomics
143 CONFIG_ARM64_LSE_ATOMICS=y
145 CONFIG_OCTEONTX2_MBOX=y
146 CONFIG_OCTEONTX2_AF=y
147 # Enable if netdev PF driver required
148 CONFIG_OCTEONTX2_PF=y
149 # Enable if netdev VF driver required
150 CONFIG_OCTEONTX2_VF=y
151 CONFIG_CRYPTO_DEV_OCTEONTX2_CPT=y
152 # Enable if OCTEONTX2 DMA PF driver required
153 CONFIG_OCTEONTX2_DPI_PF=n
155 2. **ARM64 Linux Tool Chain**
157 For example, the *aarch64* Linaro Toolchain, which can be obtained from
158 `here <https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/>`_.
160 Alternatively, the Marvell SDK also provides GNU GCC toolchain, which is
161 optimized for OCTEON TX2 CPU.
163 3. **Rootfile system**
165 Any *aarch64* supporting filesystem may be used. For example,
166 Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained
167 from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_.
169 Alternatively, the Marvell SDK provides the buildroot based root filesystem.
170 The SDK includes all the above prerequisites necessary to bring up the OCTEON TX2 board.
172 - Follow the DPDK :doc:`../linux_gsg/index` to setup the basic DPDK environment.
178 .. _table_octeontx2_common_debug_options:
180 .. table:: OCTEON TX2 common debug options
182 +---+------------+-------------------------------------------------------+
183 | # | Component | EAL log command |
184 +===+============+=======================================================+
185 | 1 | Common | --log-level='pmd\.octeontx2\.base,8' |
186 +---+------------+-------------------------------------------------------+
187 | 2 | Mailbox | --log-level='pmd\.octeontx2\.mbox,8' |
188 +---+------------+-------------------------------------------------------+
193 The **OCTEON TX2 Linux kernel driver** provides support to dump RVU blocks
194 context or stats using debugfs.
196 Enable ``debugfs`` by:
198 1. Compile kernel with debugfs enabled, i.e ``CONFIG_DEBUGFS=y``.
199 2. Boot OCTEON TX2 with debugfs supported kernel.
200 3. Verify ``debugfs`` mounted by default "mount | grep -i debugfs" or mount it manually by using.
202 .. code-block:: console
204 # mount -t debugfs none /sys/kernel/debug
206 Currently ``debugfs`` supports the following RVU blocks NIX, NPA, NPC, NDC,
209 The file structure under ``/sys/kernel/debug`` is as follows
211 .. code-block:: console
227 │ ├── cpt_engines_info
228 │ ├── cpt_engines_sts
235 │ ├── ndc_rx_hits_miss
237 │ ├── ndc_tx_hits_miss
241 │ └── tx_stall_hwissue
250 │ └── rx_miss_act_stats
256 ├── sso_hwgrp_aq_thresh
257 ├── sso_hwgrp_iaq_walk
259 ├── sso_hwgrp_free_list_walk
260 ├── sso_hwgrp_ient_walk
261 └── sso_hwgrp_taq_walk
263 RVU block LF allocation:
265 .. code-block:: console
267 cat /sys/kernel/debug/octeontx2/rsrc_alloc
269 pcifunc NPA NIX SSO GROUP SSOWS TIM CPT
276 .. code-block:: console
278 cat /sys/kernel/debug/octeontx2/cgx/cgx2/lmac0/stats
280 =======Link Status======
281 Link is UP 40000 Mbps
282 =======RX_STATS======
284 Octets of received packets: 0
285 Received PAUSE packets: 0
286 Received PAUSE and control packets: 0
287 Filtered DMAC0 (NIX-bound) packets: 0
288 Filtered DMAC0 (NIX-bound) octets: 0
289 Packets dropped due to RX FIFO full: 0
290 Octets dropped due to RX FIFO full: 0
292 Filtered DMAC1 (NCSI-bound) packets: 0
293 Filtered DMAC1 (NCSI-bound) octets: 0
294 NCSI-bound packets dropped: 0
295 NCSI-bound octets dropped: 0
296 =======TX_STATS======
297 Packets dropped due to excessive collisions: 0
298 Packets dropped due to excessive deferral: 0
299 Multiple collisions before successful transmission: 0
300 Single collisions before successful transmission: 0
301 Total octets sent on the interface: 0
302 Total frames sent on the interface: 0
303 Packets sent with an octet count < 64: 0
304 Packets sent with an octet count == 64: 0
305 Packets sent with an octet count of 65127: 0
306 Packets sent with an octet count of 128-255: 0
307 Packets sent with an octet count of 256-511: 0
308 Packets sent with an octet count of 512-1023: 0
309 Packets sent with an octet count of 1024-1518: 0
310 Packets sent with an octet count of > 1518: 0
311 Packets sent to a broadcast DMAC: 0
312 Packets sent to the multicast DMAC: 0
313 Transmit underflow and were truncated: 0
314 Control/PAUSE packets sent: 0
318 .. code-block:: console
320 cat /sys/kernel/debug/octeontx2/cpt/cpt_pc
322 CPT instruction requests 0
323 CPT instruction latency 0
324 CPT NCB read requests 0
325 CPT NCB read latency 0
326 CPT read requests caused by UC fills 0
327 CPT active cycles pc 1395642
328 CPT clock count pc 5579867595493
332 .. code-block:: console
334 Usage: echo <nixlf> [cq number/all] > /sys/kernel/debug/octeontx2/nix/cq_ctx
335 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
336 echo 0 0 > /sys/kernel/debug/octeontx2/nix/cq_ctx
337 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
339 =====cq_ctx for nixlf:0 and qidx:0 is=====
350 W2: update_time 31043
367 .. code-block:: console
369 Usage: echo <npalf> [pool number/all] > /sys/kernel/debug/octeontx2/npa/pool_ctx
370 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
371 echo 0 0 > /sys/kernel/debug/octeontx2/npa/pool_ctx
372 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
374 ======POOL : 0=======
375 W0: Stack base 1375bff00
382 W2: stack_max_pages 24315
383 W2: stack_pages 24314
393 W4: update_time 62993
395 W6: ptr_start 1593adf00
396 W7: ptr_end 180000000
402 W8: thresh_qint_idx 0
407 .. code-block:: console
409 cat /sys/kernel/debug/octeontx2/npc/mcam_info
412 RX keywidth : 224bits
413 TX keywidth : 224bits
425 .. code-block:: console
427 Usage: echo [<hws>/all] > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
428 echo 0 > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
430 ==================================================
431 SSOW HWS[0] Arbitration State 0x0
432 SSOW HWS[0] Guest Machine Control 0x0
433 SSOW HWS[0] SET[0] Group Mask[0] 0xffffffffffffffff
434 SSOW HWS[0] SET[0] Group Mask[1] 0xffffffffffffffff
435 SSOW HWS[0] SET[0] Group Mask[2] 0xffffffffffffffff
436 SSOW HWS[0] SET[0] Group Mask[3] 0xffffffffffffffff
437 SSOW HWS[0] SET[1] Group Mask[0] 0xffffffffffffffff
438 SSOW HWS[0] SET[1] Group Mask[1] 0xffffffffffffffff
439 SSOW HWS[0] SET[1] Group Mask[2] 0xffffffffffffffff
440 SSOW HWS[0] SET[1] Group Mask[3] 0xffffffffffffffff
441 ==================================================
446 DPDK may be compiled either natively on OCTEON TX2 platform or cross-compiled on
447 an x86 based platform.
455 .. code-block:: console
457 make config T=arm64-octeontx2-linux-gcc
460 The example applications can be compiled using the following:
462 .. code-block:: console
466 export RTE_TARGET=build
467 cd examples/<application>
473 .. code-block:: console
481 Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` for generic arm64 details.
486 .. code-block:: console
488 make config T=arm64-octeontx2-linux-gcc
489 make -j CROSS=aarch64-marvell-linux-gnu- CONFIG_RTE_KNI_KMOD=n
494 .. code-block:: console
496 meson build --cross-file config/arm/arm64_octeontx2_linux_gcc
501 By default, meson cross compilation uses ``aarch64-linux-gnu-gcc`` toolchain,
502 if Marvell toolchain is available then it can be used by overriding the
503 c, cpp, ar, strip ``binaries`` attributes to respective Marvell
504 toolchain binaries in ``config/arm/arm64_octeontx2_linux_gcc`` file.