1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Marvell International Ltd.
4 Marvell OCTEON TX2 Platform Guide
5 =================================
7 This document gives an overview of **Marvell OCTEON TX2** RVU H/W block,
8 packet flow and procedure to build DPDK on OCTEON TX2 platform.
10 More information about OCTEON TX2 SoC can be found at `Marvell Official Website
11 <https://www.marvell.com/embedded-processors/infrastructure-processors/>`_.
13 Supported OCTEON TX2 SoCs
14 -------------------------
19 OCTEON TX2 Resource Virtualization Unit architecture
20 ----------------------------------------------------
22 The :numref:`figure_octeontx2_resource_virtualization` diagram depicts the
23 RVU architecture and a resource provisioning example.
25 .. _figure_octeontx2_resource_virtualization:
27 .. figure:: img/octeontx2_resource_virtualization.*
29 OCTEON TX2 Resource virtualization architecture and provisioning example
32 Resource Virtualization Unit (RVU) on Marvell's OCTEON TX2 SoC maps HW
33 resources belonging to the network, crypto and other functional blocks onto
34 PCI-compatible physical and virtual functions.
36 Each functional block has multiple local functions (LFs) for
37 provisioning to different PCIe devices. RVU supports multiple PCIe SRIOV
38 physical functions (PFs) and virtual functions (VFs).
40 The :numref:`table_octeontx2_rvu_dpdk_mapping` shows the various local
41 functions (LFs) provided by the RVU and its functional mapping to
44 .. _table_octeontx2_rvu_dpdk_mapping:
46 .. table:: RVU managed functional blocks and its mapping to DPDK subsystem
48 +---+-----+--------------------------------------------------------------+
49 | # | LF | DPDK subsystem mapping |
50 +===+=====+==============================================================+
51 | 1 | NIX | rte_ethdev, rte_tm, rte_event_eth_[rt]x_adapter, rte_security|
52 +---+-----+--------------------------------------------------------------+
53 | 2 | NPA | rte_mempool |
54 +---+-----+--------------------------------------------------------------+
55 | 3 | NPC | rte_flow |
56 +---+-----+--------------------------------------------------------------+
57 | 4 | CPT | rte_cryptodev, rte_event_crypto_adapter |
58 +---+-----+--------------------------------------------------------------+
59 | 5 | SSO | rte_eventdev |
60 +---+-----+--------------------------------------------------------------+
61 | 6 | TIM | rte_event_timer_adapter |
62 +---+-----+--------------------------------------------------------------+
64 PF0 is called the administrative / admin function (AF) and has exclusive
65 privileges to provision RVU functional block's LFs to each of the PF/VF.
67 PF/VFs communicates with AF via a shared memory region (mailbox).Upon receiving
68 requests from PF/VF, AF does resource provisioning and other HW configuration.
70 AF is always attached to host, but PF/VFs may be used by host kernel itself,
71 or attached to VMs or to userspace applications like DPDK, etc. So, AF has to
72 handle provisioning/configuration requests sent by any device from any domain.
74 The AF driver does not receive or process any data.
75 It is only a configuration driver used in control path.
77 The :numref:`figure_octeontx2_resource_virtualization` diagram also shows a
78 resource provisioning example where,
80 1. PFx and PFx-VF0 bound to Linux netdev driver.
81 2. PFx-VF1 ethdev driver bound to the first DPDK application.
82 3. PFy ethdev driver, PFy-VF0 ethdev driver, PFz eventdev driver, PFm-VF0 cryptodev driver bound to the second DPDK application.
84 OCTEON TX2 packet flow
85 ----------------------
87 The :numref:`figure_octeontx2_packet_flow_hw_accelerators` diagram depicts
88 the packet flow on OCTEON TX2 SoC in conjunction with use of various HW accelerators.
90 .. _figure_octeontx2_packet_flow_hw_accelerators:
92 .. figure:: img/octeontx2_packet_flow_hw_accelerators.*
94 OCTEON TX2 packet flow in conjunction with use of HW accelerators
99 This section lists dataplane H/W block(s) available in OCTEON TX2 SoC.
102 See :doc:`../nics/octeontx2` for NIX Ethdev driver information.
104 #. **Mempool Driver**
105 See :doc:`../mempool/octeontx2` for NPA mempool driver information.
107 #. **Event Device Driver**
108 See :doc:`../eventdevs/octeontx2` for SSO event device driver information.
110 #. **DMA Rawdev Driver**
111 See :doc:`../rawdevs/octeontx2_dma` for DMA driver information.
113 Procedure to Setup Platform
114 ---------------------------
116 There are three main prerequisites for setting up DPDK on OCTEON TX2
119 1. **OCTEON TX2 Linux kernel driver**
121 The dependent kernel drivers can be obtained from the
122 `kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/marvell/octeontx2>`_.
124 Alternatively, the Marvell SDK also provides the required kernel drivers.
126 Linux kernel should be configured with the following features enabled:
128 .. code-block:: console
130 # 64K pages enabled for better performance
131 CONFIG_ARM64_64K_PAGES=y
132 CONFIG_ARM64_VA_BITS_48=y
133 # huge pages support enabled
135 CONFIG_HUGETLB_PAGE=y
136 # VFIO enabled with TYPE1 IOMMU at minimum
137 CONFIG_VFIO_IOMMU_TYPE1=y
140 CONFIG_VFIO_NOIOMMU=y
142 CONFIG_VFIO_PCI_MMAP=y
145 # ARMv8.1 LSE atomics
146 CONFIG_ARM64_LSE_ATOMICS=y
148 CONFIG_OCTEONTX2_MBOX=y
149 CONFIG_OCTEONTX2_AF=y
150 # Enable if netdev PF driver required
151 CONFIG_OCTEONTX2_PF=y
152 # Enable if netdev VF driver required
153 CONFIG_OCTEONTX2_VF=y
154 CONFIG_CRYPTO_DEV_OCTEONTX2_CPT=y
155 # Enable if OCTEONTX2 DMA PF driver required
156 CONFIG_OCTEONTX2_DPI_PF=n
158 2. **ARM64 Linux Tool Chain**
160 For example, the *aarch64* Linaro Toolchain, which can be obtained from
161 `here <https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/>`_.
163 Alternatively, the Marvell SDK also provides GNU GCC toolchain, which is
164 optimized for OCTEON TX2 CPU.
166 3. **Rootfile system**
168 Any *aarch64* supporting filesystem may be used. For example,
169 Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained
170 from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_.
172 Alternatively, the Marvell SDK provides the buildroot based root filesystem.
173 The SDK includes all the above prerequisites necessary to bring up the OCTEON TX2 board.
175 - Follow the DPDK :doc:`../linux_gsg/index` to setup the basic DPDK environment.
181 .. _table_octeontx2_common_debug_options:
183 .. table:: OCTEON TX2 common debug options
185 +---+------------+-------------------------------------------------------+
186 | # | Component | EAL log command |
187 +===+============+=======================================================+
188 | 1 | Common | --log-level='pmd\.octeontx2\.base,8' |
189 +---+------------+-------------------------------------------------------+
190 | 2 | Mailbox | --log-level='pmd\.octeontx2\.mbox,8' |
191 +---+------------+-------------------------------------------------------+
196 The **OCTEON TX2 Linux kernel driver** provides support to dump RVU blocks
197 context or stats using debugfs.
199 Enable ``debugfs`` by:
201 1. Compile kernel with debugfs enabled, i.e ``CONFIG_DEBUGFS=y``.
202 2. Boot OCTEON TX2 with debugfs supported kernel.
203 3. Verify ``debugfs`` mounted by default "mount | grep -i debugfs" or mount it manually by using.
205 .. code-block:: console
207 # mount -t debugfs none /sys/kernel/debug
209 Currently ``debugfs`` supports the following RVU blocks NIX, NPA, NPC, NDC,
212 The file structure under ``/sys/kernel/debug`` is as follows
214 .. code-block:: console
230 │ ├── cpt_engines_info
231 │ ├── cpt_engines_sts
238 │ ├── ndc_rx_hits_miss
240 │ ├── ndc_tx_hits_miss
244 │ └── tx_stall_hwissue
253 │ └── rx_miss_act_stats
259 ├── sso_hwgrp_aq_thresh
260 ├── sso_hwgrp_iaq_walk
262 ├── sso_hwgrp_free_list_walk
263 ├── sso_hwgrp_ient_walk
264 └── sso_hwgrp_taq_walk
266 RVU block LF allocation:
268 .. code-block:: console
270 cat /sys/kernel/debug/octeontx2/rsrc_alloc
272 pcifunc NPA NIX SSO GROUP SSOWS TIM CPT
279 .. code-block:: console
281 cat /sys/kernel/debug/octeontx2/cgx/cgx2/lmac0/stats
283 =======Link Status======
284 Link is UP 40000 Mbps
285 =======RX_STATS======
287 Octets of received packets: 0
288 Received PAUSE packets: 0
289 Received PAUSE and control packets: 0
290 Filtered DMAC0 (NIX-bound) packets: 0
291 Filtered DMAC0 (NIX-bound) octets: 0
292 Packets dropped due to RX FIFO full: 0
293 Octets dropped due to RX FIFO full: 0
295 Filtered DMAC1 (NCSI-bound) packets: 0
296 Filtered DMAC1 (NCSI-bound) octets: 0
297 NCSI-bound packets dropped: 0
298 NCSI-bound octets dropped: 0
299 =======TX_STATS======
300 Packets dropped due to excessive collisions: 0
301 Packets dropped due to excessive deferral: 0
302 Multiple collisions before successful transmission: 0
303 Single collisions before successful transmission: 0
304 Total octets sent on the interface: 0
305 Total frames sent on the interface: 0
306 Packets sent with an octet count < 64: 0
307 Packets sent with an octet count == 64: 0
308 Packets sent with an octet count of 65127: 0
309 Packets sent with an octet count of 128-255: 0
310 Packets sent with an octet count of 256-511: 0
311 Packets sent with an octet count of 512-1023: 0
312 Packets sent with an octet count of 1024-1518: 0
313 Packets sent with an octet count of > 1518: 0
314 Packets sent to a broadcast DMAC: 0
315 Packets sent to the multicast DMAC: 0
316 Transmit underflow and were truncated: 0
317 Control/PAUSE packets sent: 0
321 .. code-block:: console
323 cat /sys/kernel/debug/octeontx2/cpt/cpt_pc
325 CPT instruction requests 0
326 CPT instruction latency 0
327 CPT NCB read requests 0
328 CPT NCB read latency 0
329 CPT read requests caused by UC fills 0
330 CPT active cycles pc 1395642
331 CPT clock count pc 5579867595493
335 .. code-block:: console
337 Usage: echo <nixlf> [cq number/all] > /sys/kernel/debug/octeontx2/nix/cq_ctx
338 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
339 echo 0 0 > /sys/kernel/debug/octeontx2/nix/cq_ctx
340 cat /sys/kernel/debug/octeontx2/nix/cq_ctx
342 =====cq_ctx for nixlf:0 and qidx:0 is=====
353 W2: update_time 31043
370 .. code-block:: console
372 Usage: echo <npalf> [pool number/all] > /sys/kernel/debug/octeontx2/npa/pool_ctx
373 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
374 echo 0 0 > /sys/kernel/debug/octeontx2/npa/pool_ctx
375 cat /sys/kernel/debug/octeontx2/npa/pool_ctx
377 ======POOL : 0=======
378 W0: Stack base 1375bff00
385 W2: stack_max_pages 24315
386 W2: stack_pages 24314
396 W4: update_time 62993
398 W6: ptr_start 1593adf00
399 W7: ptr_end 180000000
405 W8: thresh_qint_idx 0
410 .. code-block:: console
412 cat /sys/kernel/debug/octeontx2/npc/mcam_info
415 RX keywidth : 224bits
416 TX keywidth : 224bits
428 .. code-block:: console
430 Usage: echo [<hws>/all] > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
431 echo 0 > /sys/kernel/debug/octeontx2/sso/hws/sso_hws_info
433 ==================================================
434 SSOW HWS[0] Arbitration State 0x0
435 SSOW HWS[0] Guest Machine Control 0x0
436 SSOW HWS[0] SET[0] Group Mask[0] 0xffffffffffffffff
437 SSOW HWS[0] SET[0] Group Mask[1] 0xffffffffffffffff
438 SSOW HWS[0] SET[0] Group Mask[2] 0xffffffffffffffff
439 SSOW HWS[0] SET[0] Group Mask[3] 0xffffffffffffffff
440 SSOW HWS[0] SET[1] Group Mask[0] 0xffffffffffffffff
441 SSOW HWS[0] SET[1] Group Mask[1] 0xffffffffffffffff
442 SSOW HWS[0] SET[1] Group Mask[2] 0xffffffffffffffff
443 SSOW HWS[0] SET[1] Group Mask[3] 0xffffffffffffffff
444 ==================================================
449 DPDK may be compiled either natively on OCTEON TX2 platform or cross-compiled on
450 an x86 based platform.
458 .. code-block:: console
460 make config T=arm64-octeontx2-linux-gcc
463 The example applications can be compiled using the following:
465 .. code-block:: console
469 export RTE_TARGET=build
470 cd examples/<application>
476 .. code-block:: console
484 Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` for generic arm64 details.
489 .. code-block:: console
491 make config T=arm64-octeontx2-linux-gcc
492 make -j CROSS=aarch64-marvell-linux-gnu- CONFIG_RTE_KNI_KMOD=n
497 .. code-block:: console
499 meson build --cross-file config/arm/arm64_octeontx2_linux_gcc
504 By default, meson cross compilation uses ``aarch64-linux-gnu-gcc`` toolchain,
505 if Marvell toolchain is available then it can be used by overriding the
506 c, cpp, ar, strip ``binaries`` attributes to respective Marvell
507 toolchain binaries in ``config/arm/arm64_octeontx2_linux_gcc`` file.