3 Copyright (c) 2015-2017 Atomic Rules LLC
6 Redistribution and use in source and binary forms, with or without
7 modification, are permitted provided that the following conditions
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in
14 the documentation and/or other materials provided with the
16 * Neither the name of Atomic Rules LLC nor the names of its
17 contributors may be used to endorse or promote products derived
18 from this software without specific prior written permission.
20 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
35 The ARK PMD is a DPDK poll-mode driver for the Atomic Rules Arkville
36 (ARK) family of devices.
38 More information can be found at the `Atomic Rules website
39 <http://atomicrules.com>`_.
44 The Atomic Rules Arkville product is DPDK and AXI compliant product
45 that marshals packets across a PCIe conduit between host DPDK mbufs and
48 The ARK PMD, and the spirit of the overall Arkville product,
49 has been to take the DPDK API/ABI as a fixed specification;
50 then implement much of the business logic in FPGA RTL circuits.
51 The approach of *working backwards* from the DPDK API/ABI and having
52 the GPP host software *dictate*, while the FPGA hardware *copes*,
53 results in significant performance gains over a naive implementation.
55 While this document describes the ARK PMD software, it is helpful to
56 understand what the FPGA hardware is and is not. The Arkville RTL
57 component provides a single PCIe Physical Function (PF) supporting
58 some number of RX/Ingress and TX/Egress Queues. The ARK PMD controls
59 the Arkville core through a dedicated opaque Core BAR (CBAR).
60 To allow users full freedom for their own FPGA application IP,
61 an independent FPGA Application BAR (ABAR) is provided.
63 One popular way to imagine Arkville's FPGA hardware aspect is as the
64 FPGA PCIe-facing side of a so-called Smart NIC. The Arkville core does
65 not contain any MACs, and is link-speed independent, as well as
66 agnostic to the number of physical ports the application chooses to
67 use. The ARK driver exposes the familiar PMD interface to allow packet
68 movement to and from mbufs across multiple queues.
70 However FPGA RTL applications could contain a universe of added
71 functionality that an Arkville RTL core does not provide or can
72 not anticipate. To allow for this expectation of user-defined
73 innovation, the ARK PMD provides a dynamic mechanism of adding
74 capabilities without having to modify the ARK PMD.
76 The ARK PMD is intended to support all instances of the Arkville
77 RTL Core, regardless of configuration, FPGA vendor, or target
78 board. While specific capabilities such as number of physical
79 hardware queue-pairs are negotiated; the driver is designed to
80 remain constant over a broad and extendable feature set.
82 Intentionally, Arkville by itself DOES NOT provide common NIC
83 capabilities such as offload or receive-side scaling (RSS).
84 These capabilities would be viewed as a gate-level "tax" on
85 Green-box FPGA applications that do not require such function.
86 Instead, they can be added as needed with essentially no
87 overhead to the FPGA Application.
89 The ARK PMD also supports optional user extensions, through dynamic linking.
90 The ARK PMD user extensions are a feature of Arkville’s DPDK
91 net/ark poll mode driver, allowing users to add their
92 own code to extend the net/ark functionality without
93 having to make source code changes to the driver. One motivation for
94 this capability is that while DPDK provides a rich set of functions
95 to interact with NIC-like capabilities (e.g. MAC addresses and statistics),
96 the Arkville RTL IP does not include a MAC. Users can supply their
97 own MAC or custom FPGA applications, which may require control from
98 the PMD. The user extension is the means providing the control
99 between the user's FPGA application and the existing DPDK features via
105 The ARK PMD supports device parameters that are used for packet
106 routing and for internal packet generation and packet checking. This
107 section describes the supported parameters. These features are
108 primarily used for diagnostics, testing, and performance verification
109 under the guidance of an Arkville specialist. The nominal use of
110 Arkville does not require any configuration using these parameters.
114 The Packet Director controls connectivity between Arkville's internal
115 hardware components. The features of the Pkt_dir are only used for
116 diagnostics and testing; it is not intended for nominal use. The full
117 set of features are not published at this level.
124 The packet generator parameter takes a file as its argument. The file
125 contains configuration parameters used internally for regression
126 testing and are not intended to be published at this level. The
127 packet generator is an internal Arkville hardware component.
130 Pkt_gen=./config/pg.conf
134 The packet checker parameter takes a file as its argument. The file
135 contains configuration parameters used internally for regression
136 testing and are not intended to be published at this level. The
137 packet checker is an internal Arkville hardware component.
140 Pkt_chkr=./config/pc.conf
146 Ingress RX and Egress TX operation is by the nominal DPDK API .
147 The driver supports single-port, multi-queue for both RX and TX.
149 Refer to ``ark_ethdev.h`` for the list of supported methods to
150 act upon RX and TX Queues.
152 Configuration Information
153 -------------------------
155 **DPDK Configuration Parameters**
157 The following configuration options are available for the ARK PMD:
159 * **CONFIG_RTE_LIBRTE_ARK_PMD** (default y): Enables or disables inclusion
160 of the ARK PMD driver in the DPDK compilation.
162 * **CONFIG_RTE_LIBRTE_ARK_PAD_TX** (default y): When enabled TX
163 packets are padded to 60 bytes to support downstream MACS.
165 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_RX** (default n): Enables or disables debug
166 logging and internal checking of RX ingress logic within the ARK PMD driver.
168 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TX** (default n): Enables or disables debug
169 logging and internal checking of TX egress logic within the ARK PMD driver.
171 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS** (default n): Enables or disables debug
172 logging of detailed packet and performance statistics gathered in
175 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE** (default n): Enables or disables debug
176 logging of detailed PMD events and status.
182 See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
183 instructions on how to build DPDK.
185 By default the ARK PMD library will be built into the DPDK library.
187 For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
188 documentation that comes with DPDK suite <linux_gsg>`.
190 Supported ARK RTL PCIe Instances
191 --------------------------------
193 ARK PMD supports the following Arkville RTL PCIe instances including:
195 * ``1d6c:100d`` - AR-ARKA-FX0 [Arkville 32B DPDK Data Mover]
196 * ``1d6c:100e`` - AR-ARKA-FX1 [Arkville 64B DPDK Data Mover]
198 Supported Operating Systems
199 ---------------------------
201 Any Linux distribution fulfilling the conditions described in ``System Requirements``
202 section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK
203 Release Notes*. ARM and PowerPC architectures are not supported at this time.
209 * Dynamic ARK PMD extensions
210 * Multiple receive and transmit queues
211 * Jumbo frames up to 9K
212 * Hardware Statistics
217 Features that may be part of, or become part of, the Arkville RTL IP that are
218 not currently supported or exposed by the ARK PMD include:
220 * PCIe SR-IOV Virtual Functions (VFs)
221 * Arkville's Packet Generator Control and Status
222 * Arkville's Packet Director Control and Status
223 * Arkville's Packet Checker Control and Status
224 * Arkville's Timebase Management
229 #. Prepare the system as recommended by DPDK suite. This includes environment
230 variables, hugepages configuration, tool-chains and configuration
232 #. Insert igb_uio kernel module using the command 'modprobe igb_uio'
234 #. Bind the intended ARK device to igb_uio module
236 At this point the system should be ready to run DPDK applications. Once the
237 application runs to completion, the ARK PMD can be detached from igb_uio if necessary.
242 Follow instructions available in the document
243 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to launch
244 **testpmd** with Atomic Rules ARK devices managed by librte_pmd_ark.
248 .. code-block:: console
251 EAL: PCI device 0000:01:00.0 on NUMA socket -1
252 EAL: probe driver: 1d6c:100e rte_ark_pmd
253 EAL: PCI memory mapped at 0x7f9b6c400000
254 PMD: eth_ark_dev_init(): Initializing 0:2:0.1
255 ARKP PMD CommitID: 378f3a67
256 Configuring Port 0 (socket 0)
257 Port 0: DC:3C:F6:00:00:01
258 Checking link statuses...
259 Port 0 Link Up - speed 100000 Mbps - full-duplex