3 Copyright (c) 2015-2017 Atomic Rules LLC
6 Redistribution and use in source and binary forms, with or without
7 modification, are permitted provided that the following conditions
10 * Redistributions of source code must retain the above copyright
11 notice, this list of conditions and the following disclaimer.
12 * Redistributions in binary form must reproduce the above copyright
13 notice, this list of conditions and the following disclaimer in
14 the documentation and/or other materials provided with the
16 * Neither the name of Atomic Rules LLC nor the names of its
17 contributors may be used to endorse or promote products derived
18 from this software without specific prior written permission.
20 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
35 The ARK PMD is a DPDK poll-mode driver for the Atomic Rules Arkville
36 (ARK) family of devices.
38 More information can be found at the `Atomic Rules website
39 <http://atomicrules.com>`_.
44 The Atomic Rules Arkville product is DPDK and AXI compliant product
45 that marshals packets across a PCIe conduit between host DPDK mbufs and
48 The ARK PMD, and the spirit of the overall Arkville product,
49 has been to take the DPDK API/ABI as a fixed specification;
50 then implement much of the business logic in FPGA RTL circuits.
51 The approach of *working backwards* from the DPDK API/ABI and having
52 the GPP host software *dictate*, while the FPGA hardware *copes*,
53 results in significant performance gains over a naive implementation.
55 While this document describes the ARK PMD software, it is helpful to
56 understand what the FPGA hardware is and is not. The Arkville RTL
57 component provides a single PCIe Physical Function (PF) supporting
58 some number of RX/Ingress and TX/Egress Queues. The ARK PMD controls
59 the Arkville core through a dedicated opaque Core BAR (CBAR).
60 To allow users full freedom for their own FPGA application IP,
61 an independent FPGA Application BAR (ABAR) is provided.
63 One popular way to imagine Arkville's FPGA hardware aspect is as the
64 FPGA PCIe-facing side of a so-called Smart NIC. The Arkville core does
65 not contain any MACs, and is link-speed independent, as well as
66 agnostic to the number of physical ports the application chooses to
67 use. The ARK driver exposes the familiar PMD interface to allow packet
68 movement to and from mbufs across multiple queues.
70 However FPGA RTL applications could contain a universe of added
71 functionality that an Arkville RTL core does not provide or can
72 not anticipate. To allow for this expectation of user-defined
73 innovation, the ARK PMD provides a dynamic mechanism of adding
74 capabilities without having to modify the ARK PMD.
76 The ARK PMD is intended to support all instances of the Arkville
77 RTL Core, regardless of configuration, FPGA vendor, or target
78 board. While specific capabilities such as number of physical
79 hardware queue-pairs are negotiated; the driver is designed to
80 remain constant over a broad and extendable feature set.
82 Intentionally, Arkville by itself DOES NOT provide common NIC
83 capabilities such as offload or receive-side scaling (RSS).
84 These capabilities would be viewed as a gate-level "tax" on
85 Green-box FPGA applications that do not require such function.
86 Instead, they can be added as needed with essentially no
87 overhead to the FPGA Application.
89 The ARK PMD also supports optional user extensions, through dynamic linking.
90 The ARK PMD user extensions are a feature of Arkville’s DPDK
91 net/ark poll mode driver, allowing users to add their
92 own code to extend the net/ark functionality without
93 having to make source code changes to the driver. One motivation for
94 this capability is that while DPDK provides a rich set of functions
95 to interact with NIC-like capabilities (e.g. MAC addresses and statistics),
96 the Arkville RTL IP does not include a MAC. Users can supply their
97 own MAC or custom FPGA applications, which may require control from
98 the PMD. The user extension is the means providing the control
99 between the user's FPGA application and the existing DPDK features via
105 The ARK PMD supports device parameters that are used for packet
106 routing and for internal packet generation and packet checking. This
107 section describes the supported parameters. These features are
108 primarily used for diagnostics, testing, and performance verification
109 under the guidance of an Arkville specialist. The nominal use of
110 Arkville does not require any configuration using these parameters.
114 The Packet Director controls connectivity between Arkville's internal
115 hardware components. The features of the Pkt_dir are only used for
116 diagnostics and testing; it is not intended for nominal use. The full
117 set of features are not published at this level.
124 The packet generator parameter takes a file as its argument. The file
125 contains configuration parameters used internally for regression
126 testing and are not intended to be published at this level. The
127 packet generator is an internal Arkville hardware component.
130 Pkt_gen=./config/pg.conf
134 The packet checker parameter takes a file as its argument. The file
135 contains configuration parameters used internally for regression
136 testing and are not intended to be published at this level. The
137 packet checker is an internal Arkville hardware component.
140 Pkt_chkr=./config/pc.conf
146 Ingress RX and Egress TX operation is by the nominal DPDK API .
147 The driver supports single-port, multi-queue for both RX and TX.
149 Configuration Information
150 -------------------------
152 **DPDK Configuration Parameters**
154 The following configuration options are available for the ARK PMD:
156 * **CONFIG_RTE_LIBRTE_ARK_PMD** (default y): Enables or disables inclusion
157 of the ARK PMD driver in the DPDK compilation.
159 * **CONFIG_RTE_LIBRTE_ARK_PAD_TX** (default y): When enabled TX
160 packets are padded to 60 bytes to support downstream MACS.
162 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_RX** (default n): Enables or disables debug
163 logging and internal checking of RX ingress logic within the ARK PMD driver.
165 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TX** (default n): Enables or disables debug
166 logging and internal checking of TX egress logic within the ARK PMD driver.
168 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS** (default n): Enables or disables debug
169 logging of detailed packet and performance statistics gathered in
172 * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE** (default n): Enables or disables debug
173 logging of detailed PMD events and status.
179 See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
180 instructions on how to build DPDK.
182 By default the ARK PMD library will be built into the DPDK library.
184 For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
185 documentation that comes with DPDK suite <linux_gsg>`.
187 Supported ARK RTL PCIe Instances
188 --------------------------------
190 ARK PMD supports the following Arkville RTL PCIe instances including:
192 * ``1d6c:100d`` - AR-ARKA-FX0 [Arkville 32B DPDK Data Mover]
193 * ``1d6c:100e`` - AR-ARKA-FX1 [Arkville 64B DPDK Data Mover]
195 Supported Operating Systems
196 ---------------------------
198 Any Linux distribution fulfilling the conditions described in ``System Requirements``
199 section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK
200 Release Notes*. ARM and PowerPC architectures are not supported at this time.
206 * Dynamic ARK PMD extensions
207 * Multiple receive and transmit queues
208 * Jumbo frames up to 9K
209 * Hardware Statistics
214 Features that may be part of, or become part of, the Arkville RTL IP that are
215 not currently supported or exposed by the ARK PMD include:
217 * PCIe SR-IOV Virtual Functions (VFs)
218 * Arkville's Packet Generator Control and Status
219 * Arkville's Packet Director Control and Status
220 * Arkville's Packet Checker Control and Status
221 * Arkville's Timebase Management
226 #. Prepare the system as recommended by DPDK suite. This includes environment
227 variables, hugepages configuration, tool-chains and configuration
229 #. Insert igb_uio kernel module using the command 'modprobe igb_uio'
231 #. Bind the intended ARK device to igb_uio module
233 At this point the system should be ready to run DPDK applications. Once the
234 application runs to completion, the ARK PMD can be detached from igb_uio if necessary.
239 Follow instructions available in the document
240 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to launch
241 **testpmd** with Atomic Rules ARK devices managed by librte_pmd_ark.
245 .. code-block:: console
248 EAL: PCI device 0000:01:00.0 on NUMA socket -1
249 EAL: probe driver: 1d6c:100e rte_ark_pmd
250 EAL: PCI memory mapped at 0x7f9b6c400000
251 PMD: eth_ark_dev_init(): Initializing 0:2:0.1
252 ARKP PMD CommitID: 378f3a67
253 Configuring Port 0 (socket 0)
254 Port 0: DC:3C:F6:00:00:01
255 Checking link statuses...
256 Port 0 Link Up - speed 100000 Mbps - full-duplex