1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Intel Corporation.
4 .. include:: <isonum.txt>
9 The ``ioat`` rawdev driver provides a poll-mode driver (PMD) for Intel\ |reg|
10 Data Streaming Accelerator `(Intel DSA)
11 <https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator>`_ and for Intel\ |reg|
12 QuickData Technology, part of Intel\ |reg| I/O Acceleration Technology
14 <https://www.intel.com/content/www/us/en/wireless-network/accel-technology.html>`_.
15 This PMD, when used on supported hardware, allows data copies, for example,
16 cloning packet data, to be accelerated by that hardware rather than having to
17 be done by software, freeing up CPU cycles for other tasks.
20 ----------------------
22 The ``dpdk-devbind.py`` script, included with DPDK,
23 can be used to show the presence of supported hardware.
24 Running ``dpdk-devbind.py --status-dev misc`` will show all the miscellaneous,
25 or rawdev-based devices on the system.
26 For Intel\ |reg| QuickData Technology devices, the hardware will be often listed as "Crystal Beach DMA",
28 For Intel\ |reg| DSA devices, they are currently (at time of writing) appearing as devices with type "0b25",
29 due to the absence of pci-id database entries for them at this point.
34 For builds using ``meson`` and ``ninja``, the driver will be built when the target platform is x86-based.
35 No additional compilation steps are necessary.
40 Depending on support provided by the PMD, HW devices can either use the kernel configured driver
41 or be bound to a user-space IO driver for use.
42 For example, Intel\ |reg| DSA devices can use the IDXD kernel driver or DPDK-supported drivers,
45 Intel\ |reg| DSA devices using idxd kernel driver
46 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
48 To use a Intel\ |reg| DSA device bound to the IDXD kernel driver, the device must first be configured.
49 The `accel-config <https://github.com/intel/idxd-config>`_ utility library can be used for configuration.
52 The device configuration can also be done by directly interacting with the sysfs nodes.
54 There are some mandatory configuration steps before being able to use a device with an application.
55 The internal engines, which do the copies or other operations,
56 and the work-queues, which are used by applications to assign work to the device,
57 need to be assigned to groups, and the various other configuration options,
58 such as priority or queue depth, need to be set for each queue.
60 To assign an engine to a group::
62 $ accel-config config-engine dsa0/engine0.0 --group-id=0
63 $ accel-config config-engine dsa0/engine0.1 --group-id=1
65 To assign work queues to groups for passing descriptors to the engines a similar accel-config command can be used.
66 However, the work queues also need to be configured depending on the use-case.
67 Some configuration options include:
69 * mode (Dedicated/Shared): Indicates whether a WQ may accept jobs from multiple queues simultaneously.
70 * priority: WQ priority between 1 and 15. Larger value means higher priority.
71 * wq-size: the size of the WQ. Sum of all WQ sizes must be less that the total-size defined by the device.
72 * type: WQ type (kernel/mdev/user). Determines how the device is presented.
73 * name: identifier given to the WQ.
75 Example configuration for a work queue::
77 $ accel-config config-wq dsa0/wq0.0 --group-id=0 \
78 --mode=dedicated --priority=10 --wq-size=8 \
79 --type=user --name=app1
81 Once the devices have been configured, they need to be enabled::
83 $ accel-config enable-device dsa0
84 $ accel-config enable-wq dsa0/wq0.0
86 Check the device configuration::
90 Devices using VFIO/UIO drivers
91 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93 The HW devices to be used will need to be bound to a user-space IO driver for use.
94 The ``dpdk-devbind.py`` script can be used to view the state of the devices
95 and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``.
98 $ dpdk-devbind.py -b vfio-pci 00:04.0 00:04.1
100 Device Probing and Initialization
101 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
103 For devices bound to a suitable DPDK-supported VFIO/UIO driver, the HW devices will
104 be found as part of the device scan done at application initialization time without
105 the need to pass parameters to the application.
107 If the device is bound to the IDXD kernel driver (and previously configured with sysfs),
108 then a specific work queue needs to be passed to the application via a vdev parameter.
109 This vdev parameter take the driver name and work queue name as parameters.
110 For example, to use work queue 0 on Intel\ |reg| DSA instance 0::
112 $ dpdk-test --no-pci --vdev=rawdev_idxd,wq=0.0
114 Once probed successfully, the device will appear as a ``rawdev``, that is a
115 "raw device type" inside DPDK, and can be accessed using APIs from the
116 ``rte_rawdev`` library.
118 Using IOAT Rawdev Devices
119 --------------------------
121 To use the devices from an application, the rawdev API can be used, along
122 with definitions taken from the device-specific header file
123 ``rte_ioat_rawdev.h``. This header is needed to get the definition of
124 structure parameters used by some of the rawdev APIs for IOAT rawdev
125 devices, as well as providing key functions for using the device for memory
128 Getting Device Information
129 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
131 Basic information about each rawdev device can be queried using the
132 ``rte_rawdev_info_get()`` API. For most applications, this API will be
133 needed to verify that the rawdev in question is of the expected type. For
134 example, the following code snippet can be used to identify an IOAT
135 rawdev device for use by an application:
139 for (i = 0; i < count && !found; i++) {
140 struct rte_rawdev_info info = { .dev_private = NULL };
141 found = (rte_rawdev_info_get(i, &info, 0) == 0 &&
142 strcmp(info.driver_name,
143 IOAT_PMD_RAWDEV_NAME_STR) == 0);
146 When calling the ``rte_rawdev_info_get()`` API for an IOAT rawdev device,
147 the ``dev_private`` field in the ``rte_rawdev_info`` struct should either
148 be NULL, or else be set to point to a structure of type
149 ``rte_ioat_rawdev_config``, in which case the size of the configured device
150 input ring will be returned in that structure.
153 ~~~~~~~~~~~~~~~~~~~~~
155 Configuring an IOAT rawdev device is done using the
156 ``rte_rawdev_configure()`` API, which takes the same structure parameters
157 as the, previously referenced, ``rte_rawdev_info_get()`` API. The main
158 difference is that, because the parameter is used as input rather than
159 output, the ``dev_private`` structure element cannot be NULL, and must
160 point to a valid ``rte_ioat_rawdev_config`` structure, containing the ring
161 size to be used by the device. The ring size must be a power of two,
163 If it is not needed, the tracking by the driver of user-provided completion
164 handles may be disabled by setting the ``hdls_disable`` flag in
165 the configuration structure also.
167 The following code shows how the device is configured in
168 ``test_ioat_rawdev.c``:
172 #define IOAT_TEST_RINGSIZE 512
173 struct rte_ioat_rawdev_config p = { .ring_size = -1 };
174 struct rte_rawdev_info info = { .dev_private = &p };
178 p.ring_size = IOAT_TEST_RINGSIZE;
179 if (rte_rawdev_configure(dev_id, &info, sizeof(p)) != 0) {
180 printf("Error with rte_rawdev_configure()\n");
184 Once configured, the device can then be made ready for use by calling the
185 ``rte_rawdev_start()`` API.
187 Performing Data Copies
188 ~~~~~~~~~~~~~~~~~~~~~~~
190 To perform data copies using IOAT rawdev devices, the functions
191 ``rte_ioat_enqueue_copy()`` and ``rte_ioat_perform_ops()`` should be used.
192 Once copies have been completed, the completion will be reported back when
193 the application calls ``rte_ioat_completed_ops()``.
195 The ``rte_ioat_enqueue_copy()`` function enqueues a single copy to the
196 device ring for copying at a later point. The parameters to that function
197 include the IOVA addresses of both the source and destination buffers,
198 as well as two "handles" to be returned to the user when the copy is
199 completed. These handles can be arbitrary values, but two are provided so
200 that the library can track handles for both source and destination on
201 behalf of the user, e.g. virtual addresses for the buffers, or mbuf
202 pointers if packet data is being copied.
204 While the ``rte_ioat_enqueue_copy()`` function enqueues a copy operation on
205 the device ring, the copy will not actually be performed until after the
206 application calls the ``rte_ioat_perform_ops()`` function. This function
207 informs the device hardware of the elements enqueued on the ring, and the
208 device will begin to process them. It is expected that, for efficiency
209 reasons, a burst of operations will be enqueued to the device via multiple
210 enqueue calls between calls to the ``rte_ioat_perform_ops()`` function.
212 The following code from ``test_ioat_rawdev.c`` demonstrates how to enqueue
213 a burst of copies to the device and start the hardware processing of them:
217 struct rte_mbuf *srcs[32], *dsts[32];
220 for (i = 0; i < RTE_DIM(srcs); i++) {
223 srcs[i] = rte_pktmbuf_alloc(pool);
224 dsts[i] = rte_pktmbuf_alloc(pool);
225 srcs[i]->data_len = srcs[i]->pkt_len = length;
226 dsts[i]->data_len = dsts[i]->pkt_len = length;
227 src_data = rte_pktmbuf_mtod(srcs[i], char *);
229 for (j = 0; j < length; j++)
230 src_data[j] = rand() & 0xFF;
232 if (rte_ioat_enqueue_copy(dev_id,
233 srcs[i]->buf_iova + srcs[i]->data_off,
234 dsts[i]->buf_iova + dsts[i]->data_off,
237 (uintptr_t)dsts[i]) != 1) {
238 printf("Error with rte_ioat_enqueue_copy for buffer %u\n",
243 rte_ioat_perform_ops(dev_id);
245 To retrieve information about completed copies, the API
246 ``rte_ioat_completed_ops()`` should be used. This API will return to the
247 application a set of completion handles passed in when the relevant copies
250 The following code from ``test_ioat_rawdev.c`` shows the test code
251 retrieving information about the completed copies and validating the data
252 is correct before freeing the data buffers using the returned handles:
256 if (rte_ioat_completed_ops(dev_id, 64, (void *)completed_src,
257 (void *)completed_dst) != RTE_DIM(srcs)) {
258 printf("Error with rte_ioat_completed_ops\n");
261 for (i = 0; i < RTE_DIM(srcs); i++) {
262 char *src_data, *dst_data;
264 if (completed_src[i] != srcs[i]) {
265 printf("Error with source pointer %u\n", i);
268 if (completed_dst[i] != dsts[i]) {
269 printf("Error with dest pointer %u\n", i);
273 src_data = rte_pktmbuf_mtod(srcs[i], char *);
274 dst_data = rte_pktmbuf_mtod(dsts[i], char *);
275 for (j = 0; j < length; j++)
276 if (src_data[j] != dst_data[j]) {
277 printf("Error with copy of packet %u, byte %u\n",
281 rte_pktmbuf_free(srcs[i]);
282 rte_pktmbuf_free(dsts[i]);
286 Querying Device Statistics
287 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
289 The statistics from the IOAT rawdev device can be got via the xstats
290 functions in the ``rte_rawdev`` library, i.e.
291 ``rte_rawdev_xstats_names_get()``, ``rte_rawdev_xstats_get()`` and
292 ``rte_rawdev_xstats_by_name_get``. The statistics returned for each device
295 * ``failed_enqueues``
296 * ``successful_enqueues``
298 * ``copies_completed``