1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Intel Corporation.
4 .. include:: <isonum.txt>
6 IOAT Rawdev Driver for Intel\ |reg| QuickData Technology
7 ======================================================================
9 The ``ioat`` rawdev driver provides a poll-mode driver (PMD) for Intel\ |reg|
10 QuickData Technology, part of Intel\ |reg| I/O Acceleration Technology
12 <https://www.intel.com/content/www/us/en/wireless-network/accel-technology.html>`_.
13 This PMD, when used on supported hardware, allows data copies, for example,
14 cloning packet data, to be accelerated by that hardware rather than having to
15 be done by software, freeing up CPU cycles for other tasks.
18 ----------------------
20 On Linux, the presence of an Intel\ |reg| QuickData Technology hardware can
21 be detected by checking the output of the ``lspci`` command, where the
22 hardware will be often listed as "Crystal Beach DMA" or "CBDMA". For
23 example, on a system with Intel\ |reg| Xeon\ |reg| CPU E5-2699 v4 @2.20GHz,
26 .. code-block:: console
29 00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01)
30 00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01)
31 00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01)
32 00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01)
33 00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01)
34 00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01)
35 00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01)
36 00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01)
38 On a system with Intel\ |reg| Xeon\ |reg| Gold 6154 CPU @ 3.00GHz, lspci
41 .. code-block:: console
44 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
45 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
46 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
47 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
48 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
49 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
50 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
51 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
57 For builds done with ``make``, the driver compilation is enabled by the
58 ``CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV`` build configuration option. This is
59 enabled by default in builds for x86 platforms, and disabled in other
62 For builds using ``meson`` and ``ninja``, the driver will be built when the
63 target platform is x86-based.
68 The Intel\ |reg| QuickData Technology HW devices will need to be bound to a
69 user-space IO driver for use. The script ``dpdk-devbind.py`` script
70 included with DPDK can be used to view the state of the devices and to bind
71 them to a suitable DPDK-supported kernel driver. When querying the status
72 of the devices, they will appear under the category of "Misc (rawdev)
73 devices", i.e. the command ``dpdk-devbind.py --status-dev misc`` can be
74 used to see the state of those devices alone.
76 Device Probing and Initialization
77 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
79 Once bound to a suitable kernel device driver, the HW devices will be found
80 as part of the PCI scan done at application initialization time. No vdev
81 parameters need to be passed to create or initialize the device.
83 Once probed successfully, the device will appear as a ``rawdev``, that is a
84 "raw device type" inside DPDK, and can be accessed using APIs from the
85 ``rte_rawdev`` library.
87 Using IOAT Rawdev Devices
88 --------------------------
90 To use the devices from an application, the rawdev API can be used, along
91 with definitions taken from the device-specific header file
92 ``rte_ioat_rawdev.h``. This header is needed to get the definition of
93 structure parameters used by some of the rawdev APIs for IOAT rawdev
94 devices, as well as providing key functions for using the device for memory
97 Getting Device Information
98 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
100 Basic information about each rawdev device can be queried using the
101 ``rte_rawdev_info_get()`` API. For most applications, this API will be
102 needed to verify that the rawdev in question is of the expected type. For
103 example, the following code snippet can be used to identify an IOAT
104 rawdev device for use by an application:
108 for (i = 0; i < count && !found; i++) {
109 struct rte_rawdev_info info = { .dev_private = NULL };
110 found = (rte_rawdev_info_get(i, &info, 0) == 0 &&
111 strcmp(info.driver_name,
112 IOAT_PMD_RAWDEV_NAME_STR) == 0);
115 When calling the ``rte_rawdev_info_get()`` API for an IOAT rawdev device,
116 the ``dev_private`` field in the ``rte_rawdev_info`` struct should either
117 be NULL, or else be set to point to a structure of type
118 ``rte_ioat_rawdev_config``, in which case the size of the configured device
119 input ring will be returned in that structure.
122 ~~~~~~~~~~~~~~~~~~~~~
124 Configuring an IOAT rawdev device is done using the
125 ``rte_rawdev_configure()`` API, which takes the same structure parameters
126 as the, previously referenced, ``rte_rawdev_info_get()`` API. The main
127 difference is that, because the parameter is used as input rather than
128 output, the ``dev_private`` structure element cannot be NULL, and must
129 point to a valid ``rte_ioat_rawdev_config`` structure, containing the ring
130 size to be used by the device. The ring size must be a power of two,
132 If it is not needed, the tracking by the driver of user-provided completion
133 handles may be disabled by setting the ``hdls_disable`` flag in
134 the configuration structure also.
136 The following code shows how the device is configured in
137 ``test_ioat_rawdev.c``:
141 #define IOAT_TEST_RINGSIZE 512
142 struct rte_ioat_rawdev_config p = { .ring_size = -1 };
143 struct rte_rawdev_info info = { .dev_private = &p };
147 p.ring_size = IOAT_TEST_RINGSIZE;
148 if (rte_rawdev_configure(dev_id, &info, sizeof(p)) != 0) {
149 printf("Error with rte_rawdev_configure()\n");
153 Once configured, the device can then be made ready for use by calling the
154 ``rte_rawdev_start()`` API.
156 Performing Data Copies
157 ~~~~~~~~~~~~~~~~~~~~~~~
159 To perform data copies using IOAT rawdev devices, the functions
160 ``rte_ioat_enqueue_copy()`` and ``rte_ioat_do_copies()`` should be used.
161 Once copies have been completed, the completion will be reported back when
162 the application calls ``rte_ioat_completed_copies()``.
164 The ``rte_ioat_enqueue_copy()`` function enqueues a single copy to the
165 device ring for copying at a later point. The parameters to that function
166 include the IOVA addresses of both the source and destination buffers,
167 as well as two "handles" to be returned to the user when the copy is
168 completed. These handles can be arbitrary values, but two are provided so
169 that the library can track handles for both source and destination on
170 behalf of the user, e.g. virtual addresses for the buffers, or mbuf
171 pointers if packet data is being copied.
173 While the ``rte_ioat_enqueue_copy()`` function enqueues a copy operation on
174 the device ring, the copy will not actually be performed until after the
175 application calls the ``rte_ioat_do_copies()`` function. This function
176 informs the device hardware of the elements enqueued on the ring, and the
177 device will begin to process them. It is expected that, for efficiency
178 reasons, a burst of operations will be enqueued to the device via multiple
179 enqueue calls between calls to the ``rte_ioat_do_copies()`` function.
181 The following code from ``test_ioat_rawdev.c`` demonstrates how to enqueue
182 a burst of copies to the device and start the hardware processing of them:
186 struct rte_mbuf *srcs[32], *dsts[32];
189 for (i = 0; i < RTE_DIM(srcs); i++) {
192 srcs[i] = rte_pktmbuf_alloc(pool);
193 dsts[i] = rte_pktmbuf_alloc(pool);
194 srcs[i]->data_len = srcs[i]->pkt_len = length;
195 dsts[i]->data_len = dsts[i]->pkt_len = length;
196 src_data = rte_pktmbuf_mtod(srcs[i], char *);
198 for (j = 0; j < length; j++)
199 src_data[j] = rand() & 0xFF;
201 if (rte_ioat_enqueue_copy(dev_id,
202 srcs[i]->buf_iova + srcs[i]->data_off,
203 dsts[i]->buf_iova + dsts[i]->data_off,
207 0 /* nofence */) != 1) {
208 printf("Error with rte_ioat_enqueue_copy for buffer %u\n",
213 rte_ioat_do_copies(dev_id);
215 To retrieve information about completed copies, the API
216 ``rte_ioat_completed_copies()`` should be used. This API will return to the
217 application a set of completion handles passed in when the relevant copies
220 The following code from ``test_ioat_rawdev.c`` shows the test code
221 retrieving information about the completed copies and validating the data
222 is correct before freeing the data buffers using the returned handles:
226 if (rte_ioat_completed_copies(dev_id, 64, (void *)completed_src,
227 (void *)completed_dst) != RTE_DIM(srcs)) {
228 printf("Error with rte_ioat_completed_copies\n");
231 for (i = 0; i < RTE_DIM(srcs); i++) {
232 char *src_data, *dst_data;
234 if (completed_src[i] != srcs[i]) {
235 printf("Error with source pointer %u\n", i);
238 if (completed_dst[i] != dsts[i]) {
239 printf("Error with dest pointer %u\n", i);
243 src_data = rte_pktmbuf_mtod(srcs[i], char *);
244 dst_data = rte_pktmbuf_mtod(dsts[i], char *);
245 for (j = 0; j < length; j++)
246 if (src_data[j] != dst_data[j]) {
247 printf("Error with copy of packet %u, byte %u\n",
251 rte_pktmbuf_free(srcs[i]);
252 rte_pktmbuf_free(dsts[i]);
256 Querying Device Statistics
257 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
259 The statistics from the IOAT rawdev device can be got via the xstats
260 functions in the ``rte_rawdev`` library, i.e.
261 ``rte_rawdev_xstats_names_get()``, ``rte_rawdev_xstats_get()`` and
262 ``rte_rawdev_xstats_by_name_get``. The statistics returned for each device
265 * ``failed_enqueues``
266 * ``successful_enqueues``
268 * ``copies_completed``