1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2019 Intel Corporation
4 Intel(R) FPGA 5GNR FEC Poll Mode Driver
5 =======================================
7 The BBDEV FPGA 5GNR FEC poll mode driver (PMD) supports an FPGA implementation of a VRAN
8 LDPC Encode / Decode 5GNR wireless acceleration function, using Intel's PCI-e and FPGA
9 based Vista Creek device.
14 FPGA 5GNR FEC PMD supports the following features:
16 - LDPC Encode in the DL
17 - LDPC Decode in the UL
18 - 8 VFs per PF (physical device)
19 - Maximum of 32 UL queues per VF
20 - Maximum of 32 DL queues per VF
21 - PCIe Gen-3 x8 Interface
25 FPGA 5GNR FEC PMD supports the following BBDEV capabilities:
27 * For the LDPC encode operation:
28 - ``RTE_BBDEV_LDPC_CRC_24B_ATTACH`` : set to attach CRC24B to CB(s)
29 - ``RTE_BBDEV_LDPC_RATE_MATCH`` : if set then do not do Rate Match bypass
31 * For the LDPC decode operation:
32 - ``RTE_BBDEV_LDPC_CRC_TYPE_24B_CHECK`` : check CRC24B from CB(s)
33 - ``RTE_BBDEV_LDPC_ITERATION_STOP_ENABLE`` : disable early termination
34 - ``RTE_BBDEV_LDPC_CRC_TYPE_24B_DROP`` : drops CRC24B bits appended while decoding
35 - ``RTE_BBDEV_LDPC_HQ_COMBINE_IN_ENABLE`` : provides an input for HARQ combining
36 - ``RTE_BBDEV_LDPC_HQ_COMBINE_OUT_ENABLE`` : provides an input for HARQ combining
37 - ``RTE_BBDEV_LDPC_INTERNAL_HARQ_MEMORY_IN_ENABLE`` : HARQ memory input is internal
38 - ``RTE_BBDEV_LDPC_INTERNAL_HARQ_MEMORY_OUT_ENABLE`` : HARQ memory output is internal
39 - ``RTE_BBDEV_LDPC_INTERNAL_HARQ_MEMORY_LOOPBACK`` : loopback data to/from HARQ memory
40 - ``RTE_BBDEV_LDPC_INTERNAL_HARQ_MEMORY_FILLERS`` : HARQ memory includes the fillers bits
46 FPGA 5GNR FEC does not support the following:
48 - Scatter-Gather function
54 Section 3 of the DPDK manual provides instructions on installing and compiling DPDK. The
55 default set of bbdev compile flags may be found in config/common_base, where for example
56 the flag to build the FPGA 5GNR FEC device, ``CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_5GNR_FEC``,
59 DPDK requires hugepages to be configured as detailed in section 2 of the DPDK manual.
60 The bbdev test application has been tested with a configuration 40 x 1GB hugepages. The
61 hugepage configuration of a server may be examined using:
63 .. code-block:: console
65 grep Huge* /proc/meminfo
71 When the device first powers up, its PCI Physical Functions (PF) can be listed through this command:
73 .. code-block:: console
75 sudo lspci -vd8086:0d8f
77 The physical and virtual functions are compatible with Linux UIO drivers:
78 ``vfio`` and ``igb_uio``. However, in order to work the FPGA 5GNR FEC device firstly needs
79 to be bound to one of these linux drivers through DPDK.
85 Install the DPDK igb_uio driver, bind it with the PF PCI device ID and use
86 ``lspci`` to confirm the PF device is under use by ``igb_uio`` DPDK UIO driver.
88 The igb_uio driver may be bound to the PF PCI device using one of three methods:
91 1. PCI functions (physical or virtual, depending on the use case) can be bound to
92 the UIO driver by repeating this command for every function.
94 .. code-block:: console
96 cd <dpdk-top-level-directory>
97 insmod ./build/kmod/igb_uio.ko
98 echo "8086 0d8f" > /sys/bus/pci/drivers/igb_uio/new_id
102 2. Another way to bind PF with DPDK UIO driver is by using the ``dpdk-devbind.py`` tool
104 .. code-block:: console
106 cd <dpdk-top-level-directory>
107 ./usertools/dpdk-devbind.py -b igb_uio 0000:06:00.0
109 where the PCI device ID (example: 0000:06:00.0) is obtained using lspci -vd8086:0d8f
112 3. A third way to bind is to use ``dpdk-setup.sh`` tool
114 .. code-block:: console
116 cd <dpdk-top-level-directory>
117 ./usertools/dpdk-setup.sh
119 select 'Bind Ethernet/Crypto/Baseband device to IGB UIO module'
121 select 'Bind Ethernet/Crypto/Baseband device to VFIO module' depending on driver required
123 select 'Display current Ethernet/Crypto/Baseband device settings' to confirm binding
126 In the same way the FPGA 5GNR FEC PF can be bound with vfio, but vfio driver does not
127 support SR-IOV configuration right out of the box, so it will need to be patched.
130 Enable Virtual Functions
131 ~~~~~~~~~~~~~~~~~~~~~~~~
133 Now, it should be visible in the printouts that PCI PF is under igb_uio control
134 "``Kernel driver in use: igb_uio``"
136 To show the number of available VFs on the device, read ``sriov_totalvfs`` file..
138 .. code-block:: console
140 cat /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_totalvfs
142 where 0000\:<b>\:<d>.<f> is the PCI device ID
145 To enable VFs via igb_uio, echo the number of virtual functions intended to
146 enable to ``max_vfs`` file..
148 .. code-block:: console
150 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/max_vfs
153 Afterwards, all VFs must be bound to appropriate UIO drivers as required, same
154 way it was done with the physical function previously.
156 Enabling SR-IOV via vfio driver is pretty much the same, except that the file
159 .. code-block:: console
161 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_numvfs
164 Configure the VFs through PF
165 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
167 The PCI virtual functions must be configured before working or getting assigned
168 to VMs/Containers. The configuration involves allocating the number of hardware
169 queues, priorities, load balance, bandwidth and other settings necessary for the
170 device to perform FEC functions.
172 This configuration needs to be executed at least once after reboot or PCI FLR and can
173 be achieved by using the function ``fpga_5gnr_fec_configure()``, which sets up the
174 parameters defined in ``fpga_5gnr_fec_conf`` structure:
178 struct fpga_5gnr_fec_conf {
180 uint8_t vf_ul_queues_number[FPGA_5GNR_FEC_NUM_VFS];
181 uint8_t vf_dl_queues_number[FPGA_5GNR_FEC_NUM_VFS];
182 uint8_t ul_bandwidth;
183 uint8_t dl_bandwidth;
184 uint8_t ul_load_balance;
185 uint8_t dl_load_balance;
186 uint16_t flr_time_out;
189 - ``pf_mode_en``: identifies whether only PF is to be used, or the VFs. PF and
190 VFs are mutually exclusive and cannot run simultaneously.
191 Set to 1 for PF mode enabled.
192 If PF mode is enabled all queues available in the device are assigned
193 exclusively to PF and 0 queues given to VFs.
195 - ``vf_*l_queues_number``: defines the hardware queue mapping for every VF.
197 - ``*l_bandwidth``: in case of congestion on PCIe interface. The device
198 allocates different bandwidth to UL and DL. The weight is configured by this
199 setting. The unit of weight is 3 code blocks. For example, if the code block
200 cbps (code block per second) ratio between UL and DL is 12:1, then the
201 configuration value should be set to 36:3. The schedule algorithm is based
202 on code block regardless the length of each block.
204 - ``*l_load_balance``: hardware queues are load-balanced in a round-robin
205 fashion. Queues get filled first-in first-out until they reach a pre-defined
206 watermark level, if exceeded, they won't get assigned new code blocks..
207 This watermark is defined by this setting.
209 If all hardware queues exceeds the watermark, no code blocks will be
210 streamed in from UL/DL code block FIFO.
212 - ``flr_time_out``: specifies how many 16.384us to be FLR time out. The
213 time_out = flr_time_out x 16.384us. For instance, if you want to set 10ms for
214 the FLR time out then set this setting to 0x262=610.
217 An example configuration code calling the function ``fpga_5gnr_fec_configure()`` is shown
222 struct fpga_5gnr_fec_conf conf;
225 memset(&conf, 0, sizeof(struct fpga_5gnr_fec_conf));
228 for (i = 0; i < FPGA_5GNR_FEC_NUM_VFS; ++i) {
229 conf.vf_ul_queues_number[i] = 4;
230 conf.vf_dl_queues_number[i] = 4;
232 conf.ul_bandwidth = 12;
233 conf.dl_bandwidth = 5;
234 conf.dl_load_balance = 64;
235 conf.ul_load_balance = 64;
238 ret = fpga_5gnr_fec_configure(info->dev_name, &conf);
239 TEST_ASSERT_SUCCESS(ret,
240 "Failed to configure 4G FPGA PF for bbdev %s",
247 BBDEV provides a test application, ``test-bbdev.py`` and range of test data for testing
248 the functionality of FPGA 5GNR FEC encode and decode, depending on the device's
249 capabilities. The test application is located under app->test-bbdev folder and has the
252 .. code-block:: console
254 "-p", "--testapp-path": specifies path to the bbdev test app.
255 "-e", "--eal-params" : EAL arguments which are passed to the test app.
256 "-t", "--timeout" : Timeout in seconds (default=300).
257 "-c", "--test-cases" : Defines test cases to run. Run all if not specified.
258 "-v", "--test-vector" : Test vector path (default=dpdk_path+/app/test-bbdev/test_vectors/bbdev_null.data).
259 "-n", "--num-ops" : Number of operations to process on device (default=32).
260 "-b", "--burst-size" : Operations enqueue/dequeue burst size (default=32).
261 "-l", "--num-lcores" : Number of lcores to run (default=16).
262 "-i", "--init-device" : Initialise PF device with default values.
265 To execute the test application tool using simple decode or encode data,
266 type one of the following:
268 .. code-block:: console
270 ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_dec_default.data
271 ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_enc_default.data
274 The test application ``test-bbdev.py``, supports the ability to configure the PF device with
275 a default set of values, if the "-i" or "- -init-device" option is included. The default values
276 are defined in test_bbdev_perf.c as:
278 - VF_UL_QUEUE_VALUE 4
279 - VF_DL_QUEUE_VALUE 4
282 - UL_LOAD_BALANCE 128
283 - DL_LOAD_BALANCE 128
290 In addition to the simple LDPC decoder and LDPC encoder tests, bbdev also provides
291 a range of additional tests under the test_vectors folder, which may be useful. The results
292 of these tests will depend on the FPGA 5GNR FEC capabilities.