-.. BSD LICENSE
- Copyright (C) Cavium networks Ltd. 2016.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Cavium networks nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2016 Cavium, Inc
ThunderX NICVF Poll Mode Driver
===============================
support for the inbuilt NIC found in the **Cavium ThunderX** SoC family
as well as their virtual functions (VF) in SR-IOV context.
-More information can be found at `Cavium Networks Official Website
+More information can be found at `Cavium, Inc Official Website
<http://www.cavium.com/ThunderX_ARM_Processors.html>`_.
Features
- Port hardware statistics
- Jumbo frames
- Link state information
+- Setting up link state.
- Scattered and gather for TX and RX
- VLAN stripping
- SR-IOV VF
- NUMA support
+- Multi queue set support (up to 96 queues (12 queue sets)) per port
+- Skip data bytes
Supported ThunderX SoCs
-----------------------
- CN88xx
- CN81xx
+- CN83xx
Prerequisites
-------------
The following options can be modified in the ``config`` file.
Please note that enabling debugging options may affect system performance.
-- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD`` (default ``n``)
+- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD`` (default ``y``)
- By default it is enabled only for defconfig_arm64-thunderx-* config.
Toggle compilation of the ``librte_pmd_thunderx_nicvf`` driver.
-- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_INIT`` (default ``n``)
-
- Toggle display of initialization related messages.
-
- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_RX`` (default ``n``)
- Toggle display of receive fast path run-time message
+ Toggle asserts of receive fast path.
- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_TX`` (default ``n``)
- Toggle display of transmit fast path run-time message
+ Toggle asserts of transmit fast path.
-- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER`` (default ``n``)
+Driver compilation and testing
+------------------------------
- Toggle display of generic debugging messages
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
-- ``CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX`` (default ``n``)
+To compile the ThunderX NICVF PMD for Linux arm64 gcc,
+use arm64-thunderx-linux-gcc as target.
- Toggle display of PF mailbox related run-time check messages
+Linux
+-----
-Driver Compilation
-~~~~~~~~~~~~~~~~~~
+SR-IOV: Prerequisites and sample Application Notes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To compile the ThunderX NICVF PMD for Linux arm64 gcc target, run the
-following “make” command:
+Current ThunderX NIC PF/VF kernel modules maps each physical Ethernet port
+automatically to virtual function (VF) and presented them as PCIe-like SR-IOV device.
+This section provides instructions to configure SR-IOV with Linux OS.
-.. code-block:: console
+#. Verify PF devices capabilities using ``lspci``:
- cd <DPDK-source-directory>
- make config T=arm64-thunderx-linuxapp-gcc install
+ .. code-block:: console
-Linux
------
+ lspci -vvv
-.. _thunderx_testpmd_example:
+ Example output:
-Running testpmd
-~~~~~~~~~~~~~~~
+ .. code-block:: console
-This section demonstrates how to launch ``testpmd`` with ThunderX NIC VF device
-managed by ``librte_pmd_thunderx_nicvf`` in the Linux operating system.
+ 0002:01:00.0 Ethernet controller: Cavium Networks Device a01e (rev 01)
+ ...
+ Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
+ ...
+ Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)
+ ...
+ Kernel driver in use: thunder-nic
+ ...
-#. Load ``vfio-pci`` driver:
+ .. note::
- .. code-block:: console
+ Unless ``thunder-nic`` driver is in use make sure your kernel config includes ``CONFIG_THUNDER_NIC_PF`` setting.
- modprobe vfio-pci
+#. Verify VF devices capabilities and drivers using ``lspci``:
- .. _thunderx_vfio_noiommu:
+ .. code-block:: console
-#. Enable **VFIO-NOIOMMU** mode (optional):
+ lspci -vvv
+
+ Example output:
.. code-block:: console
- echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+ 0002:01:00.1 Ethernet controller: Cavium Networks Device 0011 (rev 01)
+ ...
+ Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
+ ...
+ Kernel driver in use: thunder-nicvf
+ ...
+
+ 0002:01:00.2 Ethernet controller: Cavium Networks Device 0011 (rev 01)
+ ...
+ Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
+ ...
+ Kernel driver in use: thunder-nicvf
+ ...
.. note::
- **VFIO-NOIOMMU** is required only when running in VM context and should not be enabled otherwise.
- See also :ref:`SR-IOV: Prerequisites and sample Application Notes <thunderx_sriov_example>`.
+ Unless ``thunder-nicvf`` driver is in use make sure your kernel config includes ``CONFIG_THUNDER_NIC_VF`` setting.
+
+#. Pass VF device to VM context (PCIe Passthrough):
-#. Bind the ThunderX NIC VF device to ``vfio-pci`` loaded in the previous step:
+ The VF devices may be passed through to the guest VM using qemu or
+ virt-manager or virsh etc.
- Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
+ Example qemu guest launch command:
.. code-block:: console
- ./tools/dpdk-devbind.py --bind vfio-pci 0002:01:00.2
+ sudo qemu-system-aarch64 -name vm1 \
+ -machine virt,gic_version=3,accel=kvm,usb=off \
+ -cpu host -m 4096 \
+ -smp 4,sockets=1,cores=8,threads=1 \
+ -nographic -nodefaults \
+ -kernel <kernel image> \
+ -append "root=/dev/vda console=ttyAMA0 rw hugepagesz=512M hugepages=3" \
+ -device vfio-pci,host=0002:01:00.1 \
+ -drive file=<rootfs.ext3>,if=none,id=disk1,format=raw \
+ -device virtio-blk-device,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
+ -netdev tap,id=net0,ifname=tap0,script=/etc/qemu-ifup_thunder \
+ -device virtio-net-device,netdev=net0 \
+ -serial stdio \
+ -mem-path /dev/hugepages
-#. Start ``testpmd`` with basic parameters:
+#. Enable **VFIO-NOIOMMU** mode (optional):
.. code-block:: console
- ./arm64-thunderx-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0002:01:00.2 \
- -- -i --disable-hw-vlan-filter --crc-strip --no-flush-rx \
- --port-topology=loop
+ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+
+ .. note::
+
+ **VFIO-NOIOMMU** is required only when running in VM context and should not be enabled otherwise.
+
+#. Running testpmd:
+
+ Follow instructions available in the document
+ :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+ to run testpmd.
Example output:
.. code-block:: console
+ ./arm64-thunderx-linux-gcc/app/testpmd -l 0-3 -n 4 -w 0002:01:00.2 \
+ -- -i --no-flush-rx \
+ --port-topology=loop
+
...
PMD: rte_nicvf_pmd_init(): librte_pmd_thunderx nicvf version 1.0
Done
testpmd>
-.. _thunderx_sriov_example:
+Multiple Queue Set per DPDK port configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-SR-IOV: Prerequisites and sample Application Notes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+There are two types of VFs:
-Current ThunderX NIC PF/VF kernel modules maps each physical Ethernet port
-automatically to virtual function (VF) and presented them as PCIe-like SR-IOV device.
-This section provides instructions to configure SR-IOV with Linux OS.
+- Primary VF
+- Secondary VF
-#. Verify PF devices capabilities using ``lspci``:
+Each port consists of a primary VF and n secondary VF(s). Each VF provides 8 Tx/Rx queues to a port.
+When a given port is configured to use more than 8 queues, it requires one (or more) secondary VF.
+Each secondary VF adds 8 additional queues to the queue set.
- .. code-block:: console
+During PMD driver initialization, the primary VF's are enumerated by checking the
+specific flag (see sqs message in DPDK boot log - sqs indicates secondary queue set).
+They are at the beginning of VF list (the remain ones are secondary VF's).
- lspci -vvv
+The primary VFs are used as master queue sets. Secondary VFs provide
+additional queue sets for primary ones. If a port is configured for more then
+8 queues than it will request for additional queues from secondary VFs.
- Example output:
+Secondary VFs cannot be shared between primary VFs.
- .. code-block:: console
-
- 0002:01:00.0 Ethernet controller: Cavium Networks Device a01e (rev 01)
- ...
- Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
- ...
- Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)
- ...
- Kernel driver in use: thunder-nic
- ...
+Primary VFs are present on the beginning of the 'Network devices using kernel
+driver' list, secondary VFs are on the remaining on the remaining part of the list.
.. note::
- Unless ``thunder-nic`` driver is in use make sure your kernel config includes ``CONFIG_THUNDER_NIC_PF`` setting.
-
-#. Verify VF devices capabilities and drivers using ``lspci``:
+ The VNIC driver in the multiqueue setup works differently than other drivers like `ixgbe`.
+ We need to bind separately each specific queue set device with the ``usertools/dpdk-devbind.py`` utility.
- .. code-block:: console
+ .. note::
- lspci -vvv
+ Depending on the hardware used, the kernel driver sets a threshold ``vf_id``. VFs that try to attached with an id below or equal to
+ this boundary are considered primary VFs. VFs that try to attach with an id above this boundary are considered secondary VFs.
- Example output:
+LBK HW Access
+~~~~~~~~~~~~~
- .. code-block:: console
+Loopback HW Unit (LBK) receives packets from NIC-RX and sends packets back to NIC-TX.
+The loopback block has N channels and contains data buffering that is shared across
+all channels. Four primary VFs are reserved as loopback ports.
- 0002:01:00.1 Ethernet controller: Cavium Networks Device 0011 (rev 01)
- ...
- Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
- ...
- Kernel driver in use: thunder-nicvf
- ...
+Example device binding
+~~~~~~~~~~~~~~~~~~~~~~
- 0002:01:00.2 Ethernet controller: Cavium Networks Device 0011 (rev 01)
- ...
- Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
- ...
- Kernel driver in use: thunder-nicvf
- ...
+If a system has three interfaces, a total of 18 VF devices will be created
+on a non-NUMA machine.
.. note::
- Unless ``thunder-nicvf`` driver is in use make sure your kernel config includes ``CONFIG_THUNDER_NIC_VF`` setting.
-
-#. Verify PF/VF bind using ``dpdk-devbind.py``:
+ NUMA systems have 12 VFs per port and non-NUMA 6 VFs per port.
.. code-block:: console
- ./tools/dpdk-devbind.py --status
+ # usertools/dpdk-devbind.py --status
+
+ Network devices using DPDK-compatible driver
+ ============================================
+ <none>
+
+ Network devices using kernel driver
+ ===================================
+ 0000:01:10.0 'THUNDERX BGX (Common Ethernet Interface) a026' if= drv=thunder-BGX unused=vfio-pci
+ 0000:01:10.1 'THUNDERX BGX (Common Ethernet Interface) a026' if= drv=thunder-BGX unused=vfio-pci
+ 0001:01:00.0 'THUNDERX Network Interface Controller a01e' if= drv=thunder-nic unused=vfio-pci
+ 0001:01:00.1 'Device a034' if=eth0 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.2 'Device a034' if=eth1 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.3 'Device a034' if=eth2 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.4 'Device a034' if=eth3 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.5 'Device a034' if=eth4 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.6 'Device a034' if=lbk0 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:00.7 'Device a034' if=lbk1 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.0 'Device a034' if=lbk2 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.1 'Device a034' if=lbk3 drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.2 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.3 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.4 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.5 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.6 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:01.7 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:02.0 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:02.1 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+ 0001:01:02.2 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
+
+ Other network devices
+ =====================
+ 0002:00:03.0 'Device a01f' unused=vfio-pci,uio_pci_generic
- Example output:
+ .. note::
- .. code-block:: console
+ Here total no of primary VFs = 5 (variable, depends on no of ethernet ports present) + 4 (fixed, loopback ports).
+ Ethernet ports are indicated as `if=eth0` while loopback ports as `if=lbk0`.
- ...
- 0002:01:00.0 'Device a01e' if= drv=thunder-nic unused=vfio-pci
- 0002:01:00.1 'Device 0011' if=eth0 drv=thunder-nicvf unused=vfio-pci
- 0002:01:00.2 'Device 0011' if=eth1 drv=thunder-nicvf unused=vfio-pci
- ...
+We want to bind two physical interfaces with 24 queues each device, we attach two primary VFs
+and four secondary VFs. In our example we choose two 10G interfaces eth1 (0002:01:00.2) and eth2 (0002:01:00.3).
+We will choose four secondary queue sets from the ending of the list (0001:01:01.2-0002:01:02.2).
-#. Load ``vfio-pci`` driver:
+
+#. Bind two primary VFs to the ``vfio-pci`` driver:
.. code-block:: console
- modprobe vfio-pci
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:00.2
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:00.3
-#. Bind VF devices to ``vfio-pci`` using ``dpdk-devbind.py``:
+#. Bind four primary VFs to the ``vfio-pci`` driver:
.. code-block:: console
- ./tools/dpdk-devbind.py --bind vfio-pci 0002:01:00.1
- ./tools/dpdk-devbind.py --bind vfio-pci 0002:01:00.2
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:01.7
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:02.0
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:02.1
+ usertools/dpdk-devbind.py -b vfio-pci 0002:01:02.2
-#. Verify VF bind using ``dpdk-devbind.py``:
+The nicvf thunderx driver will make use of attached secondary VFs automatically during the interface configuration stage.
- .. code-block:: console
+Thunder-nic VF's
+~~~~~~~~~~~~~~~~
- ./tools/dpdk-devbind.py --status
+Use sysfs to distinguish thunder-nic primary VFs and secondary VFs.
+ .. code-block:: console
- Example output:
+ ls -l /sys/bus/pci/drivers/thunder-nic/
+ total 0
+ drwxr-xr-x 2 root root 0 Jan 22 11:19 ./
+ drwxr-xr-x 86 root root 0 Jan 22 11:07 ../
+ lrwxrwxrwx 1 root root 0 Jan 22 11:19 0001:01:00.0 -> '../../../../devices/platform/soc@0/849000000000.pci/pci0001:00/0001:00:10.0/0001:01:00.0'/
.. code-block:: console
- ...
- 0002:01:00.1 'Device 0011' drv=vfio-pci unused=
- 0002:01:00.2 'Device 0011' drv=vfio-pci unused=
- ...
- 0002:01:00.0 'Device a01e' if= drv=thunder-nic unused=vfio-pci
- ...
+ cat /sys/bus/pci/drivers/thunder-nic/0001\:01\:00.0/sriov_sqs_assignment
+ 12
+ 0 0001:01:00.1 vfio-pci +: 12 13
+ 1 0001:01:00.2 thunder-nicvf -:
+ 2 0001:01:00.3 thunder-nicvf -:
+ 3 0001:01:00.4 thunder-nicvf -:
+ 4 0001:01:00.5 thunder-nicvf -:
+ 5 0001:01:00.6 thunder-nicvf -:
+ 6 0001:01:00.7 thunder-nicvf -:
+ 7 0001:01:01.0 thunder-nicvf -:
+ 8 0001:01:01.1 thunder-nicvf -:
+ 9 0001:01:01.2 thunder-nicvf -:
+ 10 0001:01:01.3 thunder-nicvf -:
+ 11 0001:01:01.4 thunder-nicvf -:
+ 12 0001:01:01.5 vfio-pci: 0
+ 13 0001:01:01.6 vfio-pci: 0
+ 14 0001:01:01.7 thunder-nicvf: 255
+ 15 0001:01:02.0 thunder-nicvf: 255
+ 16 0001:01:02.1 thunder-nicvf: 255
+ 17 0001:01:02.2 thunder-nicvf: 255
+ 18 0001:01:02.3 thunder-nicvf: 255
+ 19 0001:01:02.4 thunder-nicvf: 255
+ 20 0001:01:02.5 thunder-nicvf: 255
+ 21 0001:01:02.6 thunder-nicvf: 255
+ 22 0001:01:02.7 thunder-nicvf: 255
+ 23 0001:01:03.0 thunder-nicvf: 255
+ 24 0001:01:03.1 thunder-nicvf: 255
+ 25 0001:01:03.2 thunder-nicvf: 255
+ 26 0001:01:03.3 thunder-nicvf: 255
+ 27 0001:01:03.4 thunder-nicvf: 255
+ 28 0001:01:03.5 thunder-nicvf: 255
+ 29 0001:01:03.6 thunder-nicvf: 255
+ 30 0001:01:03.7 thunder-nicvf: 255
+ 31 0001:01:04.0 thunder-nicvf: 255
+
+Every column that ends with 'thunder-nicvf: number' can be used as secondary VF.
+In printout above all entres after '14 0001:01:01.7 thunder-nicvf: 255' can be used as secondary VF.
+
+Debugging Options
+-----------------
+
+EAL command option to change log level
+ .. code-block:: console
-#. Pass VF device to VM context (PCIe Passthrough):
+ --log-level=pmd.net.thunderx.driver:info
+ or
+ --log-level=pmd.net.thunderx.driver,7
- The VF devices may be passed through to the guest VM using qemu or
- virt-manager or virsh etc.
- ``librte_pmd_thunderx_nicvf`` or ``thunder-nicvf`` should be used to bind
- the VF devices in the guest VM in :ref:`VFIO-NOIOMMU <thunderx_vfio_noiommu>` mode.
+Module params
+--------------
- Example qemu guest launch command:
+skip_data_bytes
+~~~~~~~~~~~~~~~
+This feature is used to create a hole between HEADROOM and actual data. Size of hole is specified
+in bytes as module param("skip_data_bytes") to pmd.
+This scheme is useful when application would like to insert vlan header without disturbing HEADROOM.
+Example:
.. code-block:: console
- sudo qemu-system-aarch64 -name vm1 \
- -machine virt,gic_version=3,accel=kvm,usb=off \
- -cpu host -m 4096 \
- -smp 4,sockets=1,cores=8,threads=1 \
- -nographic -nodefaults \
- -kernel <kernel image> \
- -append "root=/dev/vda console=ttyAMA0 rw hugepagesz=512M hugepages=3" \
- -device vfio-pci,host=0002:01:00.1 \
- -drive file=<rootfs.ext3>,if=none,id=disk1,format=raw \
- -device virtio-blk-device,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
- -netdev tap,id=net0,ifname=tap0,script=/etc/qemu-ifup_thunder \
- -device virtio-net-device,netdev=net0 \
- -serial stdio \
- -mem-path /dev/huge
-
-#. Refer to section :ref:`Running testpmd <thunderx_testpmd_example>` for instruction
- how to launch ``testpmd`` application.
+ -w 0002:01:00.2,skip_data_bytes=8
Limitations
-----------
-CRC striping
-~~~~~~~~~~~~
+CRC stripping
+~~~~~~~~~~~~~
The ThunderX SoC family NICs strip the CRC for every packets coming into the
-host interface. So, CRC will be stripped even when the
-``rxmode.hw_strip_crc`` member is set to 0 in ``struct rte_eth_conf``.
+host interface irrespective of the offload configuration.
Maximum packet length
~~~~~~~~~~~~~~~~~~~~~
in scatter/gather mode. So, setting MTU will result with ``EINVAL`` when the
frame size does not fit in the maximum number of segments.
-Limited VFs
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+skip_data_bytes
+~~~~~~~~~~~~~~~
-The ThunderX SoC family NICs has 128VFs and each VF has 8/8 queues
-for RX/TX respectively. Current driver implementation has one to one mapping
-between physical port and VF hence only limited VFs can be used.
+Maximum limit of skip_data_bytes is 128 bytes and number of bytes should be multiple of 8.
git.droids-corp.org / dpdk.git / blobdiff