+Enable the VFs for each PF by echoing the number of VFs per PF to the pci driver::
+
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:1a:00.0/sriov_numvfs
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:3d:00.0/sriov_numvfs
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:3f:00.0/sriov_numvfs
+
+Check that the VFs are available for use. For example ``lspci -d:37c9`` should
+list 48 VF devices available for a ``C62x`` device.
+
+To complete the installation follow the instructions in
+`Binding the available VFs to the vfio-pci driver`_.
+
+.. Note::
+
+ If the QAT kernel modules are not loaded and you see an error like ``Failed
+ to load MMP firmware qat_895xcc_mmp.bin`` in kernel logs, this may be as a
+ result of not using a distribution, but just updating the kernel directly.
+
+ Download firmware from the `kernel firmware repo
+ <http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/>`_.
+
+ Copy qat binaries to ``/lib/firmware``::
+
+ cp qat_895xcc.bin /lib/firmware
+ cp qat_895xcc_mmp.bin /lib/firmware
+
+ Change to your linux source root directory and start the qat kernel modules::
+
+ insmod ./drivers/crypto/qat/qat_common/intel_qat.ko
+ insmod ./drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko
+
+.. Note::
+
+ If you see the following warning in ``/var/log/messages`` it can be ignored:
+ ``IOMMU should be enabled for SR-IOV to work correctly``.
+
+
+Installation using 01.org QAT driver
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Download the latest QuickAssist Technology Driver from `01.org
+<https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches>`_.
+Consult the *Getting Started Guide* at the same URL for further information.
+
+The steps below assume you are:
+
+* Building on a platform with one ``C62x`` device.
+* Using package ``qat1.7.l.4.2.0-000xx.tar.gz``.
+* On Fedora26 kernel ``4.11.11-300.fc26.x86_64``.
+
+In the BIOS ensure that SRIOV is enabled and VT-d is disabled.
+
+Uninstall any existing QAT driver, for example by running:
+
+* ``./installer.sh uninstall`` in the directory where originally installed.
+
+
+Build and install the SRIOV-enabled QAT driver::
+
+ mkdir /QAT
+ cd /QAT
+
+ # Copy the package to this location and unpack
+ tar zxof qat1.7.l.4.2.0-000xx.tar.gz
+
+ ./configure --enable-icp-sriov=host
+ make install
+
+You can use ``cat /sys/kernel/debug/qat<your device type and bdf>/version/fw`` to confirm the driver is correctly installed and is using firmware version 4.2.0.
+You can use ``lspci -d:37c9`` to confirm the presence of the 16 VF devices available per ``C62x`` PF.
+
+Confirm the driver is correctly installed and is using firmware version 4.2.0::
+
+ cat /sys/kernel/debug/qat<your device type and bdf>/version/fw
+
+
+Confirm the presence of 48 VF devices - 16 per PF::
+
+ lspci -d:37c9
+
+
+To complete the installation - follow instructions in
+`Binding the available VFs to the vfio-pci driver`_.
+
+.. Note::
+
+ If using a later kernel and the build fails with an error relating to
+ ``strict_stroul`` not being available apply the following patch:
+
+ .. code-block:: diff
+
+ /QAT/QAT1.6/quickassist/utilities/downloader/Target_CoreLibs/uclo/include/linux/uclo_platform.h
+ + #if LINUX_VERSION_CODE >= KERNEL_VERSION(3,18,5)
+ + #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (kstrtoul((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
+ + #else
+ #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,38)
+ #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (strict_strtoull((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
+ #else
+ #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,25)
+ #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; strict_strtoll((str), (base), (num));}
+ #else
+ #define STR_TO_64(str, base, num, endPtr) \
+ do { \
+ if (str[0] == '-') \
+ { \
+ *(num) = -(simple_strtoull((str+1), &(endPtr), (base))); \
+ }else { \
+ *(num) = simple_strtoull((str), &(endPtr), (base)); \
+ } \
+ } while(0)
+ + #endif
+ #endif
+ #endif
+
+
+.. Note::
+
+ If the build fails due to missing header files you may need to do following::
+
+ sudo yum install zlib-devel
+ sudo yum install openssl-devel
+ sudo yum install libudev-devel
+
+.. Note::
+
+ If the build or install fails due to mismatching kernel sources you may need to do the following::
+
+ sudo yum install kernel-headers-`uname -r`
+ sudo yum install kernel-src-`uname -r`
+ sudo yum install kernel-devel-`uname -r`
+
+
+Binding the available VFs to the vfio-pci driver
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Note:
+
+* Please note that due to security issues, the usage of older DPDK igb-uio
+ driver is not recommended. This document shows how to use the more secure
+ vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+ QATE-39220 and QATE-7495 issues in
+ `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+ which details the constraint about trusted guests and add `disable_denylist=1`
+ to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
+
+For an Intel(R) QuickAssist Technology DH895xCC device
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your
+VFs are different adjust the unbind command below::
+
+ cd to the top-level DPDK directory
+ for device in $(seq 1 4); do \
+ for fn in $(seq 0 7); do \
+ usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \
+ done; \
+ done
+
+For an Intel(R) QuickAssist Technology C62x device
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``,
+``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different
+adjust the unbind command below::
+
+ cd to the top-level DPDK directory
+ for device in $(seq 1 2); do \
+ for fn in $(seq 0 7); do \
+ usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \
+ usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \
+ usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \
+ done; \
+ done
+
+For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your
+VFs are different adjust the unbind command below::
+
+ cd to the top-level DPDK directory
+ for device in $(seq 1 2); do \
+ for fn in $(seq 0 7); do \
+ usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \
+ done; \
+ done
+
+Bind to the vfio-pci driver
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Load the vfio-pci driver, bind the VF PCI Device id to it using the
+``dpdk-devbind.py`` script then use the ``--status`` option
+to confirm the VF devices are now in use by vfio-pci kernel driver,
+e.g. for the C62x device::
+
+ cd to the top-level DPDK directory
+ modprobe vfio-pci
+ usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
+ usertools/dpdk-devbind.py --status