ethdev: add allocation helper for virtual drivers

This helper should be used by ethdev drivers supporting virtual devices
to help allocating a new ethdev and properly filling the default fields.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/null: internalize create function

There is no need to export this API. Remaining users should use the
rte_eal_vdev_init() function instead.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

test: do not short-circuit null device creation

A virtual device should get initialized through the rte_eal_vdev_init()
function to properly initialize the driver.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/nfp: use ethdev DMA helper

This driver can use the library function rte_eth_dma_zone_reserve()
instead of duplicating the code.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/bonding: call through EAL on create/free

To properly embed the generic rte_device into the rte_eth_dev this reworks
the bonding API to call through rte_eal_vdev_init().

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

eal: parse driver argument before probing drivers

In some cases the virtual device name should be totally different than
the driver being used for the device. Therefore lets parse the devargs for
the "driver" argument before probing drivers in vdev_probe_all_drivers().

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

eal: add name field to generic device

This adds a name field to the generic struct rte_device. The EAL is
checking for the name being populated when registering a device but
doesn't enforce global unique names as this is left to the bus
implementations.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

vdev: use generic vdev struct for probe and remove

This is a preparation to embed the generic rte_device into the rte_eth_dev
also for virtual devices.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>

vdev: make virtual bus use its device struct

This allows the virtual bus to be rescanned and probed by tracking the
creation of rte_vdev_device.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

vdev: add virtual device arguments helper function

This adds the rte_vdev_device_args() helper function to prepare for
changing the virtual drivers probe() functions take a rte_vdev_device
pointer instead of the name+args strings.

Signed-off-by: Jan Blunck <jblunck@infradead.org>

vdev: add virtual device name helper function

This adds the rte_vdev_device_name() helper function to retrieve the
rte_vdev_device name which makes moving the name of the low-level
device into struct rte_device easier in the future.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

vdev: add virtual device struct

This adds the rte_vdev_device structure which embeds a generic rte_device.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

vdev: refactor probe/remove

This is a preparation for the introduction of the struct rte_vdev_device.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

eal: remove unused device init function

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

vdev: move virtual device probing into a bus

This is a refactoring of the virtual device probing which moves into into
a proper bus structure.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>

eal: probe new virtual bus after other bus devices

Also see commit f4ce209a8ce5 ("eal: postpone vdev initialization").

Signed-off-by: Jan Blunck <jblunck@infradead.org>

eal: bump ABI version for bus refactoring

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

pci: use bus driver for scan/probe

Remove EAL initiated direct PCI scan/probe and enable PCI Bus linkage.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

pci: add bus driver

Based on EAL Bus APIs, PCI bus callbacks and support functions are
introduced in this patch.

EAL continues to have direct PCI init/scan calls as well. These would be
removed in subsequent patches to enable bus only PCI devices.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

pci: remove loop over drivers in device detach

rte_eal_pci_detach calls pci_detach_all_drivers which loops over all
PCI drivers for detaching the device. This is unnecessary as the device
already has the PCI driver reference which can be used directly.

Removing pci_detach_all_drivers and restructuring rte_eal_pci_detach
and rte_eal_pci_detach_dev to work without looping over driver list.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

pci: split match and probe

Matching of PCI device address and driver ID table is being done at two
discreet locations duplicating the code. (rte_eal_pci_probe_one_driver
and rte_eal_pci_detach_dev).

Refactor the match logic as a single function.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

test: remove PCI tests

These tests are not suited for the rte_bus PCI implementation anymore.

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/virtio-user: fix multi-process attach

Secondary process doesn't properly attach to the rte_eth_device
initialized by the primary process.

Accessing device from secondary process (e.g. via rte_eth_rx_burst),
causes process to crash. because rte_eth_dev_data is not properly set.

The issue was flood by
'commit 7f95f78a8aea ("ethdev: clear data when allocating device")'
which now clears rte_eth_dev_data entry.
For pci devices the struct is initialized by rte_eth_dev_pci_probe
->eth_dev_attach_secondary().
However, for virtio-user virtio_user_pmd_probe() is called instead of
rte_eth_dev_pci_probe().

The fix is to call rte_eth_dev_attach_secondary(), for secondary
process, from virtio_user_pmd_probe.

Fixes: 7f95f78a8aea ("ethdev: clear data when allocating device")
Cc: stable@dpdk.org
Signed-off-by: Ami Sabo <amis@radware.com>

ethdev: export secondary attach function

Today eth_dev_attach_secondary is defined as static and can only be
called by pci drivers. However, the functionality is also required for
non-pci drivers - so the patch export the function.

Signed-off-by: Ami Sabo <amis@radware.com>

ring: fix return value for single dequeue

The error return code for rte_ring_dequeue() function should be -ENOENT
rather than -ENOBUFS (which is the error value from the enqueue() fn).

Fixes: cfa7c9e6fc1f ("ring: make bulk and burst return values consistent")

Reported-by: Zhihong Wang <zhihong.wang@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

test: remove unnecessary cast of void pointer

Remove unnecessary casts of void * pointers to a specific type.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

lib: remove unnecessary void cast

Remove unnecessary casts of void * pointers to a specific type.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

app/testpmd: fix IPv6 tunnel checksum

When ipv6 packet is tunnel packet, "PKT_TX_OUTER_IPV6" flag must
be set, to let prepare the correct mbuf meta data for tx forward.

Fixes: 2b76648872c9 ("net/e1000: add Tx preparation")
Cc: stable@dpdk.org
Signed-off-by: Jeff Guo <jia.guo@intel.com>

lib: fix IPv6 tunnel checksum

When packet is flag of "PKT_TX_OUTER_IPV6", it also need to be
considered to be tunnel case, in order to calculate the correct
csum value.

Fixes: 2b76648872c9 ("net/e1000: add Tx preparation")
Cc: stable@dpdk.org
Signed-off-by: Jeff Guo <jia.guo@intel.com>

examples: enable HW CRC strip by default

Since VF can not disable/enable HW CRC strip for non-DPDK PF drivers,
and kernel driver almost default enable that feature, if disable it in
example app's rxmode, VF driver will report the VF launch failure. So
this patch default to enable HW CRC strip to let VF launch successful.

Cc: stable@dpdk.org
Signed-off-by: Jeff Guo <jia.guo@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

test: enable HW CRC strip by default

Since VF can not disable/enable HW CRC strip for non-DPDK PF drivers,
and kernel driver almost default enable that feature, if disable it in
test app's rxmode, VF driver will report the VF launch failure. So
this patch default to enable HW CRC strip to let VF launch successful.

Cc: stable@dpdk.org
Signed-off-by: Jeff Guo <jia.guo@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

app: enable HW CRC strip by default

Since VF can not disable/enable HW CRC strip for non-DPDK PF drivers,
and kernel driver almost default enable that feature, if disable it in
app's rxmode, VF driver will report the VF launch failure. So this
patch default to enable HW CRC strip to let VF launch successful.

Cc: stable@dpdk.org
Signed-off-by: Jeff Guo <jia.guo@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

mk: fix quoting for ARM mtune argument

The mtune argument needs an unquoted argument, otherwise the compiler
complains:

  arm-buildroot-linux-gnueabi-gcc.br_real: error:
    unrecognized argument in option ‘-mtune="cortex-a9"’

Fixes: 02a8686263de ("mk: introduce ARMv7 architecture")
Fixes: 4a7e4626975a ("mk: introduce NXP dpaa2 architecture based on armv8-a")
Cc: stable@dpdk.org
Reported-by: Jan Viktorin <viktorin@rehivetech.com>
Signed-off-by: Pascal Mazon <pascal.mazon@6wind.com>
Tested-by: Jan Viktorin <viktorin@rehivetech.com>

maintainers: rename section for szedata2 driver

The name of company is listed for other drivers.
Use the company name also for szedata2 driver.
Cards are available from Netcope rather than Cesnet.

Signed-off-by: Matej Vido <vido@cesnet.cz>

doc: update szedata2 guide

NICs are available from Netcope under the names NFB-*.
Change names and direct the links to Netcope web.

Signed-off-by: Matej Vido <vido@cesnet.cz>
Acked-by: John McNamara <john.mcnamara@intel.com>

app: do not build test-crypto-perf if cryptodev disabled

This fixes build in following configuration:

CONFIG_RTE_LIBRTE_CRYPTODEV=n
CONFIG_RTE_APP_CRYPTO_PERF=y

Fixes: f8be1786b1b8 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

latency: fix build without timer library

Remove the include to "rte_timer.h" which is not needed
by latencystats library (only "rte_cycles.h" is used).

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>

version: 17.05-rc1

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>

pci: do not log false failures for non-whitelisted devices

If probe of the whitelisted PCI device fails, reset ret to zero
to silently skip non-whitelisted PCI devices.

Fixes: 10f6c93cea38 ("eal: do not panic on PCI failures")

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>

eal: deprecate old cpu features check function

It's likely that this function isn't used anywhere, but since it was part
of the public API, mark the function for deprecation for at least one
release.

Signed-off-by: Aaron Conole <aconole@redhat.com>

eal: add missing function to symbol map

This function rte_cpu_is_supported is now part of the public ABI,
so should be advertised as such.

Fixes: 37e97ad2c56a ("eal: do not panic when CPU is not supported")

Signed-off-by: Aaron Conole <aconole@redhat.com>

doc: add maintainer role about replying questions

The first line of the MAINTAINERS file are:
"
The intention of this file is to provide a set of names that we can rely on
for helping in patch reviews and questions.
"

Unfortunately, some maintainers do not endorse their role for questions
asked on the mailing list.
Hope making it clear in the contribution guide will enforce more
responsive maintainer participation.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

doc: add how to use VF daemon

VFd (the VF daemon) is a mechanism which can be used to configure
features on a VF without direct access to the PF.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

doc: fix doc warning due to missing architecture

TILE-GX is no longer supported but the tap driver includes it in its
NIC overview ini file which causes a doc warning:

    $ make doc-guides-html
    sphinx processing guides-html...
    Warning generate_nic_overview_table():
    Unknown feature 'TILE-Gx' in 'tap.ini'

Fixes: 04b2c3c94346 ("doc: add tap features")

Signed-off-by: John McNamara <john.mcnamara@intel.com>

devtools: add test apps to build script

Since the test app is no longer being build by default as part of a build,
we need to update the testbuild script to take account of this.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>

config: enable crypto scheduler PMD by default

The crypto scheduler PMD has no external dependencies to enable that by
default.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

examples: enable build of performance-thread

The performance-thread example was not build by default in the make
examples build target. It will compile ok for x86_64 targets so add it to
the examples makefile list for that platform.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

devtools: make commits with stable tag outstanding

So that, as a stable maintainer while picking commits to a stable release,
I could pay less attention to those have it and pay more attention to those
don't have it.

Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>

eal: clean up interrupt handle

The patch change the prototype of callback function
(rte_intr_callback_fn) by removing the unnecessary parameter.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>

vfio: keep interrupt source read only

Remove the inappropriate modification on get_max_intr
field that keep the intr_source read only.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

drivers/net: remove unused DEPDIRS from makefiles

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/i40e: fix tunnel filter to VF

Failed to destroy tunnel filter rule if the action of
the tunnel filter is VF, root cause is the wrong vsi
used.

Fixes: c50474f31efe ("net/i40e: support tunnel filter to VF")

Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

event/sw: fix build for gcc 4.5.1

build error:
.../event/sw/sw_evdev_worker.c: In function ‘sw_event_release’:
.../event/sw/sw_evdev_worker.c:52:3: error: unknown field ‘op’ specified
in initializer

Fixed by updating struct initialization.

Fixes: 656af9180014 ("event/sw: add worker core functions")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

event/octeontx: fix build for gcc < 4.6

-Ofast option supported from gcc4.6 [1], for older versions using
"-O3 -ffast-math" instead.

[1] build error:
cc1: error: invalid option argument ‘-Ofast’

Also struct initialization build error [2] fixed.

[2] build error:
.../event/octeontx/ssovf_mbox.c: In function ‘mbox_send_request’:
.../event/octeontx/ssovf_mbox.c:95:9: error: unknown field ‘u64’
specified in initializer

Fixes: 32ff26393bb2 ("event/octeontx: add SSO HW device operations")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/thunderx: disable PMD for old compilers

Disable for gcc < 4.7 and icc <= 14.0

PMD uses some compiler builtins and new compiler options. Tested with
gcc 4.5.1 and following were not supported:

option:
-Ofast

macros:
_Static_assert

__ORDER_LITTLE_ENDIAN__
__ORDER_BIG_ENDIAN__
__BYTE_ORDER__

__atomic_fetch_add
__ATOMIC_ACQUIRE
__atomic_load_n
__ATOMIC_RELAXED
__atomic_store_n
__ATOMIC_RELEASE

It is not easy to fix all in PMD, disabling PMD for older compilers.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

crypto/scheduler: deprecate get/set mode functions

This patch deprecates the following functions in 17.05,
which will be removed in 17.08.
- rte_crpytodev_scheduler_mode_get()
- rte_crpytodev_scheduler_mode_set()

These two new functions replace them, fixing the typo in their names.
- rte_cryptodev_scheduler_mode_get()
- rte_cryptodev_scheduler_mode_set()

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/scheduler: add more options

This patch adds initial mode and ordering enable/disable EAL
options to cryptodev scheduler PMD.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/scheduler: fix session free

This patch fixes the incorrection slave session free operation.

Fixes: 57523e682bb7 ("crypto/scheduler: register operation functions")

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/qat: support device D15xx

Add support for Intel QuickAssist Technology device D15xx

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

doc: restructure QAT guide

Restructure QAT PMD instructions and add a device table to
minimise duplication for each device and make it easier to add devices.
Fix some device name typos and poor formatting.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

cryptodev: add API note

Add note to cryptodev API that chained mbufs
are not supported in DOCSISBPI mode.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

test/crypto: add ZUC test cases for QAT

This patch adds ZUC EEA3/EIA3 test cases for QAT crypto test suite

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: merge ZUC test vectors into one file

This patch merges ZUC test vectors into one header file

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: add ZUC EEA3/EIA3 capability

This commit adds ZUC EEA3 cipher and ZUC EIA3 MAC capability
to Intel(R) QuickAssist Technology driver

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: refactor capabilities infrastructure

Refactor capabilities data structures to facilitate
defining different capability sets for different devices
without duplication of data.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/scheduler: add API to get attached slaves

This patch adds an API to get the run-time slaves number and list
of a cryptodev scheduler PMD.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/qat: fix crash in session create

This patch fixes segementation fault that may occur in case
of wrong parameters being provided to the cryptographic
session. Unused fields which would cause null dereference
are removed.

Fixes: 1703e94ac5ce ("qat: add driver for QuickAssist devices")

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: add out-of-place tests

Added out-of-place (OOP) tests for AES-CBC, AES-DOCSIS and DES-DOCSIS
to tests suites for QuickAssist and openssl PMDs.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>

crypto/qat: support DOCSIS BPI mode

DOCSIS BPI mode is handled in the QAT PMD by sending full blocks to the
hardware device for encryption and using OpenSSL libcrypto for pre- or
post-processing of any partial blocks.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>

crypto/openssl: support DES DOCSIS BPI

Adds support in OpenSSL PMD for algorithm following the DOCSIS
specification, which combines DES-CBC for full DES blocks (8 bytes)
and DES-CFB for last runt block (less than 8 bytes).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
Tested-by: Yang Gang <gangx.yang@intel.com>

crypto/aesni_mb: support AES DOCSIS BPI

Underlying IPSec Multi buffer library implements
DOCSIS specification, so this commit adds support
for this new feature, which combines AES-CBC for full
AES blocks (16 bytes) and AES-CFB for last runt block
(less than 16 bytes).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/aesni_mb: support IPSec Multi-buffer lib v0.45

IPSec Multi-buffer library v0.45 has been released,
which includes, among other features, support for DOCSIS BPI
specification and include AVX512 optimizations.

This new version added const qualifiers to some of the function
prototypes, so the PMD has been updated to include these changes.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/aesni_mb: process crypto op on dequeue

Moving the crypto processing from the enqueue burst to the dequeue burst,
to remove the requirement to continually call the
rte_cryptodev_burst_enqueue function to guarantee that all operations get
flushed from the multi-buffer managers buffers.

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_gcm: do crypto op in dequeue function

There is bug when more crypto ops are enqueued than dequeued.
The return value is not checked when trying to enqueue the
processed crypto op into the internal ring, which in the case of being
full will results in crypto ops and mbufs being leaked.
The issue is more obvious with different cores doing enqueue/dequeue.

This patch moves the crypto operation to the dequeue function which
fixes the above issue without having to check for the number of free
entries in the ring.

Fixes: eec136f3c54f ("aesni_gcm: add driver for AES-GCM crypto operations")

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/scheduler: add fail-over scheduling mode

Fail-over mode works with 2 slaves, primary slave and secondary slave.
In this mode, the scheduler will enqueue the incoming crypto op burst
to the primary slave. When one or more crypto ops are failed to be
enqueued, they then will be enqueued to the secondary slave.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/scheduler: add packet size based mode

Packet-size based distribution mode is a scheduling mode works with 2
slaves, primary slave and secondary slave, and distribute the enqueued
crypto ops to them based on their data lengths. A crypto op will be
distributed to the primary slave if its data length equals or bigger
than the designated threshold, otherwise it will be handled by the
secondary slave.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

app/crypto-perf: reorg options structure

Reorganize options structure, to have all the parameters
used in a single run in the first bytes of the structure,
so all fit in a single cache line.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: add extra option checks

When using the verify test, test name is necessary
to be passed when digest is needed.

Also, when using an block cipher algorithm (CBC, ECB),
the buffer size has to be aligned to the block size.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: add range/list of sizes

So far, the crypto performance application was only able to
test one buffer size and one burst size.

With this commit, multiple sizes can be passed, either as a range
of values or as a list of values.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: display results in test runner

Instead of printing the test results in the destructor
function, print them just after each run.
This will be useful for after the commit that allows
a range of buffer and burst sizes.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: do not append digest if not used

When testing cipher only operations, there is no need to append digest
at the end of the buffer.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: move verify as single test type

In order to simplify throughput and latency tests,
verify option has been removed from these and moved
as a separate test.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

app/crypto-perf: fix AES CBC 128 test vectors

Ciphertext and digests for different buffer sizes
were incorrect.

Fixes: 5f1d85c5a8ab ("app/crypto-perf: add test vectors files")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: remove unused file

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/crypto-perf: remove cyclecount test type

Cyclecount test was not implemented, so it is removed.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

examples/ipsec-secgw: attach session-qp

adding support for attaching session to queue pairs.
This is required as underlying crypto driver may only
support limited number of sessions per queue pair
if max_nb_sessions_per_qp > 0, session should be
attached to a particular qp.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

cryptodev: add API to associate session with queue pair

HW based crypto drivers may only support limited number of
sessions per queue pair. This requires support for attaching
sessions to specific queue pair.  New APIs  are introduced to
attach/detach a session with/from a particular queue pair.
These are optional APIs.

Application can call attach API after creating a session
and can call detach API before deleting a session.

Application needs to check if max_nb_sessions_per_qp > 0,
then it should call the attach API.

max_nb_sessions_per_qp = 0 means infinite sessions per qp

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

examples/dpdk-qat: remove app

Remove DPDK QAT sample app, in favour of the newer applications
that use the cryptodev library: ipsec-gw and l2fwd-crypto,
which has support for Intel QuickAssist devices.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

crypto/scheduler: improve slave configuration

Since the new device configuration API is updated, we can make use of
this feature to the crypto scheduler PMD to configure its slaves
automatically with the same configurations it got. As originally the
slaves have to be manually configured one by one, this patch should
help reducing the coding complexity.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

cryptodev: change device configuration API

This patch changes the device configuration API for rte_cryptodev_ops
function prototype, and update all cryptodev PMDs for this change.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

doc: reformat crypto drivers overview

Follow the approach in the network devices overview,
for the feature matrix, so it improves readibility
and maintainability.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

doc: add missing supported crypto algos

AESNI MB supports MD5 and AES XCBC, but it wasn't reflected
in the driver documentation.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/scheduler: optimize crypto op ordering

This patch optimizes the crypto op ordering by replacing the
ordering method from using rte_reorder library to using rte_ring
to avoid unnecessary crypto op storing and recovering cost.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/scheduler: change enqueue and dequeue functions

This patch changes the enqueue and dequeue methods to cryptodev
scheduler PMD. Originally a 2-layer function call is carried out
upon enqueuing or dequeuing a burst of crypto ops. This patch
removes one layer to improve the performance.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

app/crypto-perf: fix uninitialized values for null ops

Some values are uninitialized for "cipher null" and "auth null"
operations. It may cause unpredictable results for some crypto pmd
drivers, or even segmentation fault.

This patch sets values for null operations to zero.

Fixes: f8be1786b1b8 ("app/crypto-perf: introduce performance test application")

Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

app/crypto-perf: avoid wrong operation type for AEAD algos

When somebody use bad --optype with aead algorithms
segmentation fault could happen.

Fixes: f8be1786b1b8 ("app/crypto-perf: introduce performance test application")

Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

app/crypto-perf: fix invalid latency for QAT

Fixes invalid latency result when using the performance application and
hardware QAT PMD. It occurred when the number of processed packets was
higher then the size of the internal QAT PMD ring buffer and the buffer
was overflowed.
Fixed by correcting the registration of the enqueued packets and freeing
memory space for not enqueued packets.

Fixes: f8be1786b1b8 ("app/crypto-perf: introduce performance test application")

Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: enable BPI for Cablelabs DOCSIS security spec

Extend the DPDK cryptodev API to enable processing of packets according
to the Baseline Privacy Interface Plus (BPI+) Specification described in
the security specification of the Cablelabs Data-over-Cable Service
Interface Specification (DOCSIS).

Brief summary of BPI+ symmetric cryptography requirements:
BPI+ cryptography uses a block cipher (AES-CBC/DES-CBC) to encrypt/decrypt
all the whole blocks in the packet. However the data length is not always
a block-multiple, so where there is a final block less than the full block
size this residual block requires special handling using AES-CFB/DES-CFB
mode. Similar special handling is specified where there is only one block,
smaller than the block size for the cipher. See spec for further details.
https://apps.cablelabs.com/specification/docsis-3-1-security-specification/

Two new elements are added to the enum rte_crypto_cipher_algorithm.
Note elements of this enum are actually a combination of an algorithm (AES,
3DES, etc) and mode (CBC, CTR, etc). The new DOCSISBPI mode is used to
convey to the PMD that the mode applied should be the specific combination
of CBC and CFB required by the DOCSIS Baseline Privacy Plus Spec.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>

examples/l2fwd-crypto: use cryptodev algorithm parser

L2fwd-crypto app was creating an array of strings for the
supported algorithms, which was different from the strings
that are now in cryptodev.

Use the new API in cryptodev to parse the string from the user,
to get the algorithm enum, instead, so it is not necessary to add
a new supported algorithm in the cryptodev library and this app.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

app/crypto-perf: use cryptodev algorithm parser

Instead of going through the array of supported algorithms
in the app, to get the algorithm enum, use the new API in
cryptodev to parse this string, so it is not necessary to add
a new supported algorithm in the cryptodev library and this app.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>