crypto/armv8: add PMD optimized for ARMv8 processors

This patch introduces crypto poll mode driver
using ARMv8 cryptographic extensions.
CPU compatibility with this driver is detected in
run-time and virtual crypto device will not be
created if CPU doesn't provide:
AES, SHA1, SHA2 and NEON.

This PMD is optimized to provide performance boost
for chained crypto operations processing,
such as encryption + HMAC generation,
decryption + HMAC validation. In particular,
cipher only or hash only operations are
not provided.

The driver currently supports AES-128-CBC
in combination with: SHA256 HMAC and SHA1 HMAC
and relies on the external armv8_crypto library:
https://github.com/caviumnetworks/armv8_crypto

Build ARMv8 crypto PMD if compiling for ARM64
and CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO option
is enable in the configuration file.
ARMV8_CRYPTO_LIB_PATH environment variable will
point to the appropriate library directory.

Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Reviewed-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

doc: add ZUC to crypto matrices

When ZUC PMD was added, it was not added in the
Crypto Device Supported Functionality Matrices.
This commit adds a column in all the matrices, plus
the ZUC EEA3/EIA3 algorithms.

Fixes: cf7685d68f00 ("crypto/zuc: add driver for ZUC library")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_gcm: test new features

Added new unit tests for AES-NI GCM PMD to verify new functionalities.

Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/aesni_gcm: migrate from MB library to ISA-L

Current Cryptodev AES-NI GCM PMD is implemented using Multi Buffer
Crypto library.This patch reimplement the device using ISA-L Crypto
library: https://github.com/01org/isa-l_crypto.

The migration entailed the following additional support for:
  * GMAC algorithm.
  * 256-bit cipher key.
  * Session-less mode.
  * Out-of place processing
  * Scatter-gatter support for chained mbufs (only out-of place and
    destination mbuf must be contiguous)

Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

app/test: fix symmetric session free in crypto perf tests

This commit fixes problem with deallocation of symmetric
session entries in cryptodev performance tests.

Fixes: 390919829fdb ("app/test: update AES SHA performance test")
Fixes: 79521c438363 ("app/test: add AES GCM performance test")
Fixes: ffbe3be0d4b5 ("app/test: add libcrypto")
Fixes: 97fe6461c7cb ("app/test: add SNOW 3G performance test")

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

cryptodev: add user defined name for vdev

This patch adds a user defined name initializing parameter to cryptodev
library.

Originally, for software cryptodev PMD, the vdev name parameter is
treated as the driver identifier, and will create an unique name for each
device automatically, which is not necessarily as same as the vdev
parameter.

This patch allows the user to either create a unique name for his software
cryptodev, or by default, let the system creates a unique one. This should
help the user managing the created cryptodevs easily.

Examples:
CLI command fragment 1: --vdev "crypto_aesni_gcm_pmd"
The above command will result in creating a AESNI-GCM PMD with name of
"crypto_aesni_gcm_X", where postfix X is the number assigned by the system,
starting from 0. This fragment can be placed in the same CLI command
multiple times, resulting the postfixs incremented by one for each new
device.

CLI command fragment 2: --vdev "crypto_aesni_gcm_pmd,name=gcm1"
The above command will result in creating a AESNI-GCM PMD with name of
"gcm1". This fragment can be placed in the same CLI command multiple
times, as long as each having a unique name value.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/qat: add scatter gather to feature flags

This commit adds scatter gather option to Intel(R) QuickAssist
Technology driver feature flags.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

examples/ipsec-secgw: support SHA256 HMAC

Add minor adjustments to support SHA256 HMAC:
- extend maximum key length to match SHA256 HMAC
- add SHA256 HMAC parameters and configuration string
- add SHA256 HMAC to inbound and outbound cases

Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

app/test: check scatter-gather for crypto drivers

This patch provides unit tests for set of cipher/hash combinations covering
currently implemented crypto PMD's and allowing to verify scatter gather
support.

Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/openssl: support scatter-gather

Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto: support scatter-gather in software drivers

This patch introduces RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER feature flag
informing that selected crypto device supports segmented mbufs natively
and doesn't need to be coalesced before crypto operation.

While using segmented buffers in crypto devices may have unpredictable
results, for PMDs which doesn't support it natively, additional check is
made for debug compilation.

Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/openssl: fix indentation in guide

The code section was lacking indentation to be be correctly formatted.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for OpenSSL library")

Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: fix loop in device query

This patch fixes the dev value update problem in
rte_cryptodev_pmd_get_named_dev, orginally, dev won't be updated
after the initial step in the loop.

Fixes: d11b0f30df88 ("cryptodev: introduce API and framework for crypto devices")

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/openssl: remove unneeded check

EVP_CIPHER_CTX_set_padding() function always returns 1, so the check is
unneeded.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for OpenSSL library")

Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Tested-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

app/test: add integrity check for crypto mbuf data

In block cipher test cases, add checks that the source
and destination mbufs are not modified except where expected.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

crypto/qat: fix IV size in capabilities

This patch sets iv size in qat PMD to 12 bytes to be
conformant with nist SP800-38D.

Fixes: 26c2e4ad5ad4 ("cryptodev: add capabilities discovery")

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/aesni_gcm: fix IV size in capabilities

This patch sets iv size in aesni gcm PMD to 12 bytes to be
conformant with nist SP800-38D.

Fixes: eec136f3c54f ("aesni_gcm: add driver for AES-GCM crypto operations")

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>

crypto/aesni_gcm: fix J0 padding bytes

This commit fixes pre-counter block (J0) padding by clearing
four most significant bytes before setting initial counter value.

Fixes: b2bb3597470c ("crypto/aesni_gcm: move pre-counter block to driver")

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>

crypto/aesni_mb: support AVX512

Release v0.44 of Intel(R) Multi-Buffer Crypto for IPsec library adds
support for AVX512 instructions. This patch enables the new AVX512
accelerated functions from the aesni_mb_pmd crypto poll mode driver.

This patch set requires that the aesni_mb_pmd is linked against the
version 0.44 or greater of the Multi-Buffer Crypto for IPsec library.

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: add missing supported algos in guide

AESNI MB PMD supports SHA224-HMAC and SHA384-HMAC,
but the documentation was not updated with this.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

crypto/aesni_mb: add single operation functionality

Update driver to use new AESNI Multibuffer IPSec library single
operation functionality (cipher only and authentication only).
This patch also adds tests for this new feature.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

crypto/aesni_mb: update dependency in guide

The Intel(R) Multi Buffer Crypto library used in the AESNI MB PMD
has been moved to a new repository, in github.
This patch updates the link where it can be downloaded.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

crypto/aesni_mb: fix incorrect crypto session

When using sessionless crypto operations, crypto session
is obtained from a pool of sessions, when processing the
operation. Once the operation is processed, the session
is put back in the pool, but for the AESNI MB PMD, this
session was not being saved in the operation and therefore,
it did not return to the session pool.

Fixes: 924e84f87306 ("aesni_mb: add driver for multi buffer based crypto")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>

app/test: check SGL on QAT

This commit adds GCM tests to use within scatter-gather list.
Test use direct chained mbufs created based on the input parameter
for max size for in place operations and out of place operations.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: add SGL capability

This commit adds scatter-gather list capability to Intel QuickAssist
Technology driver.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

app/test: check DES on QAT

This commit adds tests of Data Encryption Standard (DES)
algorithm to Intel QuickAssist technology crypto test suites

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: add DES capability

This commit adds DES capability to Intel QuickAssist
Technology Driver

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

cryptodev: add DES CBC cipher algorithm

This commit adds DES CBC ciper algorithm to available algorithms

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/openssl: fix extra bytes written at end of data

Extra bytes are being written at end of data while process standard
openssl cipher encryption. This behaviour is unexpected.

This patch disable the padding feature in openssl library, which is
causing the problem.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for OpenSSL library")

Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/qat: fix buffer overwrite in OOP case

In out-of-place operation, data is DMAed from source mbuf
to destination mbuf. To avoid header data in dest mbuf being
overwritten, the minimal data-set should be DMAed.

Fixes: 39e0bee48e81 ("crypto/qat: rework request builder for performance")

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

cryptodev: fix crash on null dereference

crypodev->data->name will be null when
rte_cryptodev_get_dev_id() invoked without a valid
crypto device instance.

Fixes: d11b0f30df88 ("cryptodev: introduce API and framework for crypto devices")

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

app/test: check AES cipher-only on QAT

Extended functional AES-CBC and AES-CTR cipher-only
tests to run on QAT PMD.
Added AES_CBC cipher-only performance tests on QAT PMD.
No driver changes, but as now tested, QAT documentation
is updated to remove constraint.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

cryptodev: remove unused digest-appended feature

The cryptodev API had specified that if the digest address field was
left empty on an authentication operation, then the PMD would assume
the digest was appended to the source or destination data.
This case was not handled at all by most PMDs and incorrectly handled
by the QAT PMD.
As no bugs were raised, it is assumed to be not needed, so this patch
removes it, rather than add handling for the case on all PMDs.
The digest can still be appended to the data, but its
address must now be provided in the op.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

doc: add flow distributor example guide

Signed-off-by: Sameh Gobriel <sameh.gobriel@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

doc: add EFD library section in programmers guide

Signed-off-by: Sameh Gobriel <sameh.gobriel@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

examples/flow_distributor: new example to demonstrate EFD

This new sample app, based on the client/server sample app,
shows the user an scenario using the EFD library.
It consists of:

- A front-end server which has an EFD table that stores the
  node id for each flow key, which will distribute the incoming
  packets to the different nodes

- A back-end node, which has a hash table where node checks,
  after reading packets coming from the server, whether the packet
  is meant to be used in such node, in which case it will be TXed,
  or not, in which case, packet will be dropped.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

app/test: add EFD functional and perf tests

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Karla Saur <karla.saur@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

efd: add AVX2 vector lookup function

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

efd: new Elastic Flow Distributor library

Elastic Flow Distributor (EFD) is a distributor library that uses
perfect hashing to determine a target/value for a given incoming flow key.
It has the following advantages:

- First, because it uses perfect hashing, it does not store
  the key itself and hence lookup performance is not dependent
  on the key size.

- Second, the target/value can be any arbitrary value hence
  the system designer and/or operator can better optimize service rates
  and inter-cluster network traffic locating.

- Third, since the storage requirement is much smaller than a hash-based
  flow table (i.e. better fit for CPU cache), EFD can scale to
  millions of flow keys.
  Finally, with current optimized library implementation performance
  is fully scalable with number of CPU cores.

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Acked-by: Christian Maciocco <christian.maciocco@intel.com>

crypto/qat: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix portability
issues across different architectures.

CC: John Griffin <john.griffin@intel.com>
CC: Fiona Trahe <fiona.trahe@intel.com>
CC: Deepak Kumar Jain <deepak.k.jain@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/vmxnet3: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Yong Wang <yongwang@vmware.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/virtio: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>

net/thunderx: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix portability
issues across different architectures.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/ena: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

Suggested-by: Jan Medala <jan@semihalf.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Jan Medala <jan@semihalf.com>

net/qede: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Harish Patil <harish.patil@cavium.com>
CC: Rasesh Mody <rasesh.mody@cavium.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/bnxt: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal abstraction
for I/O device memory read/write access to fix portability issues across
different architectures.

CC: Stephen Hurd <stephen.hurd@broadcom.com>
CC: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/bnx2x: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal abstraction
for I/O device memory read/write access to fix portability issues across
different architectures.

CC: Harish Patil <harish.patil@cavium.com>
CC: Rasesh Mody <rasesh.mody@cavium.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/cxgbe: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/nfp: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Alejandro Lucero <alejandro.lucero@netronome.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Alejandro Lucero <alejandro.lucero@netronome.com>

net/enic: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix portability
issues across different architectures.

CC: John Daley <johndale@cisco.com>
CC: Nelson Escobar <neescoba@cisco.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/fm10k: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Jing Chen <jing.d.chen@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/i40e: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal abstraction
for I/O device memory read/write access to fix portability issues across
different architectures.

CC: Helin Zhang <helin.zhang@intel.com>
CC: Jingjing Wu <jingjing.wu@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Satha Rao <skoteshwar@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/ixgbe: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Helin Zhang <helin.zhang@intel.com>
CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

net/e1000: use I/O device memory read/write API

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

CC: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm64: change barrier definitions to macros

Change rte_*wb definitions to macros in order to
keep consistent with other barrier definitions in
the file.

Suggested-by: Jianbo Liu <jianbo.liu@linaro.org>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm64: override I/O device read/write access

Override the generic I/O device memory read/write access and implement it
using armv8 instructions for arm64.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal: let all architectures use generic I/O implementation

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal: add generic I/O device read/write implementation

This patch implements the generic version of rte_read[b/w/l/q]_[relaxed]
and rte_write[b/w/l/q]_[relaxed] using rte_io_wmb() and rte_io_rmb()

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal: introduce I/O device memory read/write operations

This commit introduces 8-bit, 16-bit, 32bit, 64bit I/O device
memory read/write operations along with the relaxed versions.

The weakly-ordered machine like ARM needs additional I/O barrier for
device memory read/write access over PCI bus.
By introducing the eal abstraction for I/O device memory read/write access,
The drivers can access I/O device memory in architecture agnostic manner.

The relaxed version does not have additional I/O memory barrier, useful in
accessing the device registers of integrated controllers which
implicitly strongly ordered with respect to memory access.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm64: define I/O device memory barriers

CC: Jianbo Liu <jianbo.liu@linaro.org>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm64: define SMP barrier

dmb instruction based barrier is used for smp version of memory barrier.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm64: fix memory barrier definition

dsb instruction based barrier is used for non smp
version of memory barrier.

Fixes: d708f01b7102 ("eal/arm: add atomic operations for ARMv8")
Cc: stable@dpdk.org
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Jianbo Liu <jianbo.liu@linaro.org>

eal/armv7: define I/O device memory barriers

The patch does not provide any functional change for ARMv7.
I/O barriers are mapped to existing smp barriers.

CC: Jan Viktorin <viktorin@rehivetech.com>
CC: Jianbo Liu <jianbo.liu@linaro.org>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/arm: separate SMP barrier definition for ARMv7 and ARMv8

Separate the smp barrier definition for arm and arm64 for fine
control on smp barrier definition for each architecture.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/ppc64: define I/O device memory barriers

The patch does not provide any functional change for ppc_64.
I/O barriers are mapped to existing smp barriers.

CC: Chao Zhu <chaozhu@linux.vnet.ibm.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/tile: define I/O device memory barriers

The patch does not provide any functional change for tile.
I/O barriers are mapped to existing smp barriers.

CC: Zhigang Lu <zlu@ezchip.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal/x86: define I/O device memory barriers

The patch does not provide any functional change for IA.
I/O barriers are mapped to existing smp barriers.

CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

eal: introduce I/O device memory barriers

This commit introduce rte_io_mb(), rte_io_wmb() and rte_io_rmb(), in
order to enable memory barriers between I/O device and CPU.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>

devtools: relax tag checking in fixes

The tag "Cc: stable@dpdk.org" must be set when the commit must be
backported to a stable branch. The reminder is reworded.

It should be located just below the "Fixes:" tag (without blank line)
and followed by a blank line, separated from SoB and review tags below.
However, there is no strong need for checking blank lines.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>

devtools: fix lookup commit fixing a fix of many commits

There was a bug when looking at a commit fixing a commit which
itself was fixing many commits:

% devtools/git-log-fixes.sh 12ee45a36~..12ee45a36
devtools/git-log-fixes.sh: 96: local: 5499c1fc9baa: bad variable name

In this case, the list of commits was not quoted in variable assignment.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>

config: enable nfp driver on Linux

Because using a NFP PMD requires specific BSP installed, the PMD
support was not the default option before. This was just for making
people aware of such dependency, since there is no need for such a
BSP for just compiling DPDK with NFP PMD support.

Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>

net/mlx5: support extended statistics

Implement extended statistics callbacks.

Suggested-by: Hanoch Haim <hhaim@cisco.com>
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Signed-off-by: Elad Persiko <eladpe@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>

doc: update release notes for i40e base driver

Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>

net/i40e: support Linux VF to configure IRQ link list

i40e PF host only support to work with DPDK VF driver, Linux
VF driver is not supported. This change will enhance in
configuring IRQ link list.

This Change will identify VF client by number of vector
requested. DPDK VF will ask only single one while Linux VF
will request at least 2. It will have different configuration
for different clients. DPDK VF will be configured to link all
queue together, while Linux VF will be configured per request.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: parse more VF parameter and configure

When VF requested to configure TX queue, a few parameters are
missed to be configured in PF host. This change have more
fields parsed and configured for TX context.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: return correct VSI id

PF host didn't return correct VSI id to VF.
This change fix it.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: change version number to support Linux VF

i40e PF host only support to work with DPDK VF driver, Linux
VF driver is not supported. This change will enhance in version
number returned.

Current version info returned won't be able to be recognized
by Linux VF driver, change to values that both DPDK VF and Linux
driver can recognize.

The expense is original DPDK host specific feature like
CFG_VLAN_PVID and CONFIG_VSI_QUEUES_EXT will not available.

DPDK VF also can't identify host driver by version number returned.
It always assume talking with Linux PF.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

doc: update for VFD experimental API

Update the doc and release note.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set/clear VF stats from PF

This patch add support to get/clear VF statistics
from PF side.
Two APIs are added:
rte_pmd_i40e_get_vf_stats.
rte_pmd_i40e_reset_vf_stats.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: enhance sanity check of MAC

When VF sends request to add a new MAC address, PF host
will check if it's a non-zero or unicast address, or it
will return with error. In fact, VF still can set multicast
address. This change remove to check if it's a unicast
address.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: handle i40e in VF VLAN filter command

modify set_vf_rx_vlan function to handle the i40e PMD.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: add command for VF VLAN tag on i40e

command is: set vf vlan tag port_id vf_id on|off

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: add command for VF broadcast mode on i40e

Add command to call rte_pmd_i40e_set_vf_broadcast.
Add set vf broadcast in testpmd_funcs.rst file.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: use multicast promiscuous mode on i40e

Add testpmd CLI to set VF multicast promiscuous mode on i40e.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: use unicast promiscuous mode on i40e

Add testpmd CLI to set VF unicast promiscuous mode on i40e.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

app/testpmd: use VFD APIs on i40e

The new VF Daemon (VFD) APIs is implemented on i40e. Change
testpmd code to use them, including VF MAC anti-spoofing,
VF VLAN anti-spoofing, TX loopback, VF VLAN strip, VF VLAN
insert.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF VLAN filter from PF

add rte_pmd_i40e_set_vf_vlan_filter API.
User can call the API on PF to enable/disable
a set of VF's VLAN filters.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF VLAN tag from PF

Add rte_pmd_i40e_set_vf_vlan_tag API.
User can call the API on PF to enable/disable a specific
VF's VLAN tag.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF broadcast mode from PF

Support enabling/disabling VF broadcast mode from PF.
User can call the API on PF to enable/disable a specific
VF's broadcast mode.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF VLAN insertion from PF

Support inserting VF VLAN id from PF.
User can call the API on PF to insert a VLAN id to a
specific VF.

Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF VLAN strip from PF

Add a function to configure vlan strip enable/disable for specific
SRIOV VF device.

Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: fix VF MAC address assignment

If PF sets vf->mac_addr, in VF initialization hw->mac.addr will be set
to that same value. It is possible to check if PF set a MAC address or
not through the hw->mac.addr variable.

hw->mac.addr set by i40e_vf_parse_hw_config(), call stack is:

In PF side
i40e_pf_host_process_cmd_get_vf_resources()
    eth_addr_copy(vf->mac_addr, vf_res->vsi_res[0].default_mac_address)

In VF sise
i40evf_init_vf()
    i40evf_get_vf_resources()
            i40e_vf_parse_hw_config()
                    memcpy(hw->mac.addr, vsi_res->default_mac_addr)

Updated code is after i40evf_get_vf_resources() and can benefit from
hw->mac.addr variable.

Fixes: 89e6b86384bb ("i40evf: rework MAC address validation")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF MAC from VF

Support changing VF default MAC address.
This function is not supported if PF set the MAC
address for the PF.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF MAC from PF

Support setting VF MAC address from PF.
User can call the API on PF to set a specific
VF's MAC address.

PF should set MAC address before VF initialized,
if PF sets the MAC address after VF initialized,
new MAC address won't be effective until VF
reinitialized.

This will remove all existing MAC filters.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: fix VF reset flow

Add missing step during VF reset: PF should
set I40E_VFGEN_RSTAT to ACTIVE at end of the
VF reset operation or VF driver may not able
to detect that reset is already completed.
This patch also remove the unnecessary enum
for vfr state.

Fixes: 4861cde46116 ("i40e: new poll mode driver")

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: enable VF MTU change

This patch implement mtu_set ops for i40e VF.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF multicast promiscuous mode from PF

Support enabling/disabling VF multicast promiscuous mode from
PF.
User can call the API on PF to enable/disable a specific
VF's multicast promiscuous mode.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF unicast promiscuous mode from PF

Support enabling/disabling VF unicast promiscuous mode from
PF.
User can call the API on PF to enable/disable a specific
VF's unicast promiscuous mode.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set Tx loopback from PF

Support enabling/disabling TX loopback from PF.
User can call the API on PF to enable/disable TX loopback
for all the PF and VFs.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>

net/i40e: set VF VLAN anti-spoofing from PF

Support enabling/disabling VF VLAN anti-spoofing from
PF.
User can call the API on PF to enable/disable a specific
VF's VLAN anti-spoofing.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Vincent Jardin <vincent.jardin@6wind.com>