crypto/mvsam: update queue mapping comments

Till now comments explain queue mapping per
one and two crypto devices.
Now added comments for queue mapping for three
crypto devices supported in CN9132.

Signed-off-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Liron Himi <lironh@marvell.com>

crypto/mvsam: fix session data reset

This patch fix wrong clear memory session pointer.

When call mrvl_crypto_pmd_sym_session_clear function,
it might cause the following error:
CRYPTODEV: set_sym_session_private_data() line 497:
Set private data for driver 1 not allowed

This fix set zeroes to mrvl_crypto_session pointer instead to
rte_cryptodev_sym_session pointer, and will verify that
rte_cryptodev_sym_session pointer will not lose data such as
nb_drivers.

Bugzilla ID: 646
Fixes: 8a61c83af2fa ("crypto/mrvl: add mrvl crypto driver")
Cc: stable@dpdk.org
Signed-off-by: Dana Vardi <danat@marvell.com>
Reviewed-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Liron Himi <lironh@marvell.com>

crypto/mvsam: fix capabilities

Increased AAD max size up to 64B and allowed AAD increment step
1B according to EIP197 token HW limitation.
In addition, some capabilities were found incorrect due to
running FIPS tests. The patch fix the capabilities.

Fixes: 8a61c83af2f ("crypto/mrvl: add mrvl crypto driver")
Fixes: 2c22aa4f8f6 ("crypto/mvsam: update hash digest sizes")
Fixes: a84226fc055 ("crypto/mvsam: support HMAC SHA224")
Cc: stable@dpdk.org
Signed-off-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Liron Himi <lironh@marvell.com>
Tested-by: Liron Himi <lironh@marvell.com>

crypto/mvsam: fix AES-GCM session parameters

For AEAD session and AES-GCM mode, IV offset will
set to include the Nonce.
In addition, the following operations order will define:
first encryption and then authentication.

Fixes: 8a61c83af2fa ("crypto/mrvl: add mrvl crypto driver")
Cc: stable@dpdk.org
Signed-off-by: Dana Vardi <danat@marvell.com>
Reviewed-by: Liron Himi <lironh@marvell.com>
Tested-by: Liron Himi <lironh@marvell.com>

test/crypto: use formal array size macro

Replaces the use of local ARRAY_SIZE macro, which is not used
anywhere else, with the formal RTE_DIM macro for better
consistency in crypto unit test.

Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: fix typo in ESN case

Fixed typo in the crypto unit test ESN function name by
replacing the word 'encryt' with the correct word 'encrypt'.

test_authenticated_encryt_with_esn is now called
test_authenticated_encrypt_with_esn.

Fixes: 699741912d71 ("test/crypto: add case for auth only trailer")
Cc: stable@dpdk.org
Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

test/crypto: fix typo in AES case

Fixed typo in the testcase description for AES in the crypto
unit test suite by changing the word 'Scater' to 'Scatter'.

Fixes: c9c9c4ed8746 ("app/test: check scatter-gather for crypto drivers")
Cc: stable@dpdk.org
Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

test/crypto: rename slave to worker

Modifies the scheduler tests in the crypto unit test suite
to replace the usage of the word 'slave' with the more
appropriate word 'worker'.

The scheduler test functions were modified as follows:
test_scheduler_attach_slave_op is now called
test_scheduler_attach_worker_op,
test_scheduler_detach_slave_op is
test_scheduler_detach_worker_op.

Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: fix autotest function parameters

Fixed parameters on autotest functions by removing comments.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Fixes: 4ed1e79e7819 ("test/crypto: add tests for virtio-crypto")
Fixes: 3aafc423cf4d ("snow3g: add driver for SNOW 3G library")
Fixes: 27a1c4714d54 ("app/test: add KASUMI crypto")
Fixes: 4c99481f49c4 ("app/test: add ZUC")
Fixes: c8e69fce7046 ("crypto/scheduler: add unit test")
Fixes: ae002048bbea ("test/crypto: add DPAA2 crypto functional test")
Fixes: b674d6d0381a ("test/crypto: add dpaa crypto test cases")
Fixes: a8dbd44d6b4c ("test/crypto: add CAAM JR validation cases")
Fixes: 4868f6591c6f ("test/crypto: add cases for raw datapath API")
Cc: stable@dpdk.org
Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/aesni_gcm: fix performance on some AVX512 CPUs

This patch fixes the aesni_gcm performance issue on systems with AVX512
CPU flag presented but with VAES CPU flag missing, such as Skylake.

Fixes: 81fe96a0cece ("crypto/aesni_gcm: use architecture independent API")
Cc: stable@dpdk.org
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

crypto/octeontx: add crypto adapter data path

Added support for crypto adapter OP_FORWARD mode.

As OcteonTx CPT crypto completions could be out of order, each crypto op
is enqueued to CPT, dequeued from CPT and enqueued to SSO one-by-one.

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/octeontx: add crypto adapter framework

Set crypto adapter event device slow-path call backs.

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

doc: update dependencies for SW crypto PMDs

This patch updates the dependency requirement information
for aesni-gcm, aesni-mb, snow3g, zuc, and kasumi PMDs. Previously
building these PMDs with Make will fail when the system is
installed intel-ipsec-mb library version 1.0 or newer.

Since Make build system is deprecated already, instead of fixing
the issue the documentation is updated to state it.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>

common/cpt: add checks for offset overflow

Add checks to catch overflow of any offsets. Offset control word
specifies,

1. 16 bits encryption offset
2. 8 bits IV offset
3. 8 bits auth offset

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: fix mbuf reset after null check

Use m only after it was checked not to be NULL.

Fixes: 202d375c60bc1 ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

app/crypto-perf: fix out-of-place mempool allocation

Add missing rte_mbuf size in mempool allocation for
out-of-place operation.

Fixes: bf9d6702eca9 ("app/crypto-perf: use single mempool")
Cc: stable@dpdk.org
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/qat: fix Arm build with special memcpy

In Arm platform, when "RTE_ARCH_ARM64_MEMCPY" is set as true, gcc will
report variable uninitialized warning:

../drivers/crypto/qat/qat_sym_session.c: In function ‘partial_hash_compute’:
../lib/eal/include/generic/rte_byteorder.h:241:24: warning:
‘<U35a0>’ may be used uninitialized in this function
[-Wmaybe-uninitialized]
241 | #define rte_bswap32(x) __builtin_bswap32(x)
...

This is because "digest" will be initialized by "rte_memcpy" function
rather than "memcpy" if "RTE_ARCH_ARM64_MEMCPY" is set as true.
Furthermore, 'rte_memcpy' will initialize 'digest' with two steps
by invoking rte_mov_x functions. For example:

partial_hash_sha1 -> rte_memcpy -> rte_memcpy_ge16_lt_128 ->
step 1: rte_mov16(dst,src )
step 2: rte_mov16(dst - 16 + n, src - 16 + n)

However, gcc compiler cannot identify this multi-step initialization,
then it will report warning.

To fix this, use "memset" to initialize "digest".

Fixes: cd7fc8a84b48 ("eal/arm64: optimize memcpy")
Cc: stable@dpdk.org
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>

crypto/octeontx: increase metabuf pool

When used with crypto adapter, metabuf pool would be shared across
cores. Account for the same in pool size.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/octeontx2: increase metabuf pool

When used with crypto adapter, metabuf pool would be shared across
cores. Account for the same in pool size.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: add cnxk for asymmetric cases

Registered cn9k and cn10k for asymmetric crypto
autotest. Documentation and release notes are also
updated.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add asymmetric capabilities

Add asymmetric crypto capabilities supported
by cn9k and cn10k PMDs. Documentation is also
updated for the same.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add asymmetric datapath

Add asymmetric crypto datapath for cn9k and cn10k PMD.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add asymmetric session

Add asymmetric crypto session ops for both cn9k
and cn10k PMD.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add security capabilities

Add security capabilities supported by crypto cn10k PMD.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add IPsec datapath

Add rte_security handling in cn10k crypto enqueue
dequeue ops to support IPSec protocol offload.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add security session operations

Add security session ops in cn10k crypto PMD
to support lookaside protocol offload IPsec.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add security context skeleton

Add security ctx in cn10k crypto PMD.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: enable cnxk

Enable tests for cn9k & cn10k crypto PMDs.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add symmetric capabilities

Add symmetric crypto capabilities for cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add digest support

Add support for digest support for various algorithms.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add KASUMI decryption

Add KASUMI decrypt support.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add KASUMI encryption

Add KASUMI encrypt support.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add ZUC and SNOW3G decryption

Add PDCP opcode which handles ZUC and SNOW3G.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add ZUC and SNOW3G encryption

Add PDCP opcode which handles ZUC and SNOW3G.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add flexi cipher decryption

Add flexi crypto cipher decrypt support in enqueue API. Flexi crypto
opcode covers a broad set of ciphers including variants of AES.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add flexi cipher encryption

Add flexi crypto cipher encrypt in enqueue API. Flexi crypto
opcode covers a broad set of ciphers including variants of AES.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add chained operation in session

Add support for chained operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add AEAD operation in session

Add support for AEAD operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add auth operation in session

Add support for auth operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add cipher operation in session

Add support for cipher operation in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add dequeue burst

Add dequeue_burst op in cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add enqueue burst

Add enqueue_burst op in cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add session operations framework

Add session ops
- sym_session_get_size
- sym_session_configure
- sym_session_clear

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add queue pair operations

Add ops for
- queue_pair_setup()
- queue_pair_release()

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add device control operations

Add ops for
- dev_configure()
- dev_start()
- dev_stop()
- dev_close()
- dev_infos_get()

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add probe and remove

Add probe & remove for cn9k & cn10k crypto PMDs.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add driver skeleton

Add driver skeleton for crypto_cn9k & crypto_cn10k
PMDs leveraging cnxk common framework.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add SE set key for crypto

The set key functions are added in roc to be used
in crypto operations.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add IPsec common code

Add code that can be leveraged across
lookaside and inline IPsec on cn10k.

Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add EC vectors for asymmetric crypto

EC group static vectors are required for CPT asymmetric
crypto operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add FPM tables for asymmetric crypto

Add scalar FPM tables to be used for asymmetric crypto
operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add lmtline initialization

Add routine to initialize LMTLINE which facilitates instruction
submission to CPT. Add common macros required in the enqueue
operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add AE definitions for asymmetric crypto

Microcode AE opcodes support asymmetric operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add IE definitions for IPsec

Microcode IE opcodes support IPsec operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add SE definitions for symmetric crypto

Microcode SE opcodes support symmetric operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add inline IPsec configuration mailbox

Add mbox to configure inbound & outbound inline IPsec.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT LF flush

Add routine to flush context from CPT context processor cache.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT diagnostics

Add routines to fetch and dump CPT statistics and states.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT LF configuration

Add routines to init & fini CPT LFs. CPT LFs are
queues to the hardware enabling instruction submissions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add mailbox to configure RXC

Add mailbox to configure timeouts and thresholds in
CPT RXC unit.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT get/set

Add APIs to set & get CPT device.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT configuration

Add routines to init, fini, configure & clear CPT device.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: update Rx inline IPsec mailbox format

Updates Rx inline IPSEC mailbox message format to sync
with latest CPT PF driver.

Signed-off-by: Srujana Challa <schalla@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT hardware definitions

Add CPT hardware definitions. CPT is the hardware block on
cnxk family of processors, that can be used to offload
cryptographic operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

net/tap: define offload capabilities constants

Since the offload values are always the same, these can
just be data instead of code.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Keith Wiles <keith.wiles@intel.com>

net/tap: remove useless offload capability functions

Since these always return 0, they were doing nothing useful.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Keith Wiles <keith.wiles@intel.com>

app/testpmd: change port link speed without stopping all

When we use the following cmd to modify the link speed of specified
port: "port config <port_id> speed xxx duplex xxx", we have to stop
all ports. It's not necessary.

Fixes: 82113036e4e5 ("ethdev: redesign link speed config")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Xiaoyun Li <xiaoyun.li@intel.com>

net/cxgbe: remove basic queue statistics

Remove queue stats from basic stats because they're now available
via xstats API. Also remove RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support extended statistics

Add support to fetch port and queue stats via xstats API.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

doc: add link status event requirements in hns3 guide

Add link status event pre-conditions in the driver documentation.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/memif: improve memcpy performance

Replacing memcpy() with rte_memcpy() improved 64 byte packet
performance by 33% on a POWER9 system and by 10% on an x86_64
system.

Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

ethdev: fix doc of flow action

The struct rte_flow_action was missing from DPDK API documentation.

Fixes: 3850cf0c8c37 ("ethdev: add tunnel encap/decap actions")
Cc: stable@dpdk.org
Signed-off-by: Jan Viktorin <viktorin@cesnet.cz>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Aman Deep Singh <aman.deep.singh@intel.com>

net/hns3: fix traffic management

In a multi-TC scenario, if the length of packets destined for different
TCs is different, for example, 64B and 1500B packets destined for TC0 and
TC1 respectively. There is a problem that the bandwidth of the TC to which
large packets are sent is preempted by the TC to which small packets are
sent on the Kunpeng 920 network engine. As a result, the TC bandwidth
accuracy is inaccurate.

To solve this problem, this patch made the following adjustments:
1/ During initialization, firmware reports the capability bit indicating
whether the TM function is supported.
2/ The command word for configuring TC and port rate limiting is added,
instead of reusing the existing command word. And firmware configured
to the correct module.
3/ When the PF driver is loaded, firmware completes the default
initialization of the TC and port.

Fixes: c09c7847d892 ("net/hns3: support traffic management")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/nfp: fix multi-process PF probing

This patch creates a new function for handling PF probing of a secondary
process. A CPP handle is obtained for the CPP bridge service and the
service itself is also registered during secondary process
initialization. DPDK services aren't shared between processes so it is
not enough to only have the primary register the service if it is also
needed in a secondary process. This implies that both the primary and
secondary will have their own copy of the bridge service.

Fixes: 5e15e799d697 ("net/nfp: create separate entity for PF device")
Cc: stable@dpdk.org
Signed-off-by: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Signed-off-by: Simon Horman <simon.horman@corigine.com>

net/nfp: improve PF probing logic

When using rte_eth_dev_pci_generic_probe() during probing a
rte_eth_dev will be created with the name field corresponding to the PCI
address of the device. NFP4000/6000 devices only have a single PF (but
potentially multiple physical ports). This means that in a simple two
port example the rte_eth_devices[] array will be populated with two
devices: 0000:02:00.0 and 0000:02:00.0_port1. This is inconsistent and
not ideal. It will also cause issues when a secondary process tries to
attach to these ports.

This patch removes the use of rte_eth_dev_pci_generic_probe() and
allocates eth_dev's for each physical port during PF initialization,
giving them more consistent names.

Fixes: 5e15e799d697 ("net/nfp: create separate entity for PF device")
Cc: stable@dpdk.org
Signed-off-by: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Signed-off-by: Simon Horman <simon.horman@corigine.com>

app/testpmd: build on Windows

- Disable unsupported apps on Windows
- Enable building of testpmd on Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: disable flow director functions if unneeded

Function print_fdir_mask and print_fdir_flex_payload are only called
when either i40e or ixgbe presents. Extend existing #if defined to
include these two functions, to remove "unused function" compilation
warning.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: replace POSIX-specific code

- Make printf format OS independent
- Replace htons with RTE_BE16
- Replace POSIX specific inet_aton with OS independent inet_pton
- Replace sleep with rte_delay_us_sleep
- Replace random with rte_rand
- #ifndef mman related code for now
- Fix header inclusion
- Include rte_os_shim.h in testpmd.h
- Remove redundant headers

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: fix type of FEC mode parsing output

Passing an uint32_t pointer to an enum pointer parameter causes
pointer-sign warning on Windows (converts between pointers to
integer types with different sign), since enum is implicitly
converted to int on Windows.

And the current enum pointer parameter of that function is actually
misleading and should be fixed as an uint32_t pointer parameter.

Fixes: b19da32e3151 ("app/testpmd: add FEC command")
Cc: stable@dpdk.org
Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

app/testpmd: resolve name collisions with Windows types

Resolve name collisions with Windows types.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add clock function

Add clock_gettime() on Windows in rte_os_shim.h.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add device event stubs

Add device event stubs in eal_dev.c for Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add macros required by testpmd

Add required macros by testpmd on Windows in rte_os_shim.h

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

lib: build testpmd dependencies on Windows

Enable building libraries that testpmd depends on for Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

test/cksum: add L3/L4 checksum API test

Add a simple unit test for checksum API.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net: introduce functions to verify L4 checksums

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() and
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

Since these functions should be used to calculate the checksum to set in
a packet, introduce 2 new helpers for checksum verification. They return
0 if the checksum is valid in the packet.

Use this new helper in net/tap driver.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on TCP packets

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() or
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

This new behavior broke the checksum verification in tap driver for TCP
packets: these packets are marked with PKT_RX_L4_CKSUM_BAD.

Fix this by checking the 2 possible values. A next commit will introduce
a checksum verification helper to simplify this a bit.

Fixes: d5df2ae0428a ("net: fix unneeded replacement of TCP checksum 0")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on IP options packets

When packet type is IPV4_EXT, the checksum is always marked as good in
the mbuf offload flags.

Since we know the header lengths, we can easily call
rte_ipv4_udptcp_cksum() in this case too.

Fixes: 8ae3023387e9 ("net/tap: add Rx/Tx checksum offload support")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/cxgbe: add MAC match-all to track promiscuous traffic

Chelsio T6 ASIC doesn't track Rx promisc traffic dropped due to lack
of Rx buffers and hence the imissed counter doesn't increment. Add
support for RAW MAC filter to insert a wildcard matchall rule at
the end of MPS TCAM to make MPS track the promisc traffic. This
rule will only be added/removed when promisc mode is turned on/off
on the interface.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: use C11-style compiler builtins for atomics

Replace rte_atomic ops with C11 atomics.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/sfc: fix outer L4 checksum Rx

Do not return "GOOD" flag if the status was bad.

Fixes: 9e6e7f479ae9b ("net/sfc: support Rx checksum offload for EF100")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/pfe: remove unnecessary null check

ethdev library does the check of dev before calling the link update.

Fixes: acd4818ea2a45 ("net/pfe: add link status update")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Gagandeep Singh <g.singh@nxp.com>

app/testpmd: move conntrack update after null check

Use out only after it was checked not to be null.

Fixes: 4d07cbefe3ba0 ("app/testpmd: add commands for conntrack")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Aman Deep Singh <aman.deep.singh@intel.com>

net/hns3: support Tx push quick doorbell for performance

Kunpeng 930 support Tx push mode which could improve performance.
It works like below:
 1. Add PCIe bar45 which support driver direct write the Tx descriptor
    or tail reg to it.
 2. Support three operations: a) direct write one Tx descriptor, b)
    direct write two Tx descriptors, c) direct write tail reg.
 3. The original tail reg located at bar23, the above bar45 tail reg
    could provide better bandwidth from the hardware perspective.

The hns3 driver only support direct write tail reg (also have the name
of quick doorbell), the detail:
Considering compatibility, firmware will report Tx push capa if the
hardware support it.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: remove unnecessary blank lines

Delete redundant blank lines to make:
1.Return value judgment follow the function call.
2.No blank lines at the end of a code block defined by braces.

Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix maximum queues on configuration failure

The "cfg_max_queues" maintains configured max queue numbers from user,
and is equal to the maximum of "nb_rx_queues" and "nb_tx_queues" in
"dev->data".

From the ethdev layer framework, "nb_rx/tx_queues" in "dev->data" were set
to zero in rte_eth_dev_configure() if ops.dev_configure in PMD fails to be
executed, In addition, if ops.dev_configure in HNS3 PMD failed, the fake
queues are also cleared on a device that does not support independent Rx/Tx
queues.

Therefore, the "cfg_max_queues" should be also set to zero when
dev_configure fails.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix VLAN strip log

When the current VLAN stripping is set, the log print always prompts
that the enabling fails, bug if may actually be the disabling failure.

Fixes: 411d23b9eafb ("net/hns3: support VLAN")
Cc: stable@dpdk.org
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix fake queue rollback

When the device supports independent Rx/Tx queues, fake queues do not need
to be created in unequal Rx/Tx queues case. However, dev_configure fails
to be executed on the device supported independent Rx/Tx queues, the
current rollback code logic contains the fake queue. As a result, the fake
queue is created. When dev_configure is successfully called again, these
fake queues still exists and are configured to the hardware.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix delay for waiting to stop Rx/Tx

When the primary process executes dev_stop or is being reset, the packet
sending and receiving functions is changed. In this moment, the primary
process requests secondary processes to change their Rx/Tx functions, and
delays a period of time in case of crashes when queues are still in use.
The delay time depends on the number of queues actually used, instead of
the maximum number of queues supported by the device.

Fixes: 23d4b61fee5d ("net/hns3: support multiple process")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: increase VF reset retry maximum

When the device is very busy, VF reset may have to be
retried many times to succeed, leading to the current
max reset fail retry count not enough.

Modify max reset fail retry count to 30 to enhance
the reliability of reset function.

Fixes: 2790c6464725 ("net/hns3: support device reset")
Cc: stable@dpdk.org
Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>