event/octeontx: validate events requested against available

Validate events configured in ssopf against the total number of
events configured across all the RX/TIM event adapters.

Events available to ssopf can be reconfigured by passing the required
amount to kernel bootargs and are only limited by DRAM size.
Example:
ssopf.max_events= 2097152

Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

eventdev: check allocation in Tx adapter

The function rte_zmalloc() could return NULL, the return value
need to be checked.

Fixes: a3bbf2e09756 ("eventdev: add eth Tx adapter implementation")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>

eventdev: support telemetry with xstats info

The telemetry library is connected with eventdev xstats and
port link info. The following new telemetry commands are added:

/eventdev/dev_list
/eventdev/port_list,DevID
/eventdev/queue_list,DevID
/eventdev/dev_xstats,DevID
/eventdev/port_xstats,DevID,PortID
/eventdev/queue_xstats,DevID,PortID
/eventdev/queue_links,DevID,PortID

queue_links command displays a list of queues linked with a specified
eventdev port and a service priority associated with each link.

Signed-off-by: Mike Ximing Chen <mike.ximing.chen@intel.com>
Reviewed-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>

ethdev: make flow API thread safe

Currently, the rte_flow functions are not defined as thread safe.
DPDK applications either call the functions in single thread or
protect any concurrent calling for the rte_flow operations using
a lock.

For PMDs support the flow operations thread safe natively, the
redundant protection in application hurts the performance of the
rte_flow operation functions.

And the restriction of thread safe is not guaranteed for the
rte_flow functions also limits the applications' expectation.

This feature is going to change the rte_flow functions to be thread
safe. As different PMDs have different flow operations, some may
support thread safe already and others may not. For PMDs don't
support flow thread safe operation, a new lock is defined in ethdev
in order to protects thread unsafe PMDs from rte_flow level.

A new RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE device flag is added to
determine whether the PMD supports thread safe flow operation or not.
For PMDs support thread safe flow operations, set the
RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE flag, rte_flow level functions will
skip the thread safe helper lock for these PMDs. Again the rte_flow
level thread safe lock only works when PMD operation functions are
not thread safe.

For the PMDs which don't want the default mutex lock, just set the
flag in the PMD, and add the prefer type of lock in the PMD. Then
the default mutex lock is easily replaced by the PMD level lock.

The change has no effect on the current DPDK applications. No change
is required for the current DPDK applications. For the standard posix
pthread_mutex, if no lock contention with the added rte_flow level
mutex, the mutex only does the atomic increasing in
pthread_mutex_lock() and decreasing in
pthread_mutex_unlock(). No futex() syscall will be involved.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

eal/windows: add pthread mutex

Add pthread mutex lock as it is needed for the thread safe rte_flow
functions.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Tested-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

mbuf: extend meaning of QinQ stripped bit

Clarify the documentation of QinQ flags, and extend the meaning of the
flag: if PKT_RX_QINQ_STRIPPED is set and PKT_RX_VLAN_STRIPPED is unset,
only the outer VLAN is removed from packet data, but both tci are saved
in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

crypto/bcmfs: optimize crypto request processing

Reduce number of source BDs to submit a request to crypto engine.
This improves the performance as crypto engine fetches all the BDs in
single cycle. Adjust optional metadata (OMD) in continuation of
fixed meta data (FMD).

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

test/crypto: add PDCP-SDAP cases

A new functions which uses the structure of the test vectors for SDAP
is added and call a functions responsible to call the test_pdcp_proto
with the test vector both for encapsulation and decapsulation.

Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: rework PDCP function

The function test_pdcp_proto was relying to heavily on the structure
of test vectors for PDCP making it difficult to be reusable.

The function is changed to take all the test parameters as input and
does not need access to the tests vectors anymore.

Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>

test/crypto: add test vectors for PDCP-SDAP

The test vectors are structured in a more readable way compared
to test vector for PDCP. This structure allows to have all the
information about a test vector at the same place.

Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: enable PDCP-SDAP sessions

Based on the new field in PDCP xform, a decision is made
to create a PDCP session with or without SDAP enabled.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: enable PDCP-SDAP sessions

Based on the new field in PDCP xform, a decision is made
to create a PDCP session with or without SDAP enabled.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: remove unnecessary API code from security guide

Various xform structures are being copied in
rte_security guide which can be referred from the
API documentation generated by Doxygen. The security guide
does not talk about specific details of these xforms and
thus are removed from the security guide.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

security: modify PDCP xform to support SDAP

The SDAP is a protocol in the LTE stack on top of PDCP for
QOS. A particular PDCP session may or may not have
SDAP enabled. But if it is enabled, SDAP header should be
authenticated but not encrypted if both confidentiality and
integrity is enabled. Hence, the driver should be intimated
from the xform so that it skip the SDAP header while encryption.

A new field is added in the PDCP xform to specify SDAP is enabled.
The overall size of the xform is not changed, as hfn_ovrd is just
a flag and does not need uint32. Hence, it is converted to uint8_t
and a 16 bit reserved field is added for future.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

common/dpaax/caamflib: support PDCP-SDAP

The SDAP is a protocol in the LTE stack on top of PDCP. It is
dedicated to QoS.

The difficulty of implementing this protocol is because the
PDCP change behavior regarding encryption and authentication
of the SDU it receives. In effect PDCP shall not encrypt the
SDAP SDU but must authenticate it (when encryption and
authentication is enabled).

The current version of SEC does not support the SDAP and the
change of behavior of PDCP prevent the use of the PDCP
protocol command available.

The way to do it is to reuse the PDCP implementation but to
not use the PDCP protocol and to have descriptors which
performs the PDCP protocol.

It is implemented by doing small changes of code:
        #ifdef SDAP_SUPPORT
                length += SDAP_BYTE_SIZE;
                offset -= SDAP_BYTE_SIZE;
        #endif
after having computed the size of the SN to read from the
input data, then
        #ifdef SDAP_SUPPORT
                MATHI(p, MATH0, LSHIFT, 8, MATH1, 8, 0);
                MATHB(p, MATH1, AND, sn_mask, MATH1, 8, IFB | IMMED2);
        #else
                MATHB(p, MATH0, AND, sn_mask, MATH1, 8, IFB | IMMED2);
        #endif
It will keep the SN and the SDAP header in MATH0, then shift
it to remove the SDAP header and store the result in MATH1.

Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove algo lists end

This patch removes enumerators RTE_CRYPTO_CIPHER_LIST_END,
RTE_CRYPTO_AUTH_LIST_END, RTE_CRYPTO_AEAD_LIST_END to prevent
ABI breakage that may arise when adding new crypto algorithms.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/event_crypto_adapter: fix configuration

This patch updates the xform with right configuration.
For session based ops, sym session pool is created with
valid userdata size.

Fixes: 24054e3640a2 ("test/crypto: use separate session mempools")
Cc: stable@dpdk.org
Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add cases for raw datapath API

This patch adds the cryptodev raw API test support to unit test.
In addition a new test-case for QAT PMD for the test type is
enabled.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/qat: support raw datapath API

This patch updates QAT PMD to add raw data-path API support.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>

cryptodev: add raw crypto datapath API

This patch adds raw data-path APIs for enqueue and dequeue
operations to cryptodev. The APIs support flexible user-define
enqueue and dequeue behaviors.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add
support for both cpu_crypto synchronous operation and
asynchronous raw data-path APIs. The patch also includes
AESNI-MB and AESNI-GCM PMD changes, unit test changes and
documentation updates.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support AES-CCM-256

This patch adds support for AES-CCM-256 when using AESNI-MB

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: support Chacha20-Poly1305

Add support for Chacha20-Poly1305 AEAD algorithm.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: fix GCM digest size check

GCM digest sizes should be between 1 and 16 bytes.

Fixes: 7b2d4706c90e ("crypto/aesni_mb: support newer library version only")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: fix CCM digest size check

Digest size for CCM was being checked for other algorithms
apart from CCM.

Fixes: c4c0c312a823 ("crypto/aesni_mb: check for invalid digest size")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

test/crypto: add GMAC SGL

Add Scatter-Gather List tests for AES-GMAC.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_gcm: support SGL on AES-GMAC

Add Scatter-gather list support for AES-GMAC.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Tested-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

examples/fips_validation: update GCM test

This patch updates fips validation GCM test capabilities:

- In NIST GCMVS spec GMAC test vectors are the GCM ones with
plaintext length as 0 and uses AAD as input data. Originally
fips_validation tests treats them both as GCM test vectors.
This patch introduce automatic test type recognition between
the two: when plaintext length is 0 the prepare_gmac_xform
and prepare_auth_op functions are called, otherwise
prepare_gcm_xform and prepare_aead_op functions are called.

- NIST GCMVS also specified externally or internally IV
generation. When IV is to be generated by IUT internally IUT
shall store the generated IV in the response file. This patch
also adds the support to that.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Weqaar Janjua <weqaar.a.janjua@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

examples/fips_validation: support scatter gather list

This patch adds SGL support to FIPS sample application.
Originally the application allocates single mbuf of 64KB - 1
bytes data room. With the change the user may reduce the
mbuf dataroom size by using the add cmdline option. If the
input test data is longer than the user provided data room
size the application will automatically build chained mbufs
for the target cryptodev PMD to test.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

crypto/mvsam: remove algo lists end

Remove enumerators RTE_CRYPTO_CIPHER_LIST_END,
RTE_CRYPTO_AUTH_LIST_END, RTE_CRYPTO_AEAD_LIST_END to prevent
ABI breakages that may arise when adding new crypto algorithms.

Signed-off-by: Michael Shamis <michaelsh@marvell.com>

crypto/armv8: remove algo lists end

Removed references to RTE_CRYPTO_CIPHER_LIST_END and
RTE_CRYPTO_AUTH_LIST_END to prevent ABI breakages
that may arise when adding new crypto algorithms.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>

crypto/aesni_mb: fix security session clearing

When destroying a security session, the AESNI-MB PMD attempted to clear
the private aesni_mb session object to remove any key material. However,
the function aesni_mb_pmd_sec_sess_destroy() cleared the security session
object instead of the private session object.

This patch fixes this issue by now clearing the private session object.

Fixes: fda5216fba55 ("crypto/aesni_mb: support DOCSIS protocol")
Cc: stable@dpdk.org
Signed-off-by: David Coyle <david.coyle@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/dpaa2_sec: change descriptor sharing for ERA10

In case of LX2160 or SEC ERA 10, share wait has performance
optimizations wrt to ok-to-share signal which allows multiple
DECOs to work together even in case of single queue and single SA.
Hence updated the descriptor sharing only in case of ERA10.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

crypto/dpaa2_sec: increase max anti-replay window size

In case of LX2160 or SEC ERA >= 10, max anti replay window
size supported is 1024. For all other versions of SEC, the
maximum value is capped at 128 even if application gives
more than that.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Signed-off-by: Yi Liu <yi.liu@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

crypto/dpaa2_sec: support non-HMAC auth algo versions

added support for non-HMAC for auth algorithms
(SHA1, SHA2, MD5).
Corresponding capabilities are enabled so that test
application can enable those test cases.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: support DES-CBC

add DES-CBC support for cipher_only, chain and ipsec protocol.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: support stats for secondary process

DPAA2 crypto object access need availability of MCP object
pointer. In case of secondary process, we need to use local
MCP pointer instead of primary process.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: fix stats query without queue pair

dpdk-procinfo calls the crypto stats API, which results segmentation
fault on DPAA2_SEC.
The queue pair array will be NULL, when it is used without
configuring the SEC device.

Fixes: 02f35eee264b ("crypto/dpaa2_sec: support statistics")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/octeontx2: add replay check for inline inbound packets

The function handling anti replay is added. If replay window
is enabled the rx packets will be validated against the window. The
rx offload fails in case of error.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/octeontx2: support anti-replay for security session

Initialize the inbound session for anti replay. The replay
window is allocated during session create and freed in session destroy.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

examples/fips_validation: fix version compatibility

Separate out CAVS request file version 21.4 code to support
lower versions.

Fixes: 32440cdf2af9 ("examples/fips_validation: fix parsing of TDES vectors")
Fixes: 2b84d2bd47df ("examples/fips_validation: fix count overwrite for TDES")
Cc: stable@dpdk.org
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Tested-by: Yu Jiang <yux.jiang@intel.com>

examples/fips_validation: bypass unsupported vectors

Bypass the test vectors of unsupported crypto transform
for SHA.

Signed-off-by: Archana Muniganti <marchana@marvell.com>

examples/fips_validation: support self-test only

Make it possible to pass the self-tests when no req path is set.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

examples/fips_validation: fix buffer overflow

If the file name is larger than MAX_STRING_SIZE (64), strcpy()
will overwrite the content of memory.

Replace strcpy() by rte_strscpy(), check its return value, and
increase file_name size to 256.

Fixes: 3d0fad56b74a ("examples/fips_validation: add crypto FIPS application")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

cryptodev: fix parameter parsing

The rte_cryptodev_pmd_parse_input_args function crashes with a
segmentation fault when passing a non-empty argument string.

The function passes cryptodev_pmd_valid_params to rte_kvargs_parse,
which accepts a NULL-terminated list of valid keys, yet
cryptodev_pmd_valid_params does not end with NULL. The patch adds the
missing NULL pointer.

Fixes: 9e6edea41805 ("cryptodev: add APIs to assist PMD initialisation")
Cc: stable@dpdk.org
Signed-off-by: Haggai Eran <haggaie@nvidia.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove v20 ABI compatibility

This reverts commit a0f0de06d457753c94688d551a6e8659b4d4e041 as the
rte_cryptodev_info_get function versioning was a temporary solution
to maintain ABI compatibility for ChaCha20-Poly1305 and is not
needed in 20.11.

Fixes: a0f0de06d457 ("cryptodev: fix ABI compatibility for ChaCha20-Poly1305")

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Reviewed-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add bcmfs

Add global test suite for bcmfs crypto pmd

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add crypto HW module

Add crypto h/w module to process crypto op. Crypto op is processed via
sym_engine module before submitting the crypto request to HW queues.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add session handling and capabilities

Add session handling and capabilities supported by crypto HW
accelerator

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: create a symmetric cryptodev

Create a symmetric crypto device and add supported cryptodev ops.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add HW queue pair operations

Add queue pair operations exported by supported devices.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add queue pair management

Add queue pair management APIs which will be used by Crypto device to
manage h/w queues. A bcmfs device structure owns multiple queue-pairs
based on the mapped address allocated to it.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: support VFIO

Add VFIO support for BCMFS PMD.
The BCMFS PMD functionality is dependent on the VFIO_PRESENT flag,
which gets enabled in the rte_vfio.h.
If this flag is not enabled in the compiling platform driver will
silently return with error, when executed.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: introduce BCMFS driver

Add Broadcom FlexSparc(FS) device creation driver which registers to a
vdev and create a device. Add APIs for logs, supportive documentation and
maintainers file.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support KASUMI F8/F9

Add support for KASUMI-F8/F9 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support SNOW3G-UEA2/UIA2

Add support for SNOW3G-UEA2/UIA2 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support ZUC-EEA3/EIA3

Add support for ZUC-EEA3/EIA3 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

common/cpt: check MAC length

HMAC/HASH opcode algorithms supports fixed mac length.
Allowed session creation to fail when requested for
unsupported MAC length for HMAC/HASH-only use cases.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/cpt: remove useless macros

The macros can be replaced with actual constants.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/nitrox: support cipher-only operations

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/nitrox: support AES-GCM

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/octeontx2: support lookaside IPsec IPv6

Adding IPv6 tunnel mode support in lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/octeontx2: check cpt kernel driver version

This patch checks if cpt pmd is compatible with kernel
cpt driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/octeontx2: sync cpt mailbox

This patch syncs the pmd mailbox with the cpt pf driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

test/crypto: replace armv8 test suite

Switched from device specific test suite to unified
cryptodev test suite. Removed the armv8 device specific test suite.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/armv8: fix mempool object returning

Crypto session and device session private data were put back
to wrong pools.
This caused data corruption when the object was reallocated and used.
Because objects from different mempools have different element size,
and wrong mempool info caused out of bound write.
Fix the issue by putting back objects to correct mempools.

Fixes: b3bbd9e5f265 ("cryptodev: support device independent sessions")
Fixes: 725d2a7fbf71 ("cryptodev: change queue pair configure structure")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: fix stats test

ut_setup / ut_teardown are invoked for each test case by test framework.
The call inside test_stats is unnecessary and even incorrect.
This caused double free of objects such as crypto operation structure.
Trapped the issue when RTE_LIBRTE_MEMPOOL_DEBUG was enabled.
Fix issue by removing ut_setup / ut_teardown from test case implementation.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Tested-by: Adam Dybkowski <adamx.dybkowski@intel.com>

crypto/aesni_mb: support AES-ECB

This patch adds AES-ECB 128, 192 and 256 support to the aesni_mb PMD.
AES-ECB 128, 192 and 256 test vectors added to cryptodev tests.

Signed-off-by: Marcel Cornu <marcel.d.cornu@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

baseband/turbo_sw: detect dependencies automatically

The meson for the turbo_sw PMD is updated to prevent the
requirement for any device specific toplevel flags to be
passed down (unlike what used to be the case with make).
The linking to the optional libraries is purely auto
detected at build time and flags are then set appropriately.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

doc: update bbdev guide

Clarify the capability assumptions for LLR and HARQ
compression format.
Correct one historical typo.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>

baseband/fpga_lte_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_lte_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead

Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

baseband/fpga_5gnr_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_5gnr_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead.

Fixes: 0b5927cbcba7 ("baseband/fpga_5gnr_fec: add PMD for FPGA 5GNR FEC")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

app/bbdev: fix test vector symlink

5G DL default symlink was pointing to a 4G vector.

Fixes: d762705308c4 ("app/bbdev: add test vectors for 5GNR")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>

crypto/scheduler: rename slave to worker

This patch replaces the usage of the word 'slave' with more
appropriate word 'worker' in QAT PMD and Scheduler PMD
as well as in their docs. Also the test app was modified
to use the new wording.

The Scheduler PMD's public API was modified according to the
previous deprecation notice:
rte_cryptodev_scheduler_slave_attach is now called
rte_cryptodev_scheduler_worker_attach,
rte_cryptodev_scheduler_slave_detach is
rte_cryptodev_scheduler_worker_detach,
rte_cryptodev_scheduler_slaves_get is
rte_cryptodev_scheduler_workers_get.

Also, the configuration value RTE_CRYPTODEV_SCHEDULER_MAX_NB_SLAVES
was renamed to RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: fix a null pointer dereference

This patch fixes a null pointer dereference after null check detected by
coverity scan.

Coverity issue: 349904
Fixes: 6a0c9d364afc ("crypto/dpaax_sec: support HFN override")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: replace NITROX PMD specific test suite

Replace NITROX PMD specific tests with generic test suite.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: remove orphan bbdev PMD feature table

Removing a feature table referring erroneously
to a PMD not present in DPDK.

Fixes: 65f1eec ("doc: add feature matrix table for bbdev")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/fpga_lte_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_lte_fec API is still experimental.
Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

baseband/fpga_5gnr_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_5gnr_fec API is still experimental.

Fixes: 2d4306438c92 ("baseband/fpga_5gnr_fec: add configure function")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

ipsec: promote library as stable

Since librte_ipsec was first introduced in 19.02 and there were no changes
in it's public API since 19.11, it should be considered mature enough to
remove the 'experimental' tag from it.
The RTE_SATP_LOG2_NUM enum is also being dropped from rte_ipsec_sa.h to
avoid possible ABI problems in the future.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/acc100: add configure function

Add configure function to configure the PF from within
the bbdev-test itself without external application
configuration the device.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add debug function to validate input

Debug functions to validate the input API from user
Only enabled in DEBUG mode at build time

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support interrupt

Adding capability and functions to support MSI
interrupts, call backs and inforing.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support 4G processing

Adding capability for 4G encode and decoder processing

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support HARQ loopback

Additional support for HARQ memory loopback

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add LDPC processing functions

Adding LDPC decode and encode processing operations

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add queue configuration

Adding function to create and configure queues for
the device. Still no capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add info get function

Add in the "info_get" function to the driver, to allow us to query the
device.
No processing capability are available yet.
Linking bbdev-test to support the PMD with null capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add HW register definitions

Add in the list of registers for the device and related
HW specs definitions.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: introduce PMD for ACC100

Add stubs for the ACC100 PMD

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

test/crypto: fix device number

In testsuite_setup(), ts_params is configured for first valid device.
The same device should be used as valid device in
test_device_configure_invalid_dev_id test case.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

app/test-sad: fix uninitialized variable

Coverity issue: 362055
Fixes: 908be0651a5a ("app/test-sad: add test application for IPsec SAD")
Cc: stable@dpdk.org
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

crypto/octeontx2: fix session-less mode

A temporary session is created for sessionless crypto operations.
rte_cryptodev_sym_session_create() should be used for creating the
temporary session as it initializes the session structure in the
correct way. Also the session should be set to 0 before freeing it.

Fixes: 17ac2a72191b ("crypto/octeontx2: add enqueue/dequeue ops")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

eal: simplify exit functions

The option RTE_EAL_ALWAYS_PANIC_ON_ERROR was off by default,
and not customizable with meson. It is completely removed.

The function rte_dump_registers is a trace of the bare metal support
era, and was not supported in userland. It is completely removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: David Marchand <david.marchand@redhat.com>

eal: add new prefetch write variants

This commit adds new rte_prefetchX_write() variants, that suggest to the
compiler to use a prefetch instruction with intention to write. As a
compiler builtin, the compiler can choose based on compilation target
what the best implementation for this instruction is.

Three versions are provided, targeting the different levels of cache.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

eal: fix build with conflicting libc variable memory_order

The cited commit introduced functions with 'int memory_order' argument.
The C11 standard section 7.17.1.4 defines 'memory_order' as the
"enumerated type whose enumerators identify memory ordering constraints".

A compilation error occurs:
error: declaration of 'memory_order' shadows a global declaration
    [-Werror=shadow]
     rte_atomic_thread_fence(int memory_order)

This issue was hit when trying to compile OVS with gcc 4.8.5. This
compiler version does not provide stdatomic.h, so enum memory_order is
redefined in OVS code.
In another case, if the compiler does provide stdatomic.h header,
passing -Wsystem-headers in the CFLAGS will also cause that failure.

Fix it by changing the argument name 'memory_order' to 'memorder'.

Fixes: 672a15056380 ("eal: add wrapper for C11 atomic thread fence")

Signed-off-by: Eli Britstein <elibr@nvidia.com>
Reviewed-by: Asaf Penso <asafp@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

acl: fix build with gcc 5.4.0

gcc 5.4 fails with:
../lib/librte_acl/acl_run_avx512x8.h: In function 'match_process_avx512x8':
../lib/librte_acl/acl_run_avx512x8.h:382:31: error:
pointer targets in passing argument 1 of '_mm256_mask_i32scatter_epi32'
differ in signedness [-Werror=pointer-sign]

Later gcc versions work fine, as for them parameter type was
changed to 'void *'.
Fixed by applying explicit cast for offending argument.

Bugzilla ID: 556
Fixes: b64c2295f7fc ("acl: add 256-bit AVX512 classify method")
Fixes: 45da22e42ec3 ("acl: add 512-bit AVX512 classify method")

Reported-by: Ali Alnubani <alialnu@nvidia.com>
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>

eal: add experimental tags for write combining store

Only marking the doxygen declarations is not enough.
Arch specific implementations must be tagged as well since there is no
common declaration of those inlines.

Fixes: 8a00dfc738fe ("eal: add write combining store")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Radu Nicolau <radu.nicolau@intel.com>

sched: remove redundant subport parameters

Remove redundant data structure fields.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

test/sched: update subport rate dynamically

Modify the test_sched application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>