crypto/qat: support raw datapath API

This patch updates QAT PMD to add raw data-path API support.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>

cryptodev: add raw crypto datapath API

This patch adds raw data-path APIs for enqueue and dequeue
operations to cryptodev. The APIs support flexible user-define
enqueue and dequeue behaviors.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add
support for both cpu_crypto synchronous operation and
asynchronous raw data-path APIs. The patch also includes
AESNI-MB and AESNI-GCM PMD changes, unit test changes and
documentation updates.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support AES-CCM-256

This patch adds support for AES-CCM-256 when using AESNI-MB

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: support Chacha20-Poly1305

Add support for Chacha20-Poly1305 AEAD algorithm.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: fix GCM digest size check

GCM digest sizes should be between 1 and 16 bytes.

Fixes: 7b2d4706c90e ("crypto/aesni_mb: support newer library version only")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: fix CCM digest size check

Digest size for CCM was being checked for other algorithms
apart from CCM.

Fixes: c4c0c312a823 ("crypto/aesni_mb: check for invalid digest size")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

test/crypto: add GMAC SGL

Add Scatter-Gather List tests for AES-GMAC.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_gcm: support SGL on AES-GMAC

Add Scatter-gather list support for AES-GMAC.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Tested-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

examples/fips_validation: update GCM test

This patch updates fips validation GCM test capabilities:

- In NIST GCMVS spec GMAC test vectors are the GCM ones with
plaintext length as 0 and uses AAD as input data. Originally
fips_validation tests treats them both as GCM test vectors.
This patch introduce automatic test type recognition between
the two: when plaintext length is 0 the prepare_gmac_xform
and prepare_auth_op functions are called, otherwise
prepare_gcm_xform and prepare_aead_op functions are called.

- NIST GCMVS also specified externally or internally IV
generation. When IV is to be generated by IUT internally IUT
shall store the generated IV in the response file. This patch
also adds the support to that.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Weqaar Janjua <weqaar.a.janjua@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

examples/fips_validation: support scatter gather list

This patch adds SGL support to FIPS sample application.
Originally the application allocates single mbuf of 64KB - 1
bytes data room. With the change the user may reduce the
mbuf dataroom size by using the add cmdline option. If the
input test data is longer than the user provided data room
size the application will automatically build chained mbufs
for the target cryptodev PMD to test.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>

crypto/mvsam: remove algo lists end

Remove enumerators RTE_CRYPTO_CIPHER_LIST_END,
RTE_CRYPTO_AUTH_LIST_END, RTE_CRYPTO_AEAD_LIST_END to prevent
ABI breakages that may arise when adding new crypto algorithms.

Signed-off-by: Michael Shamis <michaelsh@marvell.com>

crypto/armv8: remove algo lists end

Removed references to RTE_CRYPTO_CIPHER_LIST_END and
RTE_CRYPTO_AUTH_LIST_END to prevent ABI breakages
that may arise when adding new crypto algorithms.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>

crypto/aesni_mb: fix security session clearing

When destroying a security session, the AESNI-MB PMD attempted to clear
the private aesni_mb session object to remove any key material. However,
the function aesni_mb_pmd_sec_sess_destroy() cleared the security session
object instead of the private session object.

This patch fixes this issue by now clearing the private session object.

Fixes: fda5216fba55 ("crypto/aesni_mb: support DOCSIS protocol")
Cc: stable@dpdk.org
Signed-off-by: David Coyle <david.coyle@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/dpaa2_sec: change descriptor sharing for ERA10

In case of LX2160 or SEC ERA 10, share wait has performance
optimizations wrt to ok-to-share signal which allows multiple
DECOs to work together even in case of single queue and single SA.
Hence updated the descriptor sharing only in case of ERA10.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

crypto/dpaa2_sec: increase max anti-replay window size

In case of LX2160 or SEC ERA >= 10, max anti replay window
size supported is 1024. For all other versions of SEC, the
maximum value is capped at 128 even if application gives
more than that.

Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Signed-off-by: Yi Liu <yi.liu@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

crypto/dpaa2_sec: support non-HMAC auth algo versions

added support for non-HMAC for auth algorithms
(SHA1, SHA2, MD5).
Corresponding capabilities are enabled so that test
application can enable those test cases.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: support DES-CBC

add DES-CBC support for cipher_only, chain and ipsec protocol.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: support stats for secondary process

DPAA2 crypto object access need availability of MCP object
pointer. In case of secondary process, we need to use local
MCP pointer instead of primary process.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: fix stats query without queue pair

dpdk-procinfo calls the crypto stats API, which results segmentation
fault on DPAA2_SEC.
The queue pair array will be NULL, when it is used without
configuring the SEC device.

Fixes: 02f35eee264b ("crypto/dpaa2_sec: support statistics")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/octeontx2: add replay check for inline inbound packets

The function handling anti replay is added. If replay window
is enabled the rx packets will be validated against the window. The
rx offload fails in case of error.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/octeontx2: support anti-replay for security session

Initialize the inbound session for anti replay. The replay
window is allocated during session create and freed in session destroy.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

examples/fips_validation: fix version compatibility

Separate out CAVS request file version 21.4 code to support
lower versions.

Fixes: 32440cdf2af9 ("examples/fips_validation: fix parsing of TDES vectors")
Fixes: 2b84d2bd47df ("examples/fips_validation: fix count overwrite for TDES")
Cc: stable@dpdk.org
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Tested-by: Yu Jiang <yux.jiang@intel.com>

examples/fips_validation: bypass unsupported vectors

Bypass the test vectors of unsupported crypto transform
for SHA.

Signed-off-by: Archana Muniganti <marchana@marvell.com>

examples/fips_validation: support self-test only

Make it possible to pass the self-tests when no req path is set.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

examples/fips_validation: fix buffer overflow

If the file name is larger than MAX_STRING_SIZE (64), strcpy()
will overwrite the content of memory.

Replace strcpy() by rte_strscpy(), check its return value, and
increase file_name size to 256.

Fixes: 3d0fad56b74a ("examples/fips_validation: add crypto FIPS application")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

cryptodev: fix parameter parsing

The rte_cryptodev_pmd_parse_input_args function crashes with a
segmentation fault when passing a non-empty argument string.

The function passes cryptodev_pmd_valid_params to rte_kvargs_parse,
which accepts a NULL-terminated list of valid keys, yet
cryptodev_pmd_valid_params does not end with NULL. The patch adds the
missing NULL pointer.

Fixes: 9e6edea41805 ("cryptodev: add APIs to assist PMD initialisation")
Cc: stable@dpdk.org
Signed-off-by: Haggai Eran <haggaie@nvidia.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove v20 ABI compatibility

This reverts commit a0f0de06d457753c94688d551a6e8659b4d4e041 as the
rte_cryptodev_info_get function versioning was a temporary solution
to maintain ABI compatibility for ChaCha20-Poly1305 and is not
needed in 20.11.

Fixes: a0f0de06d457 ("cryptodev: fix ABI compatibility for ChaCha20-Poly1305")

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Reviewed-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add bcmfs

Add global test suite for bcmfs crypto pmd

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add crypto HW module

Add crypto h/w module to process crypto op. Crypto op is processed via
sym_engine module before submitting the crypto request to HW queues.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add session handling and capabilities

Add session handling and capabilities supported by crypto HW
accelerator

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: create a symmetric cryptodev

Create a symmetric crypto device and add supported cryptodev ops.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add HW queue pair operations

Add queue pair operations exported by supported devices.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: add queue pair management

Add queue pair management APIs which will be used by Crypto device to
manage h/w queues. A bcmfs device structure owns multiple queue-pairs
based on the mapped address allocated to it.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: support VFIO

Add VFIO support for BCMFS PMD.
The BCMFS PMD functionality is dependent on the VFIO_PRESENT flag,
which gets enabled in the rte_vfio.h.
If this flag is not enabled in the compiling platform driver will
silently return with error, when executed.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: introduce BCMFS driver

Add Broadcom FlexSparc(FS) device creation driver which registers to a
vdev and create a device. Add APIs for logs, supportive documentation and
maintainers file.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support KASUMI F8/F9

Add support for KASUMI-F8/F9 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support SNOW3G-UEA2/UIA2

Add support for SNOW3G-UEA2/UIA2 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support ZUC-EEA3/EIA3

Add support for ZUC-EEA3/EIA3 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

common/cpt: check MAC length

HMAC/HASH opcode algorithms supports fixed mac length.
Allowed session creation to fail when requested for
unsupported MAC length for HMAC/HASH-only use cases.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/cpt: remove useless macros

The macros can be replaced with actual constants.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/nitrox: support cipher-only operations

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/nitrox: support AES-GCM

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/octeontx2: support lookaside IPsec IPv6

Adding IPv6 tunnel mode support in lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/octeontx2: check cpt kernel driver version

This patch checks if cpt pmd is compatible with kernel
cpt driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/octeontx2: sync cpt mailbox

This patch syncs the pmd mailbox with the cpt pf driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

test/crypto: replace armv8 test suite

Switched from device specific test suite to unified
cryptodev test suite. Removed the armv8 device specific test suite.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/armv8: fix mempool object returning

Crypto session and device session private data were put back
to wrong pools.
This caused data corruption when the object was reallocated and used.
Because objects from different mempools have different element size,
and wrong mempool info caused out of bound write.
Fix the issue by putting back objects to correct mempools.

Fixes: b3bbd9e5f265 ("cryptodev: support device independent sessions")
Fixes: 725d2a7fbf71 ("cryptodev: change queue pair configure structure")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: fix stats test

ut_setup / ut_teardown are invoked for each test case by test framework.
The call inside test_stats is unnecessary and even incorrect.
This caused double free of objects such as crypto operation structure.
Trapped the issue when RTE_LIBRTE_MEMPOOL_DEBUG was enabled.
Fix issue by removing ut_setup / ut_teardown from test case implementation.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Tested-by: Adam Dybkowski <adamx.dybkowski@intel.com>

crypto/aesni_mb: support AES-ECB

This patch adds AES-ECB 128, 192 and 256 support to the aesni_mb PMD.
AES-ECB 128, 192 and 256 test vectors added to cryptodev tests.

Signed-off-by: Marcel Cornu <marcel.d.cornu@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

baseband/turbo_sw: detect dependencies automatically

The meson for the turbo_sw PMD is updated to prevent the
requirement for any device specific toplevel flags to be
passed down (unlike what used to be the case with make).
The linking to the optional libraries is purely auto
detected at build time and flags are then set appropriately.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

doc: update bbdev guide

Clarify the capability assumptions for LLR and HARQ
compression format.
Correct one historical typo.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>

baseband/fpga_lte_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_lte_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead

Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

baseband/fpga_5gnr_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_5gnr_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead.

Fixes: 0b5927cbcba7 ("baseband/fpga_5gnr_fec: add PMD for FPGA 5GNR FEC")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

app/bbdev: fix test vector symlink

5G DL default symlink was pointing to a 4G vector.

Fixes: d762705308c4 ("app/bbdev: add test vectors for 5GNR")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>

crypto/scheduler: rename slave to worker

This patch replaces the usage of the word 'slave' with more
appropriate word 'worker' in QAT PMD and Scheduler PMD
as well as in their docs. Also the test app was modified
to use the new wording.

The Scheduler PMD's public API was modified according to the
previous deprecation notice:
rte_cryptodev_scheduler_slave_attach is now called
rte_cryptodev_scheduler_worker_attach,
rte_cryptodev_scheduler_slave_detach is
rte_cryptodev_scheduler_worker_detach,
rte_cryptodev_scheduler_slaves_get is
rte_cryptodev_scheduler_workers_get.

Also, the configuration value RTE_CRYPTODEV_SCHEDULER_MAX_NB_SLAVES
was renamed to RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: fix a null pointer dereference

This patch fixes a null pointer dereference after null check detected by
coverity scan.

Coverity issue: 349904
Fixes: 6a0c9d364afc ("crypto/dpaax_sec: support HFN override")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: replace NITROX PMD specific test suite

Replace NITROX PMD specific tests with generic test suite.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: remove orphan bbdev PMD feature table

Removing a feature table referring erroneously
to a PMD not present in DPDK.

Fixes: 65f1eec ("doc: add feature matrix table for bbdev")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/fpga_lte_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_lte_fec API is still experimental.
Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

baseband/fpga_5gnr_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_5gnr_fec API is still experimental.

Fixes: 2d4306438c92 ("baseband/fpga_5gnr_fec: add configure function")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

ipsec: promote library as stable

Since librte_ipsec was first introduced in 19.02 and there were no changes
in it's public API since 19.11, it should be considered mature enough to
remove the 'experimental' tag from it.
The RTE_SATP_LOG2_NUM enum is also being dropped from rte_ipsec_sa.h to
avoid possible ABI problems in the future.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/acc100: add configure function

Add configure function to configure the PF from within
the bbdev-test itself without external application
configuration the device.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add debug function to validate input

Debug functions to validate the input API from user
Only enabled in DEBUG mode at build time

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support interrupt

Adding capability and functions to support MSI
interrupts, call backs and inforing.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support 4G processing

Adding capability for 4G encode and decoder processing

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support HARQ loopback

Additional support for HARQ memory loopback

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add LDPC processing functions

Adding LDPC decode and encode processing operations

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add queue configuration

Adding function to create and configure queues for
the device. Still no capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add info get function

Add in the "info_get" function to the driver, to allow us to query the
device.
No processing capability are available yet.
Linking bbdev-test to support the PMD with null capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add HW register definitions

Add in the list of registers for the device and related
HW specs definitions.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: introduce PMD for ACC100

Add stubs for the ACC100 PMD

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

test/crypto: fix device number

In testsuite_setup(), ts_params is configured for first valid device.
The same device should be used as valid device in
test_device_configure_invalid_dev_id test case.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

app/test-sad: fix uninitialized variable

Coverity issue: 362055
Fixes: 908be0651a5a ("app/test-sad: add test application for IPsec SAD")
Cc: stable@dpdk.org
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

crypto/octeontx2: fix session-less mode

A temporary session is created for sessionless crypto operations.
rte_cryptodev_sym_session_create() should be used for creating the
temporary session as it initializes the session structure in the
correct way. Also the session should be set to 0 before freeing it.

Fixes: 17ac2a72191b ("crypto/octeontx2: add enqueue/dequeue ops")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

eal: simplify exit functions

The option RTE_EAL_ALWAYS_PANIC_ON_ERROR was off by default,
and not customizable with meson. It is completely removed.

The function rte_dump_registers is a trace of the bare metal support
era, and was not supported in userland. It is completely removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: David Marchand <david.marchand@redhat.com>

eal: add new prefetch write variants

This commit adds new rte_prefetchX_write() variants, that suggest to the
compiler to use a prefetch instruction with intention to write. As a
compiler builtin, the compiler can choose based on compilation target
what the best implementation for this instruction is.

Three versions are provided, targeting the different levels of cache.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

eal: fix build with conflicting libc variable memory_order

The cited commit introduced functions with 'int memory_order' argument.
The C11 standard section 7.17.1.4 defines 'memory_order' as the
"enumerated type whose enumerators identify memory ordering constraints".

A compilation error occurs:
error: declaration of 'memory_order' shadows a global declaration
    [-Werror=shadow]
     rte_atomic_thread_fence(int memory_order)

This issue was hit when trying to compile OVS with gcc 4.8.5. This
compiler version does not provide stdatomic.h, so enum memory_order is
redefined in OVS code.
In another case, if the compiler does provide stdatomic.h header,
passing -Wsystem-headers in the CFLAGS will also cause that failure.

Fix it by changing the argument name 'memory_order' to 'memorder'.

Fixes: 672a15056380 ("eal: add wrapper for C11 atomic thread fence")

Signed-off-by: Eli Britstein <elibr@nvidia.com>
Reviewed-by: Asaf Penso <asafp@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

acl: fix build with gcc 5.4.0

gcc 5.4 fails with:
../lib/librte_acl/acl_run_avx512x8.h: In function 'match_process_avx512x8':
../lib/librte_acl/acl_run_avx512x8.h:382:31: error:
pointer targets in passing argument 1 of '_mm256_mask_i32scatter_epi32'
differ in signedness [-Werror=pointer-sign]

Later gcc versions work fine, as for them parameter type was
changed to 'void *'.
Fixed by applying explicit cast for offending argument.

Bugzilla ID: 556
Fixes: b64c2295f7fc ("acl: add 256-bit AVX512 classify method")
Fixes: 45da22e42ec3 ("acl: add 512-bit AVX512 classify method")

Reported-by: Ali Alnubani <alialnu@nvidia.com>
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>

eal: add experimental tags for write combining store

Only marking the doxygen declarations is not enough.
Arch specific implementations must be tagged as well since there is no
common declaration of those inlines.

Fixes: 8a00dfc738fe ("eal: add write combining store")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Radu Nicolau <radu.nicolau@intel.com>

sched: remove redundant subport parameters

Remove redundant data structure fields.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

test/sched: update subport rate dynamically

Modify the test_sched application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

net/softnic: update subport rate dynamically

Modify the softnic drivers to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/ip_pipeline: update subport rate dynamically

Modify the ip_pipeline application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/qos_sched: update subport rate dynamically

Modify the qos_sched application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: update subport rate dynamically

Add support to update subport rate dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: introduce subport profile add function

API to add new subport bandwidth profile.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: add subport profile table

Add subport profile table to internal port data structure
and update the port config function.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/cmdline: build on Windows

Enable cmdline sample application as all dependencies are met.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: support Windows

Implement terminal handling, input polling, and vdprintf() for Windows.

Because Windows I/O model differs fundamentally from Unix and there is
no concept of character device, polling is simulated depending on the
underlying input device. Supporting non-terminal input is useful for
automated testing.

Windows emulation of VT100 uses "ESC [ E" for newline instead of
standard "ESC E", so add a workaround.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eal/windows: improve compatibility networking headers

Extend compatibility header system to support librte_cmdline.

pthread.h has to include windows.h, which exposes struct in_addr, etc.
conflicting with compatibility headers. WIN32_LEAN_AND_MEAN macro
is required to disable this behavior. Use rte_windows.h to define
WIN32_LEAN_AND_MEAN for pthread library.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrapper for vdprintf

Add internal wrapper for vdprintf(3) that is only available on Unix.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrappers for character input

poll(3) is a purely Unix facility, so it cannot be directly used by
common code. read(2) is limited in device support outside of Unix.
Create wrapper functions and implement them for Unix.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrappers for terminal handling

Add functions that set up, save, and restore terminal parameters.
Use existing code as Unix implementation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: make implementation logically opaque

struct cmdline exposes platform-specific members it contains, most
notably struct termios that is only available on Unix. While ABI
considerations prevent from hinding the definition on already supported
platforms, struct cmdline is considered logically opaque from now on.
Add a deprecation notice targeted at 20.11.

* Remove tests checking struct cmdline content as meaningless.

* Fix missing cmdline_free() in unit test.

* Add cmdline_get_rdline() to access history buffer indirectly.
  The new function is currently used only in tests.

Suggested-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eal/windows: implement alarm API

Implementation is based on waitable timers Win32 API. When timer is set,
a callback and its argument are supplied to the OS, while timer handle
is stored in EAL alarm list. When timer expires, OS wakes up the
interrupt thread and runs the callback. Upon completion it removes the
alarm.

Waitable timers must be set from the thread their callback will run in,
eal_intr_thread_schedule() provides a way to schedule asyncronuous code
execution in the interrupt thread. Alarm module builds synchronous timer
setup on top of it.

Windows alarms are not a type of DPDK interrupt handle and do not
interact with interrupt module beyond executing in the same thread.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

eal/windows: add interrupt thread skeleton

Windows interrupt support is based on IO completion ports (IOCP).
Interrupt thread would send the devices requests to notify about
interrupts and then wait for any request completion. Add skeleton code
of this model without any hardware support.

Another way to wake up the interrupt thread is APC (asynchronous procedure
call), scheduled by any other thread via eal_intr_thread_schedule().
This internal API is intended for alarm implementation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

bus/pci: support netuio on Windows

This patch adds implementations to probe PCI devices bound to netuio
with the help of "netuio" class device changes.
Now Windows will support both "netuio" and "net" device class and
can set kernel driver type based on the device class selection.

Note: Few definitions and structures have been copied from
netuio_interface.h file from
("[v5] windows/netuio: add Windows NetUIO kernel driver") series
and this will be fixed once the exact path for netuio source code is known.

Signed-off-by: John Alexander <john.alexander@datapath.co.uk>
Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Reviewed-by: Tal Shnaiderman <talshn@nvidia.com>
Reviewed-by: Narcisa Vasile <navasile@linux.microsoft.com>

table: fix hash for 32-bit

When create softnic hash table with 16 keys, it failed on 32-bit
environment, because the pointer field in structure rte_bucket_4_16
is only 32 bits. Add a padding field in 32-bit environment to keep
the structure to a multiple of 64 bytes. Apply this to 8-byte and
32-byte key hash function as well.

Fixes: 8aa327214c ("table: hash")
Cc: stable@dpdk.org
Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

acl: deduplicate AVX512 code

Current rte_acl_classify_avx512x32() and rte_acl_classify_avx512x16()
code paths are very similar. The only differences are due to
256/512 register/instrincts naming conventions.
So to deduplicate the code:
  - Move common code into “acl_run_avx512_common.h”
  - Use macros to hide difference in naming conventions

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>