crypto/qat: fix AES CMAC mininum digest size

AES-CMAC is used in 3gpp specifications hence it is needed
to support 4 byte digest.

Fixes: 91c1daa4e1f0 ("crypto/qat: support AES-CMAC")
Cc: stable@dpdk.org
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: add RSA cases in QAT and openssl

This commit adds RSA tests to Intel QuickAssist Technology pmd
and OpenSSL pmd test suite for session and session-less cases
when PADDING_NONE selected

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: add session-less cases in asym mod exp

This commit adds asymmetric sessionless tests to mod exp.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support RSA CRT in asym

This commit extends RSA implementation by CRT option

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support RSA in asym

This commit adds RSA algorithm to asymmetric pmd
using pair (n, d) private key

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support session-less for asym ops

This patch adds option to use asymmetric crypto pmd with
session-less support.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

doc: update doxygen for IPsec library

Added ipsec API from:
- rte_ipsec.h
- rte_ipsec_sa.h
- rte_ipsec_group.h
- rte_ipsec_sad.h

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

app/test-sad: add test application for IPsec SAD

Introduce new application to provide user to evaluate and perform
custom functional and performance tests for IPsec SAD implementation.

According to our measurements on SKX for 1M entries average lookup
cost is ~80 cycles, average add cost ~500 cycles.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

test/ipsec: add SAD autotests

add unittests for ipsec SAD library

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add SAD add/delete/lookup implementation

Replace rte_ipsec_sad_add(), rte_ipsec_sad_del() and
rte_ipsec_sad_lookup() stubs with actual implementation.

It uses three librte_hash tables each of which contains
an entries for a specific SA type (either it is addressed by SPI only
or SPI+DIP or SPI+DIP+SIP)

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add SAD create/destroy implementation

Replace rte_ipsec_sad_create(), rte_ipsec_sad_destroy() and
rte_ipsec_sad_find_existing() API stubs with actual
implementation.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add inbound SAD API

According to RFC 4301 IPSec implementation needs an inbound SA database
(SAD).
For each incoming inbound IPSec-protected packet (ESP or AH) it has to
perform a lookup within it's SAD.
Lookup should be performed by:
Security Parameters Index (SPI) + destination IP (DIP) + source IP (SIP)
or SPI + DIP
or SPI only
and an implementation has to return the 'longest' existing match.
This patch extend DPDK IPsec library with inbound security association
database (SAD) API implementation that:
- conforms to the RFC requirements above
- can scale up to millions of entries
- supports fast lookups
- supports incremental updates

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

crypto/qat: fix null auth when using VFIO

When running auth NULL cases while using
vfio_pci, DMAR read/write faults appear. It
happens even if digest_length is set to 0.
This is caused by auth_res_addr initialized
as 0x0.

Fixes: 4e0955bddb08 ("crypto/qat: fix null auth algo overwrite")
Cc: stable@dpdk.org
Signed-off-by: Damian Nowak <damianx.nowak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/aesni_gcm: allow device init if no AES-NI is present

The IPSec Multi buffer library does not require AES-NI
instructions to be supported by the CPU, as it can emulate these
instructions in software (adding a big performance penalty when
using AES algorithms).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: allow device init if no AES-NI is present

The IPSec Multi buffer library does not require AES-NI
instructions to be supported by the CPU, as it can emulate these
instructions in software (adding a big performance penalty when
using AES algorithms).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

common/dpaax: move shared sec HW code from dpaa2_sec

The SEC HW code is being shared by multiple NXP based
drivers. It is better to place it at a common place.
Current users are:
1. DPAA2_SEC
2. DPAA_SEC
3. CAAM_JR

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: fix list of unsupported features in IPsec guide

List of unsupported features doesn't reflect latest changes.

Fixes: cd5b860c1851 ("ipsec: support header construction")
Fixes: 2c1887fad075 ("ipsec: fix transport mode for IPv6 with extensions")

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

test/crypto: support asym cases for OCTEON TX2

This patch updates asymmetric crypto unit-test application to
validate asymmetric crypto operation supported by octeontx2 PMD.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: support asymmetric in enqueue/dequeue

This patch adds asymmetric support in enqueue/dequeue ops.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add asymmetric session

This patch adds asymmetric session setup and free routines.
RSA and modexp operations are supported.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add OCTEON TX2

This patch adds the OCTEON TX2 crypto validation tests.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add enqueue/dequeue ops

This patch adds the enqueue burst and dequeue
burst callbacks for the OCTEON TX2 crypto driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add session related functions

This patch adds the symmetric session related callbacks.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add symmetric capabilities

This patch adds the symmetric capabilities
for OCTEON TX2 crypto PMD.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add queue pair functions

This patch adds the queue pair setup and
queue pair release functions for OCTEON TX2 crypto pmd.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: support control ops

This patch adds the device control functions.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add init sequence in probe

This patch adds the device init sequence for
OCTEON TX2 crypto device.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add PMD skeleton

Adding OCTEON TX2 crypto PMD skeleton.
Enabling the driver by default in common_base.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/compress: add stateless overflow recovery case

Added unit test to check out-of-space recoverable feature.

Signed-off-by: Artur Trybula <arturx.trybula@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

compress/qat: catch overflow for stateless compression

This patch adds out-of-space recoverable feature for stateless
compression scenario to QuickAssist compressdev driver

Signed-off-by: Artur Trybula <arturx.trybula@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

doc: fix typo in l2fwd-crypto guide

Unmatched double quotation mark is fixed.

Fixes: ba7b86b1419b ("doc: add l2fwd-crypto sample app guide")
Cc: stable@dpdk.org
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: add --external-mbufs option in compress perf guide

This patch documents the new compress perf tool
option --external-mbufs.

Fixes: c02e33b03075 ("app/compress-perf: add --external-mbufs option")

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

cryptodev: fix checks related to device id

Each cryptodev are indexed with dev_id in the global rte_crypto_devices
variable. nb_devs is incremented / decremented each time a cryptodev is
created / deleted. The goal of nb_devs was to prevent the user to get an
invalid dev_id.

Let's imagine DPDK has configured N cryptodevs. If the cryptodev=1 is
removed at runtime, the latest cryptodev N cannot be accessible, because
nb_devs=N-1 with the current implementaion.

In order to prevent this kind of behavior, let's remove the check with
nb_devs and iterate in all the rte_crypto_devices elements: if data is
not NULL, that means a valid cryptodev is available.

Also, remove max_devs field and use RTE_CRYPTO_MAX_DEVS in order to
unify the code.

Fixes: d11b0f30df88 ("cryptodev: introduce API and framework for crypto devices")
Cc: stable@dpdk.org
Signed-off-by: Julien Meunier <julien.meunier@nokia.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: register octeontx to asym suite

Updated asymmetric crypto unit-test application to test
asymmetric crypto operations in octeontx PMD

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx: add asymmetric enqueue/dequeue ops

Add asymmetric crypto op enqueue & dequeue routines

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

common/cpt: add helper functions for asymmetric crypto

Add helper functions to get meta len for asymmetric operations

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx: add asymmetric session operations

Add asymmetric session setup and free functions. RSA and modexp
operations are supported.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx: add device type mailbox routine

Add mailbox communication to query symmetric or asymmetric device type

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

examples/fips_validation: separate ECB and CBC init

Separate initialization of IV, PT and CT according to TDES
ECB and CBC crypto modes

Signed-off-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Marko Kovacevic <marko.kovacevic@intel.com>

examples/fips_validation: support AES ECB

Signed-off-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Marko Kovacevic <marko.kovacevic@intel.com>

examples/fips_validation: support TDES ECB

Signed-off-by: Michael Shamis <michaelsh@marvell.com>
Reviewed-by: Marko Kovacevic <marko.kovacevic@intel.com>

crypto/dpaa_sec: reorganize session management

The session related parameters shall be populated during
the session create only.
At the runtime on first packet, the CDB should just reference
the session data instead of re-interpreting data again.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: enable SNOW3G and ZUC for dpaa

This patch add the SNOW and ZUC cipher only
and auth only test cases.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: support SNOW3G and ZUC

This patch add support for ZUC and SNOW 3G in
non-PDCP offload mode.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add case for auth only trailer

This patch add support for case when there is auth only
header and auth only tailroom present simultaneously.
This simulates the case of IPSEC ESN.

This patch also enable the new test case for openssl and
NXP dpaa_sec and dpaa2_sec platforms.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Vakul Garg <vakul.garg@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: increase cases for dpaax

dpaa_sec and dpaa2_sec testsuite are added with more
cases which are valid.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaax_sec: support auth trailer in cipher-auth

For cases like ESN where authentication data can be after cipher
data, support for authentication trailer is required.
This patch adds support of auth-only data trailing after cipher data.

Signed-off-by: Vakul Garg <vakul.garg@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaax_sec: enhance GCM descs to not skip AAD

The GCM descriptors needlessly skip auth_only_len bytes from output
buffer. Due to this, workarounds have to be made in dpseci driver code.
Also this leads to failing of one cryptodev test case for gcm. In this
patch, we change the descriptor construction and adjust dpaaX_sec
accordingly. The test_AES_GCM_auth_encrypt_SGL_out_of_place_400B_1seg
now passes.

Signed-off-by: Vakul Garg <vakul.garg@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: fix auth-cipher check for AEAD

The code shall also check aead as non auth-cipher case

Fixes: 1f14d500bce1 ("crypto/dpaa_sec: support IPsec protocol offload")
Cc: stable@dpdk.org
Signed-off-by: Vakul Garg <vakul.garg@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa2_sec: fix IPv6 PDB options

HW PDB Option was being overwritten.

Fixes: 53982ba2805d ("crypto/dpaa2_sec: support IPv6 tunnel for protocol offload")

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: fix PDCP session create

session_priv_mpool should be used instead of session pool

Fixes: d883e6e7131b ("test/crypto: add PDCP C-Plane encap cases")

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: add asymmetric session-less

This commit adds asymmetric session-less option to
rte_crypto_asym_op. Feature flag for session-less is added
to rte_cryptodev.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: fix release notes for removed testpmd commands

The notes were not visible in the generated doc output because
of an indentation mistake.
While fixing the indentation, the formatting is improved.

Fixes: e5db17a1e54e ("app/testpmd: remove duplicated Rx offload commands")
Cc: flavia.musatescu@intel.com
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David Marchand <david.marchand@redhat.com>

eal: remove dead code on NUMA node detection

RTE_EAL_ALLOW_INV_SOCKET_ID had been introduced and documented as used
with xen dom0 support (dropped for some time now).

Closely looking at this, the code was changed later and ensures that the
socket id is in the [0..RTE_MAX_NUMA_NODES] range anyway.

Let's drop this dead code and the build option with it.

Fixes: 94ef2964148a ("eal/linux: fix numa node detection")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

eal/linux: restore specific hugepage ordering for ppc

An ifdef present in eal_memory.c references "RTE_ARCH_PPC64" when
it should actually use "RTE_ARCH_PPC_64".  Simple testing revealed
that both the PPC_64 and non-PPC_64 versions of the code involved
work, but the PPC_64 version of the code is retained to be
consistent with other instances in the same file where mmapped
memory is accessed in reverse order on Power platforms.

Fixes: 66cc45e293ed ("mem: replace memseg with memseg lists")
Cc: stable@dpdk.org
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

test/mempool: add unit test cases

Added UT to cover below functions:
- rte_mempool_populate_anon(),
- rte_mempool_mem_iter(),

Those additions also cover mempool internals:
rte_mempool_memchunk_anon_free() and get_anon_size().

Signed-off-by: Pallantla Poornima <pallantlax.poornima@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

test/lpm: fix measured cycles for delete

total_time needs to be reset to measure the cycles for delete API.

Fixes: af75078fece3 ("first public release")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Tested-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

test/bpf: add new test cases

This patch implements following new test cases:
- test_call4: test case to verify if stack corruption occurs
across with multiple function calls.
- test_jump2: test case with a default packet in memory, parse
the packet and check if dest ip is part of a subnet.
- test_call5: test case with string data in the stack and
calling string comaprision.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

test/ring: enhance mp/mc coverage

Run ring perf test on all available cores to really verify MPMC operations.
The old way of running on a pair of cores is not enough for MPMC rings.

Suggested-by: Gavin Hu <gavin.hu@arm.com>
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

test/mbuf: add unit test on mbuf flag names

Added UT for the below functions:
- rte_get_rx_ol_flag_list,
- rte_get_tx_ol_flag_list,
- rte_get_rx_ol_flag_name,
- rte_get_tx_ol_flag_name

Signed-off-by: Pallantla Poornima <pallantlax.poornima@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

test/mbuf: add unit test cases

Added new unit test cases to cover the below functions:
- rte_validate_tx_offload,
- rte_pktmbuf_alloc_bulk,
- rte_pktmbuf_read,
- rte_pktmbuf_ext_shinfo_init_helper,
- rte_pktmbuf_attach_extbuf,
- rte_mbuf_ext_refcnt_read,
- rte_mbuf_ext_refcnt_update,
- rte_mbuf_ext_refcnt_set,
- rte_pktmbuf_detach_extbuf

Signed-off-by: Lavanya Govindarajan <lavanyax.govindarajan@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

test/mbuf: add case for bulk alloc/free functions

Add unit test for functions for allocating and freeing a bulk of mbufs.

Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: add bulk free function

Add function for freeing a bulk of mbufs.

Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

config: set KNI preempt default with meson

Same behavior as in make build system.

Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

drivers: remove duplicated compiler flags

Now that -Wextra, and other warning flags are standard in the build, remove
any duplication in driver build specifications.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>

build: enable extra warnings with meson

While meson always adds -Wall flag to C compiles, the make build adds extra
warning flags that are not present in the meson build. This addresses that
shortcoming by adding additional warning flags to our builds. The one
omission is the -Wcast-align flag, which though present in make gcc builds,
gives a lot of warnings/errors when used with clang.

The removed warning "-Wunused-parameter" is covered by the "-Wextra"
parameter so is unnecessary.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>

raw/ifpga: remove unneeded compiler flags

The warning disable flags for the base driver code were copy-pasted from
another source, and are actually unnecessary for this driver. Therefore
remove them.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>

build: allow stricter fallthrough warnings

DPDK currently compiles with implicit-fallthrough=2 warning level. With gcc
-Wextra flag, the default level is 3, so some minor changes are needed to
support this in DPDK.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>

build: support disabling drivers with meson

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

build: process dependencies before main build check

If we want to add support for turning off components because of missing
dependencies, then we need to check for those dependencies before we
make a determination as to whether a component should be built or not,
assuming that the component says it should be built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

build: align variable names between drivers and libs

The variable names in the library and drivers meson.build files are slighty
different with "static_deps" in one and "static_objs" in the other. Rename
to use "static_deps" in both for consistency.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

build: print out dependency names for clarity

To help developers to get the correct dependency name e.g. when creating a
new example that depends on a specific component, print out the dependency
name for each lib/driver as it is processed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

examples/vm_power: fix build

Fixes: 70febdcfd60f ("examples: check status of getting MAC address")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

event/dpaa2: support Tx adapter

This patch adds the support of Tx adapter for DPAA2 platform

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa2: set priority as per DPCON device

This patch sets the priority of the dpcon dev, such that it is
within the supported range of dpcon

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa: support Tx adapter

This patch adds the support of Tx adapter for DPAA1 platform

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa: fix number of supported atomic flows

The number of atomic flows supported was not returned correctly for
DPAA driver. This patch fixes the same.

Fixes: b08dc6430abd ("event/dpaa: add queue config get/set")
Cc: stable@dpdk.org
Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

eventdev: add Tx flag for packets with same destination

This patch introduces a `flag` in the Eth TX adapter enqueue API.
Some drivers may support burst functionality only with the packets
having same destination device and queue.

The flag `RTE_EVENT_ETH_TX_ADAPTER_ENQUEUE_SAME_DEST` can be used
to indicate this so the underlying driver, for drivers to utilize
burst functionality appropriately.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

rcu: fix reference to offline function

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

rcu: update QS only when there are updates from writer

When the writer is checking the quiescent state status, it is not
deleting any entries in the data structure. This means, the readers
do not need to update their quiescent state during that period.
Readers update the quiescent state only when there are updates
available from the writer.

Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

rcu: add least acknowledged token optimization

When the rte_rcu_qsbr_check API is called, it is possible to
calculate the least valued token acknowledged by all the readers.
When the API is called next time, the readers' token counters do
not need to be scanned if the value of the token being queried is
less than the last least token acknowledged. This avoids the
cache line bounces between readers and writer.

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

test/rcu: enhance log nomenclature

Use 'quiescent state updates' instead of just 'updates'.

Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

test/rcu: fix memory size integer truncate

Variables used to store the return value of rte_rcu_qsbr_get_memsize
in variables of type 'int' or 'uint32_t'.
While the former variables are a problem, the latter have been aligned
for consistency.
The variables are of type 'size_t' now.

Fixes: b87089b0bb19 ("test/rcu: add API and functional tests")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

doc: describe token and resource retention in RCU guide

After calling rte_rcu_qsbr_start API, the token and the deleted
resource need to be stored for subsequent query/free.

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

doc: remove wrong limitation in RCU guide

There is no limitation of 1024 reader threads.

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

doc: proofread RCU guide

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>

examples/ntb: fix build

lcore indexes can't be represented with uint8_t.
Seen on armv8 target:

examples/ntb/ntb_fwd.c: In function ‘cmd_quit_parsed’:
arm64-armv8a-linux-gcc/include/rte_lcore.h:296:8:
error: comparison is always true due to limited range of data type
[-Werror=type-limits]
       i<RTE_MAX_LCORE;      \
        ^
examples/ntb/ntb_fwd.c:164:2: note: in expansion of
macro ‘RTE_LCORE_FOREACH_SLAVE’
  RTE_LCORE_FOREACH_SLAVE(lcore_id) {
  ^~~~~~~~~~~~~~~~~~~~~~~

Fixes: 5194299d6ef5 ("examples/ntb: support more functions")

Signed-off-by: David Marchand <david.marchand@redhat.com>

devtools: fix cleanup of checkpatch temporary file

The regexp part of the cleanup routine was not updated accordingly when
a common prefix for temp files was introduced.
Set the trap handler only when required.

Fixes: ff37ca5d3773 ("devtools: use a common prefix for temporary files")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>

clean bare metal support traces

Bare metal support has been gone for quite some time but we still had
some checks on system includes.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

examples/ntb: support more functions

Support to transmit files between two systems.
Support iofwd between one ethdev and NTB device.
Support rxonly and txonly for NTB device.
Support to set forwarding mode as file-trans, txonly,
rxonly or iofwd.
Support to show/clear port stats and throughput.

Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

raw/ntb: add enqueue and dequeue functions

Introduce enqueue and dequeue functions to support packet based
processing. And enable write-combining for ntb driver since it
can improve the performance a lot.

Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

raw/ntb: support xstats

Add xstats support for NTB rawdev.
Support tx-packets, tx-bytes, tx-errors and
rx-packets, rx-bytes, rx-missed.

Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

raw/ntb: setup queues

Setup and init NTB txq and rxq. And negotiate queue information
with the peer. If queue size and number of queues are not
consistent on both sides, return error.

Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

stack: enable lock-free implementation for aarch64

Enable both C11 atomic and non C11 atomic lock-free stack for aarch64.

Introduced a new header to reduce the ifdef clutter across generic and C11
files. The rte_stack_lf_stubs.h contains stub implementations of
__rte_stack_lf_count, __rte_stack_lf_push_elems and
__rte_stack_lf_pop_elems.

Suggested-by: Gage Eads <gage.eads@intel.com>
Suggested-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Tested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

test/atomic: add 128-bit atomic compare exchange test

Add 128-bit atomic compare and swap test for aarch64 and x86_64.
Extend the test iteration from 10 thousand to 1 million times to test
the stability of the atomic APIs.

Signed-off-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Gage Eads <gage.eads@intel.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Tested-by: Jerin Jacob <jerinj@marvell.com>

eal/arm64: add 128-bit atomic compare exchange

This patch adds the implementation of the 128-bit atomic compare
exchange API on aarch64. Using 64-bit 'ldxp/stxp' instructions
can perform this operation. Moreover, on the LSE atomic extension
accelerated platforms, it is implemented by 'casp' instructions for
better performance.

Since the '__ARM_FEATURE_ATOMICS' flag only supports GCC-9, this
patch adds a new config flag 'RTE_ARM_FEATURE_ATOMICS' to enable
the 'cas' version on older version compilers.
For octeontx2, we make sure that the lse (and other) extensions are
enabled even if the compiler does not know of the octeontx2 target
cpu.

Since direct x0 register used in the code and cas_op_name() and
rte_atomic128_cmp_exchange() is inline function, based on parent
function load, it may corrupt x0 register aka break aarch64 ABI.
Define CAS operations as rte_noinline functions to avoid an ABI
break [1].

1: https://git.dpdk.org/dpdk/commit/?id=5b40ec6b9662

Suggested-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Tested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

eal: calibrate TSC only in primary process

This ensures secondary processes never have to calculate the TSC rate
themselves, which can be noticeable in VMs that don't have access to
arch-specific detection mechanism (such as CPUID leaf 0x15 or MSR 0xCE
on x86).

Since rte_mem_config is now internal to the EAL library, we can add
tsc_hz without ABI breakage concerns.

Reduces rte_eal_init() execution time in a secondary process from 165ms
to 66ms on my test system.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

rcu: fix spurious thread unregister

Thread unregister returns success while unregister not been performed.
This is due to incorrect thread registration status check.
Fix this issue by correcting bitmap check.

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

service: avoid false sharing on core state

For a valid service, the core mask of the service
is checked against the current core and the corresponding
entry in the active_on_lcore array is set or reset.

Upto 8 cores share the same cache line for their
service active_on_lcore array entries since each entry is a uint8_t.
Some number of these entries also share the cache line with
the internal_flags member of struct rte_service_spec_impl,
hence this false sharing also makes the service_valid() check
expensive.

Eliminate false sharing by moving the active_on_lcore array to
a per-core data structure. The array is now indexed by service id.

Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Acked-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

timer: remove useless check on x86 TSC reliability

This code was added 7+ years ago in
commit fb022b85bae4 ("timer: check TSC reliability")
presumably when variant TSCs were still somewhat common.

But this code doesn't do anything except print a warning,
and the warning doesn't give any kind of advice to the user,
so let's just remove it.

While the warning has no functional meaning, the /proc/cpuinfo
parsing consumes a non-trivial amount of time which is especially
noticeable in secondary processes.
On my test system, it consumes 21ms out of the 66ms total execution
time for rte_eal_init() in a secondary process.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>