crypto/bcmfs: support VFIO

Add VFIO support for BCMFS PMD.
The BCMFS PMD functionality is dependent on the VFIO_PRESENT flag,
which gets enabled in the rte_vfio.h.
If this flag is not enabled in the compiling platform driver will
silently return with error, when executed.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/bcmfs: introduce BCMFS driver

Add Broadcom FlexSparc(FS) device creation driver which registers to a
vdev and create a device. Add APIs for logs, supportive documentation and
maintainers file.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support KASUMI F8/F9

Add support for KASUMI-F8/F9 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support SNOW3G-UEA2/UIA2

Add support for SNOW3G-UEA2/UIA2 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_mb: support ZUC-EEA3/EIA3

Add support for ZUC-EEA3/EIA3 algorithms through the intel-ipsec-mb
job API, allowing the mix of these algorithms with others.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

common/cpt: check MAC length

HMAC/HASH opcode algorithms supports fixed mac length.
Allowed session creation to fail when requested for
unsupported MAC length for HMAC/HASH-only use cases.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/cpt: remove useless macros

The macros can be replaced with actual constants.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/nitrox: support cipher-only operations

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/nitrox: support AES-GCM

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>

crypto/octeontx2: support lookaside IPsec IPv6

Adding IPv6 tunnel mode support in lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

crypto/octeontx2: check cpt kernel driver version

This patch checks if cpt pmd is compatible with kernel
cpt driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

common/octeontx2: sync cpt mailbox

This patch syncs the pmd mailbox with the cpt pf driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

test/crypto: replace armv8 test suite

Switched from device specific test suite to unified
cryptodev test suite. Removed the armv8 device specific test suite.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/armv8: fix mempool object returning

Crypto session and device session private data were put back
to wrong pools.
This caused data corruption when the object was reallocated and used.
Because objects from different mempools have different element size,
and wrong mempool info caused out of bound write.
Fix the issue by putting back objects to correct mempools.

Fixes: b3bbd9e5f265 ("cryptodev: support device independent sessions")
Fixes: 725d2a7fbf71 ("cryptodev: change queue pair configure structure")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: fix stats test

ut_setup / ut_teardown are invoked for each test case by test framework.
The call inside test_stats is unnecessary and even incorrect.
This caused double free of objects such as crypto operation structure.
Trapped the issue when RTE_LIBRTE_MEMPOOL_DEBUG was enabled.
Fix issue by removing ut_setup / ut_teardown from test case implementation.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Tested-by: Adam Dybkowski <adamx.dybkowski@intel.com>

crypto/aesni_mb: support AES-ECB

This patch adds AES-ECB 128, 192 and 256 support to the aesni_mb PMD.
AES-ECB 128, 192 and 256 test vectors added to cryptodev tests.

Signed-off-by: Marcel Cornu <marcel.d.cornu@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

baseband/turbo_sw: detect dependencies automatically

The meson for the turbo_sw PMD is updated to prevent the
requirement for any device specific toplevel flags to be
passed down (unlike what used to be the case with make).
The linking to the optional libraries is purely auto
detected at build time and flags are then set appropriately.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

doc: update bbdev guide

Clarify the capability assumptions for LLR and HARQ
compression format.
Correct one historical typo.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>

baseband/fpga_lte_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_lte_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead

Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

baseband/fpga_5gnr_fec: fix crash with debug

When RTE_LIBRTE_BBDEV_DEBUG is enabled, rte_device's driver
pointer is dereferenced twice in fpga_5gnr_fec's probe callback.
It causes a segmentation fault because this pointer is only
assigned after probe callback call.

This patch makes use of rte_pci_driver pointer instead.

Fixes: 0b5927cbcba7 ("baseband/fpga_5gnr_fec: add PMD for FPGA 5GNR FEC")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

app/bbdev: fix test vector symlink

5G DL default symlink was pointing to a 4G vector.

Fixes: d762705308c4 ("app/bbdev: add test vectors for 5GNR")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>

crypto/scheduler: rename slave to worker

This patch replaces the usage of the word 'slave' with more
appropriate word 'worker' in QAT PMD and Scheduler PMD
as well as in their docs. Also the test app was modified
to use the new wording.

The Scheduler PMD's public API was modified according to the
previous deprecation notice:
rte_cryptodev_scheduler_slave_attach is now called
rte_cryptodev_scheduler_worker_attach,
rte_cryptodev_scheduler_slave_detach is
rte_cryptodev_scheduler_worker_detach,
rte_cryptodev_scheduler_slaves_get is
rte_cryptodev_scheduler_workers_get.

Also, the configuration value RTE_CRYPTODEV_SCHEDULER_MAX_NB_SLAVES
was renamed to RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaa_sec: fix a null pointer dereference

This patch fixes a null pointer dereference after null check detected by
coverity scan.

Coverity issue: 349904
Fixes: 6a0c9d364afc ("crypto/dpaax_sec: support HFN override")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: replace NITROX PMD specific test suite

Replace NITROX PMD specific tests with generic test suite.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: remove orphan bbdev PMD feature table

Removing a feature table referring erroneously
to a PMD not present in DPDK.

Fixes: 65f1eec ("doc: add feature matrix table for bbdev")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/fpga_lte_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_lte_fec API is still experimental.
Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

baseband/fpga_5gnr_fec: fix API naming

DPDK APIs have to be prefixed with "rte_" in order to avoid
namespace pollution.

Let's fix it while fpga_5gnr_fec API is still experimental.

Fixes: 2d4306438c92 ("baseband/fpga_5gnr_fec: add configure function")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tom Rix <trix@redhat.com>

ipsec: promote library as stable

Since librte_ipsec was first introduced in 19.02 and there were no changes
in it's public API since 19.11, it should be considered mature enough to
remove the 'experimental' tag from it.
The RTE_SATP_LOG2_NUM enum is also being dropped from rte_ipsec_sa.h to
avoid possible ABI problems in the future.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/acc100: add configure function

Add configure function to configure the PF from within
the bbdev-test itself without external application
configuration the device.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add debug function to validate input

Debug functions to validate the input API from user
Only enabled in DEBUG mode at build time

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support interrupt

Adding capability and functions to support MSI
interrupts, call backs and inforing.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support 4G processing

Adding capability for 4G encode and decoder processing

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: support HARQ loopback

Additional support for HARQ memory loopback

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add LDPC processing functions

Adding LDPC decode and encode processing operations

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add queue configuration

Adding function to create and configure queues for
the device. Still no capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add info get function

Add in the "info_get" function to the driver, to allow us to query the
device.
No processing capability are available yet.
Linking bbdev-test to support the PMD with null capability.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: add HW register definitions

Add in the list of registers for the device and related
HW specs definitions.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

baseband/acc100: introduce PMD for ACC100

Add stubs for the ACC100 PMD

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

test/crypto: fix device number

In testsuite_setup(), ts_params is configured for first valid device.
The same device should be used as valid device in
test_device_configure_invalid_dev_id test case.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

app/test-sad: fix uninitialized variable

Coverity issue: 362055
Fixes: 908be0651a5a ("app/test-sad: add test application for IPsec SAD")
Cc: stable@dpdk.org
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

crypto/octeontx2: fix session-less mode

A temporary session is created for sessionless crypto operations.
rte_cryptodev_sym_session_create() should be used for creating the
temporary session as it initializes the session structure in the
correct way. Also the session should be set to 0 before freeing it.

Fixes: 17ac2a72191b ("crypto/octeontx2: add enqueue/dequeue ops")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

eal: simplify exit functions

The option RTE_EAL_ALWAYS_PANIC_ON_ERROR was off by default,
and not customizable with meson. It is completely removed.

The function rte_dump_registers is a trace of the bare metal support
era, and was not supported in userland. It is completely removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: David Marchand <david.marchand@redhat.com>

eal: add new prefetch write variants

This commit adds new rte_prefetchX_write() variants, that suggest to the
compiler to use a prefetch instruction with intention to write. As a
compiler builtin, the compiler can choose based on compilation target
what the best implementation for this instruction is.

Three versions are provided, targeting the different levels of cache.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

eal: fix build with conflicting libc variable memory_order

The cited commit introduced functions with 'int memory_order' argument.
The C11 standard section 7.17.1.4 defines 'memory_order' as the
"enumerated type whose enumerators identify memory ordering constraints".

A compilation error occurs:
error: declaration of 'memory_order' shadows a global declaration
    [-Werror=shadow]
     rte_atomic_thread_fence(int memory_order)

This issue was hit when trying to compile OVS with gcc 4.8.5. This
compiler version does not provide stdatomic.h, so enum memory_order is
redefined in OVS code.
In another case, if the compiler does provide stdatomic.h header,
passing -Wsystem-headers in the CFLAGS will also cause that failure.

Fix it by changing the argument name 'memory_order' to 'memorder'.

Fixes: 672a15056380 ("eal: add wrapper for C11 atomic thread fence")

Signed-off-by: Eli Britstein <elibr@nvidia.com>
Reviewed-by: Asaf Penso <asafp@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

acl: fix build with gcc 5.4.0

gcc 5.4 fails with:
../lib/librte_acl/acl_run_avx512x8.h: In function 'match_process_avx512x8':
../lib/librte_acl/acl_run_avx512x8.h:382:31: error:
pointer targets in passing argument 1 of '_mm256_mask_i32scatter_epi32'
differ in signedness [-Werror=pointer-sign]

Later gcc versions work fine, as for them parameter type was
changed to 'void *'.
Fixed by applying explicit cast for offending argument.

Bugzilla ID: 556
Fixes: b64c2295f7fc ("acl: add 256-bit AVX512 classify method")
Fixes: 45da22e42ec3 ("acl: add 512-bit AVX512 classify method")

Reported-by: Ali Alnubani <alialnu@nvidia.com>
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>

eal: add experimental tags for write combining store

Only marking the doxygen declarations is not enough.
Arch specific implementations must be tagged as well since there is no
common declaration of those inlines.

Fixes: 8a00dfc738fe ("eal: add write combining store")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Radu Nicolau <radu.nicolau@intel.com>

sched: remove redundant subport parameters

Remove redundant data structure fields.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

test/sched: update subport rate dynamically

Modify the test_sched application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

net/softnic: update subport rate dynamically

Modify the softnic drivers to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/ip_pipeline: update subport rate dynamically

Modify the ip_pipeline application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/qos_sched: update subport rate dynamically

Modify the qos_sched application to build the hierarchical scheduler
with default subport bandwidth profile. It also allows to update
a subport with different subport rates dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: update subport rate dynamically

Add support to update subport rate dynamically.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: introduce subport profile add function

API to add new subport bandwidth profile.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: add subport profile table

Add subport profile table to internal port data structure
and update the port config function.

Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

examples/cmdline: build on Windows

Enable cmdline sample application as all dependencies are met.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: support Windows

Implement terminal handling, input polling, and vdprintf() for Windows.

Because Windows I/O model differs fundamentally from Unix and there is
no concept of character device, polling is simulated depending on the
underlying input device. Supporting non-terminal input is useful for
automated testing.

Windows emulation of VT100 uses "ESC [ E" for newline instead of
standard "ESC E", so add a workaround.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eal/windows: improve compatibility networking headers

Extend compatibility header system to support librte_cmdline.

pthread.h has to include windows.h, which exposes struct in_addr, etc.
conflicting with compatibility headers. WIN32_LEAN_AND_MEAN macro
is required to disable this behavior. Use rte_windows.h to define
WIN32_LEAN_AND_MEAN for pthread library.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrapper for vdprintf

Add internal wrapper for vdprintf(3) that is only available on Unix.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrappers for character input

poll(3) is a purely Unix facility, so it cannot be directly used by
common code. read(2) is limited in device support outside of Unix.
Create wrapper functions and implement them for Unix.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: add internal wrappers for terminal handling

Add functions that set up, save, and restore terminal parameters.
Use existing code as Unix implementation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

cmdline: make implementation logically opaque

struct cmdline exposes platform-specific members it contains, most
notably struct termios that is only available on Unix. While ABI
considerations prevent from hinding the definition on already supported
platforms, struct cmdline is considered logically opaque from now on.
Add a deprecation notice targeted at 20.11.

* Remove tests checking struct cmdline content as meaningless.

* Fix missing cmdline_free() in unit test.

* Add cmdline_get_rdline() to access history buffer indirectly.
  The new function is currently used only in tests.

Suggested-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eal/windows: implement alarm API

Implementation is based on waitable timers Win32 API. When timer is set,
a callback and its argument are supplied to the OS, while timer handle
is stored in EAL alarm list. When timer expires, OS wakes up the
interrupt thread and runs the callback. Upon completion it removes the
alarm.

Waitable timers must be set from the thread their callback will run in,
eal_intr_thread_schedule() provides a way to schedule asyncronuous code
execution in the interrupt thread. Alarm module builds synchronous timer
setup on top of it.

Windows alarms are not a type of DPDK interrupt handle and do not
interact with interrupt module beyond executing in the same thread.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

eal/windows: add interrupt thread skeleton

Windows interrupt support is based on IO completion ports (IOCP).
Interrupt thread would send the devices requests to notify about
interrupts and then wait for any request completion. Add skeleton code
of this model without any hardware support.

Another way to wake up the interrupt thread is APC (asynchronous procedure
call), scheduled by any other thread via eal_intr_thread_schedule().
This internal API is intended for alarm implementation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

bus/pci: support netuio on Windows

This patch adds implementations to probe PCI devices bound to netuio
with the help of "netuio" class device changes.
Now Windows will support both "netuio" and "net" device class and
can set kernel driver type based on the device class selection.

Note: Few definitions and structures have been copied from
netuio_interface.h file from
("[v5] windows/netuio: add Windows NetUIO kernel driver") series
and this will be fixed once the exact path for netuio source code is known.

Signed-off-by: John Alexander <john.alexander@datapath.co.uk>
Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Reviewed-by: Tal Shnaiderman <talshn@nvidia.com>
Reviewed-by: Narcisa Vasile <navasile@linux.microsoft.com>

table: fix hash for 32-bit

When create softnic hash table with 16 keys, it failed on 32-bit
environment, because the pointer field in structure rte_bucket_4_16
is only 32 bits. Add a padding field in 32-bit environment to keep
the structure to a multiple of 64 bytes. Apply this to 8-byte and
32-byte key hash function as well.

Fixes: 8aa327214c ("table: hash")
Cc: stable@dpdk.org
Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

acl: deduplicate AVX512 code

Current rte_acl_classify_avx512x32() and rte_acl_classify_avx512x16()
code paths are very similar. The only differences are due to
256/512 register/instrincts naming conventions.
So to deduplicate the code:
  - Move common code into “acl_run_avx512_common.h”
  - Use macros to hide difference in naming conventions

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: optimize AVX512 classify with 4 bytes loads

With current ACL implementation first field in the rule definition
has always to be one byte long. Though for optimising classify
implementation it might be useful to do 4B reads
(as we do for rest of the fields).
So at build phase, check user provided field definitions to determine
is it safe to do 4B loads for first ACL field.
Then at run-time this information can be used to choose classify
behavior.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: add 512-bit AVX512 classify method

Introduce classify implementation that uses AVX512 specific ISA.
rte_acl_classify_avx512x32() is able to process up to 32 flows in parallel.
It uses 512-bit width registers/instructions and provides higher
performance then rte_acl_classify_avx512x16(), but can cause
frequency level change.
Note that for now only 64-bit version is supported.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: select 256-bit AVX512 classify method by default

On supported platforms, set RTE_ACL_CLASSIFY_AVX512X16 as
default ACL classify algorithm.
Note that AVX512X16 implementation uses 256-bit registers/instincts only
to avoid possibility of frequency drop.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: add 256-bit AVX512 classify method

Introduce classify implementation that uses AVX512 specific ISA.
rte_acl_classify_avx512x16() is able to process up to 16 flows in parallel.
It uses 256-bit width registers/instructions only
(to avoid frequency level change).
Note that for now only 64-bit version is supported.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: add infrastructure for AVX512 classify methods

Add necessary changes to support new AVX512 specific ACL classify
algorithm:
 - changes in meson.build to check that build tools
   (compiler, assembler, etc.) do properly support AVX512.
 - run-time checks to make sure target platform does support AVX512.
 - dummy rte_acl_classify_avx512() for targets where AVX512
   implementation couldn't be properly supported.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

test/acl: expand classify test coverage

Make classify test to run for all supported methods.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

app/acl: enhance displayed statistics

- enhance output to print extra stats
- use rte_rdtsc_precise() for cycle measurements

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: rework classify method selection

Right now ACL library determines best possible (default) classify method
on a given platform with special constructor function rte_acl_init().
This patch makes the following changes:
 - Move selection of default classify method into a separate private
   function and call it for each ACL context creation (rte_acl_create()).
 - Remove library constructor function
 - Make rte_acl_set_ctx_classify() to check that requested algorithm
   is supported on given platform.

The purpose of these changes to improve and simplify algorithm selection
process and prepare ACL library to be integrated with the
max SIMD bitwidth series in discussion.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: remove classify methods count enum

Removal of unused enum value (RTE_ACL_CLASSIFY_NUM).
This enum value is not used inside DPDK, while it prevents
to add new classify algorithms without causing an ABI breakage.

Note that this change introduce a formal ABI incompatibility
with previous versions of ACL library.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

doc: fix missing classify methods in ACL guide

Add brief description for missing ACL classify algorithms:
RTE_ACL_CLASSIFY_NEON and RTE_ACL_CLASSIFY_ALTIVEC.

Fixes: 34fa6c27c156 ("acl: add NEON optimization for ARMv8")
Fixes: 1d73135f9f1c ("acl: add AltiVec for ppc64")
Cc: stable@dpdk.org
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

acl: fix x86 build for compiler without AVX2

Right now we define dummy version of rte_acl_classify_avx2()
when both X86 and AVX2 are not detected, though it should be
for non-AVX2 case only.

Fixes: e53ce4e41379 ("acl: remove use of weak functions")
Cc: stable@dpdk.org
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

eal/x86: introduce type for AVX 512-bit

New data type to manipulate 512 bit AVX values.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

eal/windows: export all built functions for clang

export for clang build all the functions currently built
on Windows and listed in rte_eal_version.map by adding
them to rte_eal_exports.def.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

bus/pci: support segment as address domain on Windows

Set the domain value for rte_pci_addr probing on Windows
to the value of the PCI segment returned by SPDRP_BUSNUMBER.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>

usertools: add OCTEON TX2 REE device binding

Update the devbind script with new section of regex devices, also
added OCTEONTX2 REE device ID to regex device list

Signed-off-by: Guy Kaneti <guyk@marvell.com>

regex/octeontx2: introduce REE driver

Add meson based build infrastructure along with the
OTX2 regexdev (REE) device functions.
Add Marvell OCTEON TX2 regex guide.

Signed-off-by: Guy Kaneti <guyk@marvell.com>

common/octeontx2: add REE definitions and logging

Add REE mbox msg definitions, RVU and REE HW definitions

Signed-off-by: Guy Kaneti <guyk@marvell.com>

bus/pci: copy new id for inserted device on Linux

When a device is inserted into an existing BDF slot
that has not been probed, we must overwrite the old
PCI ID with the ID of the new function. Otherwise
we may not probe the function with the correct driver,
if at all.

Signed-off-by: Jim Harris <james.r.harris@intel.com>

net: add CRC AVX512 implementation

This patch enables the optimized calculation of CRC32-Ethernet and
CRC16-CCITT using the AVX512 and VPCLMULQDQ instruction sets. This CRC
implementation is built if the compiler supports the required instruction
sets. It is selected at run-time if the host CPU, again, supports the
required instruction sets.

Signed-off-by: Mairtin o Loingsigh <mairtin.oloingsigh@intel.com>
Signed-off-by: David Coyle <david.coyle@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Jasvinder Singh <jasvinder.singh@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

net: add CRC implementation runtime selection

This patch adds support for run-time selection of the optimal
architecture-specific CRC path, based on the supported instruction set(s)
of the CPU.

The compiler option checks have been moved from the C files to the meson
script. The rte_cpu_get_flag_enabled function is called automatically by
the library at process initialization time to determine which
instructions the CPU supports, with the most optimal supported CRC path
ultimately selected.

Signed-off-by: Mairtin o Loingsigh <mairtin.oloingsigh@intel.com>
Signed-off-by: David Coyle <david.coyle@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Jasvinder Singh <jasvinder.singh@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

test/cpuflags: add new Arm flags

This patch adds new flags into the test_cpuflags() functions for ARM64
platform, such as RTE_CPUFLAG_SVE, etc.

Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

eal/arm64: update CPU flags

ARM64 Linux kernel updated the CPU flags using the HWCAP scheme.
The related marco definition can be found in linux kernel:
  arch/arm64/include/uapi/asm/hwcap.h

This patch incorporates those changes to the EAL library.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

config: remap flags used for Arm platforms

RTE_ARCH_xx flags are used to distinguish platform architectures.
These flags can be used to pick different code paths for different
architectures at compile time.
For Arm platforms, there are 3 flags in use: RTE_ARCH_ARM,
RTE_ARCH_ARMv7 and RTE_ARCH_ARM64.
RTE_ARCH_ARM64 is for 64-bit aarch64 platforms,
and RTE_ARCH_ARM & RTE_ARCH_ARMv7 are for 32-bit platforms.
RTE_ARCH_ARMv7 is for ARMv7 platforms as its name suggested.

The issue is meaning of RTE_ARCH_ARM is not clear enough.
Because no info about platform word length is included in the name.
To make the flag names more clear, a naming scheme is proposed.

RTE_ARCH_ARM (all Arm platforms)
    |
    +----RTE_ARCH_32 (New. 32-bit platforms of all architectures)
    |        |
    |        +----RTE_ARCH_ARMv7 (ARMv7 platforms)
    |        |
    |        +----RTE_ARCH_ARMv8_AARCH32 (aarch32 state on aarch64 machine)
    |
    +----RTE_ARCH_64 (64-bit platforms of all architectures)
             |
             +----RTE_ARCH_ARM64 (64-bit Arm platforms)

RTE_ARCH_32 will be explicitly defined for 32-bit platforms.

To fit into the new naming scheme, current usage of RTE_ARCH_ARM in
project is mapped to (RTE_ARCH_ARM && RTE_ARCH_32).

Matching flags for other architectures are:
RTE_ARCH_X86
    |
    +----RTE_ARCH_32
    |        |
    |        +----RTE_ARCH_I686
    |        |
    |        +----RTE_ARCH_X86_X32
    |
    +----RTE_ARCH_64
             |
             +----RTE_ARCH_X86_64

RTE_ARCH_PPC_64 ---- RTE_ARCH_64

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>

config: add -moutline-atomics to default Arm build

-moutline-atomics allows LSE instructions to be used if available when
compiling for ARMv8.0 instruction set. It's enabled by default on newer
compilers, such as gcc-10.1. Enable the option in case an earlier
compiler version is used for the default build that lacks either -mcpu
or -mtune which would otherwise enable it.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

net/ice: use write combining store for tail updates

Performance improvement: use a write combining store
instead of a regular mmio write to update queue tail
registers.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Reviewed-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

net/ixgbe: use write combining store for tail updates

Performance improvement: use a write combining store
instead of a regular mmio write to update queue tail
registers.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

common/qat: use write combining store for tail updates

Performance improvement: use a write combining store
instead of a regular mmio write to update queue tail
registers.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

net/i40e: use write combining store for tail updates

Performance improvement: use a write combining store
instead of a regular mmio write to update queue tail
registers.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

eal: add write combining store

Add rte_write32_wc and rte_write32_wc_relaxed functions
that implement 32bit stores using write combining memory protocol.
Provided generic stubs and x86 implementation.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

mem: fix allocation failure on non-NUMA kernel

Running dpdk-helloworld on Linux with lib numa present, but no kernel
support for NUMA (CONFIG_NUMA=n) causes rte_service_init() to fail with
EAL: error allocating rte services array.

alloc_seg() calls get_mempolicy to verify that the allocation
has happened on the correct socket, but receives ENOSYS from
the kernel and fails the allocation.

The allocated socket should only be verified if check_numa() is true.

Fixes: 2a96c88be83e ("mem: ease init in a docker container")
Cc: stable@dpdk.org
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Reviewed-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

maintainers: update for bonding

Adding Connor as additional maintainer to bonding.

Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Reviewed-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/virtio: check raw checksum failure

rte_raw_cksum_mbuf can fail, so we should check to see if it
has. If so, return with an error.

Fixes: 96cb6711939e ("net/virtio: support Rx checksum offload")
Cc: stable@dpdk.org
Signed-off-by: Chas Williams <3chas3@gmail.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net: check segment pointer in raw checksum processing

If the overall pkt_len and segment lengths are out of agreement,
it is possible for the seg to be NULL after the loop. Add assert
to check this condition in debug builds. Otherwise, return failure.

Fixes: c442fed81bb9 ("net: add function to calculate checksum in mbuf")
Cc: stable@dpdk.org
Signed-off-by: Chas Williams <3chas3@gmail.com>

doc: fix diagram in dpaa2 guide

The diagram to show dpaa2 drivers brief
was missing in dpaa2.html file.

fix a typo in encoding for a literal block
to make it visible in generated doc file.

Fixes: 846a8305f277 ("doc: add DPAA2 NIC details")
Cc: stable@dpdk.org
Signed-off-by: Sachin Saxena <sachin.saxena@oss.nxp.com>