net/hinic: create and destroy flow director filter

Add support for creating/destroying fdir filter.
The filter rule includes LACP,ARP and ICMP packet type.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: create and destroy ntuple filter

Add support for creating/destroying ntuple filter.
The filter rule includes BGP and VRRP packet type.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: add flow validation operations

This patch is to validate the filter rules, which includes
ntuple filter, ethertype filter and fdir filter. The packets
type that supported are BGP,VRRP,LACP,ARP and ICMP.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic/base: add flow director config interface

This patch adds fdir config operation, including set fdir filter,
normal filter, set and clear fdir tcam.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: add unicast and multicast MAC set

This patch adds unicast and multicast set interfaces.
Application can add or remove unicast MAC address, also can set
multicast MAC address, the maximum multicast list size is 2048.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: add allmulticast mode and MTU set

When enable allmulticast mode, all multicast packets
can be received. This patch also adds support for MTU set,
the range of MTU is from 256 to 9600.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: add VLAN filter and offload

This patch adds support for VLAN filter and VLAN offload.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic: support VF

This patch adds VF PMD operation interfaces to support SR-IOV.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic/base: add HW interfaces for SR-IOV

This patch introduces some HW interfaces required for SR-IOV function,
including: init hwdev, set port state, get default cos,
VF DMA attribute table, VF txrx flush and so on.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/hinic/base: add mbox command channel for SR-IOV

Add mbox command channel for SR-IOV, which is used to
communicate between VF and VF, VF and PF. This patch
introduces data structures, initialization, interfaces
and commands of mbox channel.

Signed-off-by: Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>

net/bnxt: remove unnecessary variable assignment

There is no need to assign return value to a temporary variable.
Instead return error directly in case of failure.

Fixes: 1fe427fd08ee ("net/bnxt: support enable/disable interrupt")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>

net/bnxt: fix coding style

- Remove unnecessary new line
- Remove unnecessary blank lines
- Align rte_* header file inclusion at one place

Fixes: 37d6161a68ba ("net/bnxt: add ring group alloc/free")
Fixes: ec77c6298301 ("net/bnxt: add stats context allocation")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix handling interface change status

Driver issues hwrm_if_change when the port is started and stopped.
This command returns whether the fw has undergone a reset while port
was down. Fix to handle hwrm_if_change status during port start only.

Fixes: 0b533591238f ("net/bnxt: inform firmware about IF state changes")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>

net/bnxt: enable interrupts according interface status

Driver issues hwrm_if_change when the port is started. This command
returns whether the fw has undergone a reset while port was down and
driver does re-init of resources if fw has undergone a reset.
Fix to enable interrupts only after this check.

Fixes: c023cd5b2192 ("net/bnxt: fix async link handling and update")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: change MSI-X vector to queue mapping

DPDK PCIe-VFIO framework configures base MSIX vector for interrupts
which is supported by other h/w. In case of bnxt, base MSIX vector
starts with the RX completion queue 0. To comply with the DPDK
framework We need to increase the map index by 1 so that RXTX
completion queues events can be delivered to appropriate event listeners
by kernel VFIO.

Fixes: bd0a14c99f65 ("net/bnxt: use dedicated CPR for async events")
Cc: stable@dpdk.org
Signed-off-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: reduce cleanup time during reset recovery

In stop_op(), don't wait for link down event's ASYNC response from FW.

Fixes: c023cd5b2192 ("net/bnxt: fix async link handling and update")
Cc: stable@dpdk.org
Signed-off-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: get default HWRM command timeout from FW

The HWRM command timeout is set to a very high value.
VER_GET command response returns the default request timeout value.
Use this value for waiting for HWRM commands to complete.
Poll for the valid bit every 1us instead of 600us.

Fixes: cbcd375d37d2 ("net/bnxt: fix HWRM macros and locking")
Cc: stable@dpdk.org
Signed-off-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Lance Richardson <lance.richardson@broadcom.com>

net/bnxt: update trusted VF status only when it changes

We are currently logging trusted VF information everytime a VF config
change is indicated by the async message. Update it only when new
setting is different from the current setting.

Fixes: b42c15c83e88 ("net/bnxt: support trusted VF")
Cc: stable@dpdk.org
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: remove redundant header file inclusion

bnxt.h header includes bnxt_cpr.h and bnxt_util.h.
There is no need to include these headers file explicitly.

This commit does not cause any functional change.

Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: return error if setting link up fails

Currently bnxt driver does not return error in case setting link up fails.

Fixes: 5c206086feaa ("net/bnxt: add link state operations")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: free default completion ring before VF config

If the VF resources need to be reconfigured using the
bnxt_hwrm_func_reserve_vf_resc, make sure that the default completion
ring is freed first before the resources are reserved.

Reallocate the ring once the VF resources are configured.

Fixes: 7bc8e9a227cc ("net/bnxt: support async link notification")
Cc: stable@dpdk.org
Signed-off-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix stats errors handling

This patch fixes few checks and few return values while getting
and clearing device statistics.

1. Fixed to return standard error code.
2. Clubbed few error checks
3. Removed an unnecessary return check

Fixes: bfb9c2260be2 ("net/bnxt: support xstats get/reset")
Fixes: 88920136688c ("net/bnxt: support xstats get by id")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/pfe: add link status update

This patch add link related operations like
link update, up and down.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add allmulticast and promiscuous

This patch adds support to enable multicast and
promiscuous mode.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add MTU and MAC address set

To update MAC address or MTU, operations are
added to the driver.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add packet types and basic statistics

This patch add support for supported packet types
by the platform andbasic statistics like
numbers of packets/bytes receive and transmit.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add Rx/Tx

This patch add burst enqueue and dequeue operations
to the pfe PMD.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add queue setup and release

This patch add RX/TX queue setup operations
and supported checksum offloads.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: add device start/stop

This patch adds device start, stop and close
operations.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>

net/pfe: add MAC and host interface initialisation

HIF or host interface is responsible for transmit
and receive packets between physical ethernet
interfaces and HIF library defined logical interfaces.

This patch initialise that host interface and MAC.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>

net/pfe: add HW specific macros and operations

This patch add some hardware specific macros
and functions that will be used by the driver.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>

net/pfe: support dynamic logging

This patch introduces logging for the pfe PMD.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

net/pfe: introduce basic functions

pfe (packet forwarding engine) is a network
poll mode driver for NXP SoC ls1012a.

This patch introduces the framework of pfe
driver with basic functions of initialisation
and teardown.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>

common/dpaax: move OF library from DPAA bus

This code is being shared by more than 1 type of driver.
Common is most appropriate place for it.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/tap: fix blocked Rx packets

When OS sends more packets than are being read with a single
'rte_eth_rx_burst' call, rx packets are getting stucked in the tap pmd
and are unable to receive, because trigger_seen is getting updated
and consecutive calls are not getting any packets.

Do not update trigger_seen unless less than a max number of packets were
received allowing next call to receive the rest.

Remove unnecessary compiler barrier.

Fixes: a0d8e807d9de ("net/tap: add Rx trigger")
Cc: stable@dpdk.org
Signed-off-by: Marcin Smoczynski <marcinx.smoczynski@intel.com>
Tested-by: Mariusz Drost <mariuszx.drost@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>

net/enic: add flow implementation based on Flow Manager API

Flow Manager API is the new firmware interface that exposes match
action capabilities in 1400 series VIC adapters. It is intended for
virtual switch offloads and enables more advanced features than the
existing filter API. For example, it supports VXLAN encap and decap
actions, and exposes TCAM and exact match tables.

Add the new flow implementation using Flow Manager and use it when
available. When Flow Manager is not available, the driver will
continue to use the old filter-based flow implementation.

Signed-off-by: John Daley <johndale@cisco.com>
Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>

net/enic: enable GENEVE with options offload

Latest firmware supports Geneve with options offload. Current overlay
offload only supports Geneve with no options. This new feature is
disabled by default, as it is intrusive and cannot be used with flow
API. Add a new devarg 'geneve-opt' so the user can explicitly enable
it.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>

net/enic: fix crash in secondary process

Both primary and secondary processes may call the queue start/stop,
link update handlers. These functions use the rte_eth_dev pointer
cached in the adapter private data (struct enic). But, this pointer is
valid only in the primary process, as rte_eth_dev addresses may differ
in different processes. Using that cached pointer in secondary
processes leads to a crash.

For the link update handler (enic_link_update), use the rte_eth_dev
pointer passed down from the rte layer as it is valid in the current
process. For the queue start/stop handlers (enic_start_wq and
friends), cache the rte_eth_dev_data pointer in the adapter private
data, and use that. rte_eth_dev_data is in shared memory and its
address is same across processes.

Fixes: 837e68ae94a2 ("net/enic: fix queue stop and start")
Fixes: cf8d9826b7be ("net/enic: extract code for checking link status")
Cc: stable@dpdk.org
Reported-by: Dirk-Holger Lenz <dirk.lenz@ng4t.com>
Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Tested-by: Dirk-Holger Lenz <dirk.lenz@ng4t.com>
Reviewed-by: John Daley <johndale@cisco.com>

net/enic: fix probe for secondary process

Only the primary process initializes the adapter private data and
rte_eth_dev_data. For secondary processes, do not touch them.

Secondary processes need to select the right Tx and Rx handlers. Pick
the same handlers that the primary process uses.

Fixes: fefed3d1e62c ("enic: new driver")
Cc: stable@dpdk.org
Reported-by: Dirk-Holger Lenz <dirk.lenz@ng4t.com>
Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Tested-by: Dirk-Holger Lenz <dirk.lenz@ng4t.com>
Reviewed-by: John Daley <johndale@cisco.com>

app/testpmd: show supported packet types

Patch adds a runtime function to display port supported ptypes
in different layers.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

ethdev: fix QinQ offload

Use correct flag for indicating QinQ strip rx offload.

Fixes: dfebfc9882fb ("ethdev: support dynamic configuration of QinQ strip")
Cc: stable@dpdk.org
Signed-off-by: Vivek Sharma <viveksharma@marvell.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/octeontx2: enable GTPU for RSS hash index

Adding support to parse GTPU flag for RSS hash index calculation in
octeontx2.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: add GTPU flow type

Adding support to enable GTPU eth flow type for RSS hash
index calculation.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/dpaa2: update MC firmware version

MC firmware is the core component of FSLMC bus and net/DPAA2 devices.

Prior to this patch, MC firmware supported 10.14.x version. This
patch bumps the min supported version to 10.18.x.

Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: update MC firmware version

MC firmware is the core component of FSLMC bus and DPAA2 devices.

Prior to this patch, MC firmware supported 10.14.x version. This
patch bumps the min supported version to 10.18.x.

Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

ethdev: add definitions for EEPROM standards

This patch add definitions of maximal data length in module EEPROM,
values are compatible with include/uapi/linux/ethtool.h.

These definitions can be used by application to validate data length.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

test/bonding: fix LSC timeout unit

Fixes: 76d29903f5f5 ("bond: support link status interrupt")
Cc: stable@dpdk.org
Signed-off-by: Krzysztof Kanas <kkanas@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

test/bonding: fix LSC related cases

On rare situation test_link_bonding test case fail due to timespec
tv_nsec overflow, which causes pthread_cond_timedwait to return EINVAL
and test to fail.

Fixes: 76d29903f5f5 ("bond: support link status interrupt")
Cc: stable@dpdk.org
Signed-off-by: Krzysztof Kanas <kkanas@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

app/testpmd: add bits per second to statistics

Traffic bps statistics is very useful for performance testing.

Add bits per second statistics for Rx and Tx.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Moti Haimovsky <motih@mellanox.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: add AH key field to flow API

Add new rte_flow_item_ah in order to match the Authentication Header
based on RFC 2402.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: add IGMP key field to flow API

Add new rte_flow_item_igmp in order to match the Internet Group
Management Protocol based on RFC 2236.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: add NSH key field to flow API

Add new rte_flow_item_nsh in order to match the network service header
based on RFC 8300.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/mlx5: fix allocation size of RQT attribute

The receive queues list size is based on the size of uint32_t, so
when allocating the memory, the correct value should be used. Or
else there is risk to corrupt the memory, depending on the queues
number, because there is some pad area for alignment. If the queue
number is not large enough, the issue couldn't be observed.

Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX")
Cc: stable@dpdk.org
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: validate flow rule item order

The Item order validation between L2 and L3 is missing, which leading to
the following flow rule is accepted:

  testpmd> flow create 0 ingress pattern ipv4 / eth / end actions drop /
           end

Only the outer L3 layer should check whether the L2 layer is present,
because the L3 layer could directly follow the tunnel layer
without L2 layer.

Meanwhile inner L2 layer should check whether there is inner L3 layer
before it.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Cc: stable@dpdk.org
Signed-off-by: Xiaoyu Min <jackmin@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/virtio: do not require IO permissions

On x86, iopl permissions are only available to root user (or users that
have the CAP_SYS_RAWIO capability).
But those permissions are only needed when the virtio devices accesses
are done with inb/outb instructions, which is when the device is bound
to a UIO kernel module.

So far, the virtio driver was refusing to register based on the check
on IO permissions.
This check does not make sense when binding the device to vfio.

Now that the check on IO permissions has been abstracted in the ioport
API, we can remove it on virtio side.

We still need to call rte_eal_iopl_init() in the virtio constructor so
that the interrupt thread inherits this permission in the case it could
be used with UIO later.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

bus/pci: check IO permissions for UIO only

On x86, calling inb/outb special instructions (used in UIO ioport
read/write parts) is only possible if the right IO permissions has been
granted.

The only user of this API (the net/virtio pmd) checks this
unconditionnaly but this should be hidden by the rte_pci_ioport API
itself and only checked when the device is bound to a UIO driver.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

eal: use define instead of raw option name

We are using '--base-virtaddr' in a few places. We have a define for that,
so use it instead.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

eal/freebsd: support option --base-virtaddr

According to our docs, only Linuxapp supports base-virtaddr option.
That is, strictly speaking, not true because most of the things
that are attempting to respect base-virtaddr are in common files,
so FreeBSD already *mostly* supports this option in practice.

This commit fixes the remaining bits to explicitly support
base-virtaddr option, and moves the arg parsing from EAL to common
options parsing code. Documentation is also updated to reflect
that all platforms now support base-virtaddr.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

bus/pci: remove useless link dependency on ethdev

The makefile in drivers/bus/pci specified rte_ethdev as a dependency for
the library. However there are no actual symbols from librte_ethdev used
in librte_bus_pci.

Including librte_ethdev as a dependency only becomes a problem in some
niche cases like when attempting to build the rte_bus_pci library as a
shared object without building the rte_ethdev library.

I specifically ran into this when trying to build the DPDK included as
an SPDK submodule on a FreeBSD machine. I figure that since there are no
real dependencies between the two, we should enable building
librte_bus_pci without librte_ethdev.

Fixes: c752998b5e2e ("pci: introduce library and driver")
Cc: stable@dpdk.org
Signed-off-by: Seth Howell <seth.howell@intel.com>
Acked-by: Gaetan Rivet <gaetan.rivet@6wind.com>

test/distributor: fix spurious failure

Sanity test could spuriously fail with reporting flush count error.
It was caused by worker stat coherent issue between distributor and
worker thread.
Fix this issue by using atomic operations to update worker stat.

Fixes: c3eabff124e6 ("distributor: add unit tests")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: David Hunt <david.hunt@intel.com>

lib/distributor: fix deadlock on aarch64

Distributor and worker threads rely on data structs in cache line
for synchronization. The shared data structs were not protected.
This caused deadlock issue on weaker memory ordering platforms as
aarch64.
Fix this issue by adding memory barriers to ensure synchronization
among cores.

Bugzilla ID: 342
Fixes: 775003ad2f96 ("distributor: add new burst-capable library")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: David Hunt <david.hunt@intel.com>

build: fix compatibility with meson 0.47

The "in" keyword was introduced in meson 0.49, which means we need to use
an old-fashioned loop to check the array of disabled drivers.

Fixes: 6f80f1cd2247 ("build: support disabling drivers with meson")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>

mempool/octeontx2: remove minimum chunck size limitation

Memory required for the mempool objects need not be physically
contiguous on octeontx2's HW, mempool pmd supports the minimum chunk
size set by the default handler. Hence discarding the limitation set
by the pmd on the min_chunk_size value.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

cryptodev: clarify wireless inputs in digest-encrypted cases

Clarify constraints on fields specified in bits for wireless
algorithms in digest-encrypted case.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/dpaax_sec: enable build without rte_security

This patch enables the dpaaX_sec compilation when
RTE_LIBRTE_SECURITY is disabled.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

app/procinfo: fix build without rte_security

This patch fixes the compilation issues when
RTE_LIBRTE_SECURITY is disabled.

Fixes: 8a37f37fc243 ("app/procinfo: add --show-port")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: fix build without rte_security

This patch fixes the compilation issue in test_cryptodev.c
when RTE_LIBRTE_SECURITY is disabled.

Fixes: d883e6e7131b ("test/crypto: add PDCP C-Plane encap cases")

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add encrypted digest case for AES-CTR-CMAC

This patch adds tests for encrypted digest AES-CTR + AES CMAC using
mixed auth-cipher algorithm functions. Tests allow easy
extending with new cases, check in-place and
out-of-place operations and use linear and sgl
buffers as input/output.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

config: change default endianness for CAAM JR

CAAM JR can work on both LE and BE mode.
Latest platforms are in LE mode, so changing the
default mode to LE to make it more convenient for
the latest platforms.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

baseband/fpga_lte_fec: fix polling of MMIO register

Polling of MMIO register could misreport the actual value
set dynamically in hardware as the variable was not set explicitly
to volatile integer.

Fixes: efd453698c49 ("baseband/fpga_lte_fec: add driver for FEC on FPGA")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>

baseband/fpga_lte_fec: fix probing

A change to PCI mapping assumption was missed earlier, this causes
probing to fail with the fpga_lte_fec PMD when checking for name
of the rte_driver (not set yet) instead of the rte_pci_driver.

Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>

crypto/qat: fix AES CMAC mininum digest size

AES-CMAC is used in 3gpp specifications hence it is needed
to support 4 byte digest.

Fixes: 91c1daa4e1f0 ("crypto/qat: support AES-CMAC")
Cc: stable@dpdk.org
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: add RSA cases in QAT and openssl

This commit adds RSA tests to Intel QuickAssist Technology pmd
and OpenSSL pmd test suite for session and session-less cases
when PADDING_NONE selected

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

test/crypto: add session-less cases in asym mod exp

This commit adds asymmetric sessionless tests to mod exp.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support RSA CRT in asym

This commit extends RSA implementation by CRT option

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support RSA in asym

This commit adds RSA algorithm to asymmetric pmd
using pair (n, d) private key

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: support session-less for asym ops

This patch adds option to use asymmetric crypto pmd with
session-less support.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

doc: update doxygen for IPsec library

Added ipsec API from:
- rte_ipsec.h
- rte_ipsec_sa.h
- rte_ipsec_group.h
- rte_ipsec_sad.h

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

app/test-sad: add test application for IPsec SAD

Introduce new application to provide user to evaluate and perform
custom functional and performance tests for IPsec SAD implementation.

According to our measurements on SKX for 1M entries average lookup
cost is ~80 cycles, average add cost ~500 cycles.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

test/ipsec: add SAD autotests

add unittests for ipsec SAD library

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add SAD add/delete/lookup implementation

Replace rte_ipsec_sad_add(), rte_ipsec_sad_del() and
rte_ipsec_sad_lookup() stubs with actual implementation.

It uses three librte_hash tables each of which contains
an entries for a specific SA type (either it is addressed by SPI only
or SPI+DIP or SPI+DIP+SIP)

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add SAD create/destroy implementation

Replace rte_ipsec_sad_create(), rte_ipsec_sad_destroy() and
rte_ipsec_sad_find_existing() API stubs with actual
implementation.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

ipsec: add inbound SAD API

According to RFC 4301 IPSec implementation needs an inbound SA database
(SAD).
For each incoming inbound IPSec-protected packet (ESP or AH) it has to
perform a lookup within it's SAD.
Lookup should be performed by:
Security Parameters Index (SPI) + destination IP (DIP) + source IP (SIP)
or SPI + DIP
or SPI only
and an implementation has to return the 'longest' existing match.
This patch extend DPDK IPsec library with inbound security association
database (SAD) API implementation that:
- conforms to the RFC requirements above
- can scale up to millions of entries
- supports fast lookups
- supports incremental updates

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

crypto/qat: fix null auth when using VFIO

When running auth NULL cases while using
vfio_pci, DMAR read/write faults appear. It
happens even if digest_length is set to 0.
This is caused by auth_res_addr initialized
as 0x0.

Fixes: 4e0955bddb08 ("crypto/qat: fix null auth algo overwrite")
Cc: stable@dpdk.org
Signed-off-by: Damian Nowak <damianx.nowak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/aesni_gcm: allow device init if no AES-NI is present

The IPSec Multi buffer library does not require AES-NI
instructions to be supported by the CPU, as it can emulate these
instructions in software (adding a big performance penalty when
using AES algorithms).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: allow device init if no AES-NI is present

The IPSec Multi buffer library does not require AES-NI
instructions to be supported by the CPU, as it can emulate these
instructions in software (adding a big performance penalty when
using AES algorithms).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

common/dpaax: move shared sec HW code from dpaa2_sec

The SEC HW code is being shared by multiple NXP based
drivers. It is better to place it at a common place.
Current users are:
1. DPAA2_SEC
2. DPAA_SEC
3. CAAM_JR

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: fix list of unsupported features in IPsec guide

List of unsupported features doesn't reflect latest changes.

Fixes: cd5b860c1851 ("ipsec: support header construction")
Fixes: 2c1887fad075 ("ipsec: fix transport mode for IPv6 with extensions")

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

test/crypto: support asym cases for OCTEON TX2

This patch updates asymmetric crypto unit-test application to
validate asymmetric crypto operation supported by octeontx2 PMD.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: support asymmetric in enqueue/dequeue

This patch adds asymmetric support in enqueue/dequeue ops.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add asymmetric session

This patch adds asymmetric session setup and free routines.
RSA and modexp operations are supported.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kanaka Durga Kotamarthy <kkotamarthy@marvell.com>
Signed-off-by: Sunila Sahu <ssahu@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: add OCTEON TX2

This patch adds the OCTEON TX2 crypto validation tests.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add enqueue/dequeue ops

This patch adds the enqueue burst and dequeue
burst callbacks for the OCTEON TX2 crypto driver.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add session related functions

This patch adds the symmetric session related callbacks.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add symmetric capabilities

This patch adds the symmetric capabilities
for OCTEON TX2 crypto PMD.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add queue pair functions

This patch adds the queue pair setup and
queue pair release functions for OCTEON TX2 crypto pmd.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: support control ops

This patch adds the device control functions.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add init sequence in probe

This patch adds the device init sequence for
OCTEON TX2 crypto device.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: add PMD skeleton

Adding OCTEON TX2 crypto PMD skeleton.
Enabling the driver by default in common_base.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/compress: add stateless overflow recovery case

Added unit test to check out-of-space recoverable feature.

Signed-off-by: Artur Trybula <arturx.trybula@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>