net/sfc: support flow action flag in MAE backend

The action handler will use MAE action FLAG.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: support adding flag action to set

This action can be added at any point before DELIVER.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support VLAN push actions in MAE backend

A group of actions (OF_PUSH_VLAN, OF_VLAN_SET_VID and
OF_VLAN_SET_PCP) maps to MAE action VLAN_PUSH.

This action group is supported only for rules which have transfer
attribute, and can be requested once or twice per a rule.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add facilities to handle bundles of actions

There are MAE actions which do not have uniform counterparts
in terms of RTE flow. However, there are bundles of RTE flow
actions which can be considered as such counterparts.

Implement facilities to handle related RTE flow actions as
parts of a whole. These facilities will be used by a later
patch to add support for VLAN PUSH actions bundle.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: support adding VLAN push action

MAE supports pushing two tags, so this action can
be requested once or twice.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support flow action OF pop VLAN in MAE backend

This action is supported only for rules which have transfer attribute,
and can be requested once or twice per a rule.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: support adding VLAN pop action to set

MAE supports stripping two tags, so this action can
be requested once or twice.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: implement flow insert/remove in MAE backend

Exercise action set allocation / release and action rule
insertion / removal in order to let flow API callers
actually get created flows functioning.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE action rule provisioning APIs

Add APIs for action rule insert / remove operations.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE action set provisioning APIs

The patch adds APIs for action set allocation / release.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support flow action PHY port in MAE backend

The action handler will use MAE action DELIVER with
MPORT of a given physical port.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: support adding deliver action to set

Introduce a mechanism for adding actions to an action set and
add support for DELIVER action.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support flow item eth in MAE backend

Add support for this flow item to MAE-specific RTE flow implementation.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE match fields for Ethernet

Add MCDI-compatible enumeration for these fields and
provide necessary mappings for them to be inserted
directly into mask-value pairs buffer.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support flow item PHY PORT in MAE backend

Add support for this flow item to MAE-specific RTE flow implementation.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: support setting PPORT in match spec

Add an API for setting mask-value pairs in a match specification
structure and add support for MAE field INGRESS_PORT of type PPORT.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add actions parsing stub to MAE backend

If parsing a flow results in an action set specification
identical to an already existing one, duplication will
be avoided by reusing the list entry of the latter.
Using an attach helper and a reference counter
is meant to serve the said purpose.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add action set spec init/fini APIs

The engine is only able to carry out chosen actions on matching packets in
a strict order. No MCDI exists to identify supported actions and the order.
Still, the definition of the latter is available from the FW documentation.

The general idea is to define an action specification structure and supply
a client driver with APIs for adding actions individually, order-dependent.
A client driver is supposed to invoke an API on every action passed by the
application, and if an out-of-order action follows, the API will reject it.

Add an action set specification stub and supply initialise / finalise APIs.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add verify method to flow validate path

The new method is needed to make sure that a flow being
validated will have a chance to be accepted by the FW.
MAE-specific implementation of the method should
compare the class of a rule being validated with
the corresponding classes of active rules, and,
if no matches found, make a request to the FW.
Support for the latter will be added in future.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add match specs class comparison API

From MAE standpoint, a flow rule belongs to some class. Field capabilities
advertised by the FW provide a hint on whether changing a particular match
field value or its mask will affect the class of the rule. A client driver
can make use of the concept of a class by comparing a rule being validated
with already inserted ones so that if an existing rule with the same class
is encountered, it will become possible to skip making an explicit request
to the FW because the class of an already inserted rule is wittingly valid.

Implement an API for client drivers to carry out the said class comparison.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: validate match spec in MAE backend

Validate the match specification resulting from pattern
parsing within MAE backend in RTE flow implementation.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add match spec validate API

MAE has restrictions on what kind of mask a particular field can have in
a match specification. Add an API for client drivers to check
specifications.

The patch defines a field description list, whilst the list itself is
left empty. This is to provide a general idea of how field properties
will be used to validate a match specification. Particular fields
will be added to the list by follow-up patches.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add pattern parsing stub to MAE backend

Add pattern parsing stub, define and implement flow cleanup method.
The latter is needed to free any dynamic structures allocated
during flow parsing.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add match spec init/fini APIs

An MAE rule is a function of match criteria and a priority. The said match
criteria have to be provided using "mask-value pairs" packing format which
on its own should not be exposed to client drivers. The latter have to use
a functional interface of sorts in order to generate a match specification.

Define an EFX match specification and implement initialise / finalise APIs.
The "mask-value pairs" buffer itself is not used in this particular patch,
so the corresponding struct member will be added in the follow-up patch.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add concept of MAE (transfer) rules

Define the corresponding specification structure and
make the code identify MAE rules by testing transfer
attribute presence. Also, add a priority level check.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE limit query API

Add an API for client drivers to query the engine limits.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

drivers: init/fini MAE on attach/detach

These actions affect MAE supplementary resources which are
libefx-internal.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE init/fini APIs

The patch adds APIs for client drivers to initialise / finalise
MAE-specific context in NIC control structure. The context
itself will be used by the follow-up patches to store
supportive data for library-internal consumers.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: add stub for attaching to MAE

Add a stub for MAE attach / detach path and introduce MAE-specific
context.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: indicate support for MAE

Riverhead boards maintain support for MAE, a low-level Match-Action
Engine.
The feature is documented in SF-122526-TC.

The new field will help client drivers to test NIC support for MAE
status.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE definitions to MCDI

MAE stands for Match-Action-Engine and will be used to
support rte_flow API transfer rules.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>

app/testpmd: fix bonding xmit balance policy command

Currently there exists inconsistency about name of transmission
policy for a Link Bonding device. "xmit_balance_policy" is not
correct, which should be modified to "balance_xmit_policy".

Fixes: 2950a769315e ("bond: testpmd support")
Fixes: ac718398f477 ("doc: testpmd application user guide")
Cc: stable@dpdk.org
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/softnic: fix out-of-bound access

This patch fixes out of bound access of an array.

Coverity issue: 363466, 363459
Fixes: b5dfa6703d49 ("net/softnic: update subport rate dynamically")
Cc: stable@dpdk.org
Signed-off-by: Savinay Dharmappa <savinay.dharmappa@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

net/ice: support ACL filter in DCF

Add ice_acl_create_filter to create a rule and ice_acl_destroy_filter
to destroy a rule. If a flow is matched by ACL filter, filter rule
will be set to HW. Currently IPV4/IPV4_UDP/IPV4_TCP/IPV4_SCTP pattern
and drop action are supported.

Signed-off-by: Simei Su <simei.su@intel.com>
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: get PF VSI map

This patch gets PF vsi number when issuing ACL rule in DCF.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: fix unchecked Tx cleanup error

Coverity complains of unchecked return value warning of
iavf_xmit_cleanup, while this cleanup is opportunistic and will not cause
problems if it fails. So instead of checking the return value of
iavf_xmit_cleanup and return in case of cleanup failure, we directly cast
it to void function to make the Coverity happy.

Coverity issue: 363045
Fixes: 02d212ca3125 ("net/iavf: rename remaining avf strings")
Cc: stable@dpdk.org
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: fix SCTP RSS configuration

This patch configured RSS for sctp with IP address
and port as input set.

Fixes: 4717a12cfaf1 ("net/ice: initialize and update RSS based on user config")
Cc: stable@dpdk.org
Signed-off-by: Junyu Jiang <junyux.jiang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: fix bitmap set function

This patch corrects an upper limit value in for loop.

Fixes: dd4a3cef556a ("net/ice/base: introduce and use bitmap set API")
Cc: stable@dpdk.org
Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: fix adding multicast MAC address

When the multicast address list is added, it will flush
previous addresses first, and then add new ones.
If the number of multicast address in the list exceeds
the upper limit, it will cause failure, then need to
roll back previous addresses. This patch fixes the issue.

Fixes: 05e4c3aff35f ("net/iavf: support multicast configuration")
Cc: stable@dpdk.org
Signed-off-by: Guinan Sun <guinanx.sun@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>

net/mlx5: support query of age action

Recent patch [1] adds to ethdev the API for query of age action.
This patch implements in MLX5 PMD the query of age action using
this API.

[1] https://mails.dpdk.org/archives/dev/2020-October/184864.html

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: log LRO minimal size

Add debug printout showing HCA capability lro_min_mss_size - the
minimal size of TCP segment required for coalescing.
MLX5 PMD documentation is updated to note this condition.

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: fix use of atomic cmpset for age state

According to documentation [1], function rte_atomic16_cmpset()
return value is non-zero on success; 0 on failure.
In existing code this function is called, and the return value
is compared to AGE_CANDIDATE, which is defined as 1.
Such comparison is incorrect and can lead to unwanted behavior.

This patch updates the calls to rte_atomic16_cmpset(), to check
that the return value is 0 or non-zero.

[1] https://doc.dpdk.org/api/rte__atomic_8h.html

Fixes: fa2d01c87d2b ("net/mlx5: support flow aging")
Cc: stable@dpdk.org
Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: enforce limitation on IPv6 next protocol

Due to PRM requirement, the IPv6 header item 'proto' field, indicating
the next header protocol, should not be set as extension header.
This patch adds the relevant validation, and documents the limitation.

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

net/mlx5: support match on IPv6 fragment extension

rte_flow update, following RFC [1], added to ethdev the rte_flow item
ipv6_frag_ext.
This patch adds to MLX5 PMD the option to match on this item type.

[1] http://mails.dpdk.org/archives/dev/2020-March/160255.html

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

net/mlx5: support match on IPv6 fragment packets

This patch adds to MLX5 PMD the support of matching on IPv6
fragmented and non-fragmented packets, using the new field
has_frag_ext, added to rte_flow following RFC [1].

[1] https://mails.dpdk.org/archives/dev/2020-August/177257.html

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

net/mlx5: support match on IPv4 fragment packets

This patch adds to MLX5 PMD the support of matching on IPv4
fragmented and non-fragmented packets, using the IPv4 header
fragment_offset field.

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

net/mlx5: remove handling of ICMP fragmented packets

Commit [1] forced setting of match on 'frag' bit mask 1 and value 0.
Previous patch in this series added support of match on fragmented and
non-fragmented packets on L3 items, so this setting is now redundant.

This patch removes the changes done in [1].

[1] commit 85407db9f60d ("net/mlx5: fix matching for ICMP fragments")

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

net/mlx5/linux: fix Tx queue operations decision

One of the conditions to create Tx queue object by DevX is to be sure
that the DPDK mlx5 driver is not going to be the E-Switch manager of
the device. The issue is with the default FDB flows managed by the
kernel driver, which are not created by the kernel when the Tx queues
are created by DevX.

The current decision is to create the Tx queues by Verbs when E-Switch
is enabled while the current behavior uses an opposite condition to
create them by DevX.

Create the Tx queues by Verbs when E-Switch is enabled.

Fixes: 86d259cec852 ("net/mlx5: separate Tx queue object creations")
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix event queue number query

When a Rx\Tx queue is created by DevX, its CQ configuration should
include the EQ number of the interrupts.
The EQ is managed by the kernel and there is a glue API in order to
query the EQ number from the kernel.
The EQ query API gets a vector number specifies the kernel vector of
the interrupt handling.

The vector number was wrongly detected according to the configuration
CPU instead of using the device attributes of the supported vectors.
The CPU was wrongly detected by the rte_lcore_to_cpu_id API without any
check, and in case of non-EAL thread context the value was 0xFFFFFFFF
which caused a failure in the EQ number query API.

Use vector 0 for each EQ number query which must be supported by the
kernel.

Fixes: 08d1838f645a ("net/mlx5: implement CQ for Rx using DevX API")
Fixes: d133f4cdb706 ("net/mlx5: create clock queue for packet pacing")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix Tx queue release

The HW objects of the Tx queue is created/destroyed in the device
start\stop stage while the ethdev configurations for the Tx queue
starts from the tx_queue_setup stage.
The PMD should save all the last configurations it got from the ethdev
and to apply them to the device in the dev_start operation.

Wrongly, last code added to mitigate the reference counters didn't take
into account the above rule and combined the configurations and HW
objects to be created\destroyed together.

This causes to memory leak and other memory issues.

Make sure the HW object is released in stop operation when there is no
any reference to it while the configurations stay saved.

Fixes: 17a57183c0eb ("net/mlx5: mitigate Tx queue reference counters")
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix Rx queue release

The HW objects of the Rx queue is created/destroyed in the device
start\stop stage while the ethdev configurations for the Rx queue
starts from the rx_queue_setup stage.
The PMD should save all the last configurations it got from the ethdev
and to apply them to the device in the dev_start operation.

Wrongly, last code added to mitigate the reference counters didn't take
into account the above rule and combined the configurations and HW
objects to be created\destroyed together.

This causes to memory leak and other memory issues.

Make sure the HW object is released in stop operation when there is no
any reference to it while the configurations stay saved.

Fixes: 24e4b650badc ("net/mlx5: mitigate Rx queue reference counters")
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

metrics: fix memory leak on allocation failure

If an error occurred when allocating memory for metrics or names,
the function returned without freeing allocated memory. This is now
fixed to avoid the resource leak in the case that either metrics or
names had been successfully allocated memory.

Coverity issue: 362053
Fixes: c5b7197f662e ("telemetry: move some functions to metrics library")
Cc: stable@dpdk.org
Reported-by: Gaurav Singh <gaurav1086@gmail.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>

eal: fix power intrinsics API description

Currently, the intrinsics documentation refers to `rte_cpu_get_features`
as a check for whether these intrinsics are supported at runtime. This
is incorrect, because actually the user should use the
`rte_cpu_get_intrinsics_support` API to do said check. Fix the typo.

Fixes: 128021421256 ("eal: add intrinsics support check infrastructure")
Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Liang Ma <liang.j.ma@intel.com>

gso: fix mbuf freeing responsibility

rte_gso_segment decreased refcnt of pkt by one, but
it is wrong if pkt is external mbuf, pkt won't be
freed because of incorrect refcnt, the result is
application can't allocate mbuf from mempool because
mbufs in mempool are run out of.

One correct way is application should call
rte_pktmbuf_free after calling rte_gso_segment to free
pkt explicitly. rte_gso_segment must not handle it, this
should be responsibility of application.

This commit changed rte_gso_segment in functional behavior
and return value, so the application must take appropriate
actions according to return values, "ret < 0" means it
should free and drop 'pkt', "ret == 0" means 'pkt' isn't
GSOed but 'pkt' can be transmitted as a normal packet,
"ret > 0" means 'pkt' has been GSOed into two or multiple
segments, it should use "pkts_out" to transmit these
segments. The application must free 'pkt' after call
rte_gso_segment when return value isn't equal to 0.

Fixes: 119583797b6a ("gso: support TCP/IPv4 GSO")
Cc: stable@dpdk.org
Signed-off-by: Yi Yang <yangyi01@inspur.com>
Acked-by: Jiayu Hu <jiayu.hu@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

eal/windows: fix deadlock when setting alarm

Windows alarms are both armed and executed from the interrupt thread.
rte_eal_alarm_set() dispatched alarm-arming code to that thread and
waited for its completion via a spinlock. However, if called from alarm
callback (i.e. from the interrupt thread), this caused a deadlock,
because arming could not be run until its dispatcher exits, but it could
only exit after it finished waiting for arming to complete.

Call arming code directly when running in the interrupt thread.

Fixes: f4cbdbc7fbd2 ("eal/windows: implement alarm API")
Reported-by: Pallavi Kadam <pallavi.kadam@intel.com>
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Tested-by: Pallavi Kadam <pallavi.kadam@intel.com>

eal/windows: allow running as non-admin

Currently, since there is no runtime directory set, the code tries to
create a file in C:\ which is only writable with administrator
privileges. As a result, if the user is not admin, the application will
fail.
So, forcing no_shconf to 1 to prevent the code having to create files in
the runtime directory.

Suggested-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

maintainers: resign from stack library

I'm moving on to a new position in November and won't be able to continue
as a stack library maintainer.

Thanks to fellow maintainer Olivier, and the rest of the DPDK community,
for the support over the past few years.

Signed-off-by: Gage Eads <gage.eads@intel.com>

config/arm: add Graviton2

Add meson build configuration for Graviton2 platform
with 64-bit Arm Neoverse N1 cores. This patch makes the
following changes to generic Neoverse N1 config:

1. increase lcore limit to 64
2. increase memory support to 1TB
3. remove +crc from -march as that is default when setting armv8.2

For more information about Graviton2 platform, refer to:
https://aws.amazon.com/ec2/graviton/

Signed-off-by: Vimal Chungath <vcchunga@amazon.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

test/crypto: disable wireless cases for CPU crypto API

SNOW3G, KASUMI and ZUC algorithms have been added to
aesni_mb PMD in this release. These algorithms are not supported
for the crypto synchronous API, so the crypto tests for these algorithms
are disabled.

Fixes: ae8e085c608d ("crypto/aesni_mb: support KASUMI F8/F9")
Fixes: 6c42e0cf4d12 ("crypto/aesni_mb: support SNOW3G-UEA2/UIA2")
Fixes: fd8df85487c4 ("crypto/aesni_mb: support ZUC-EEA3/EIA3")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: support Chacha-Poly in synchronous mode

Add support for Chacha20-Poly1305 in the CPU crypto synchronous API.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

examples/ipsec-secgw: fix session mempool initialisation

Creation of a session mempool may fail in the case of a single lcore
and a low number of SA.

In case when there is only one lcore and number of configures SA less
then (0.5 * CDEV_MP_CACHE_SZ) then creation of the mempool fails with
EINVAL. This is because the number of requested items is less than
(cache size * CDEV_MP_CACHE_MULTIPLIER).

Hence, the number of elements in mempool is increased in such cases.

Fixes: e30b2833c47c ("security: update session create API")
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/octeontx2: fix out-of-place support

Out of place with linear buffers is supported by octeontx2
while not advertised.

Fixes: 6aa9ceaddf1d ("crypto/octeontx2: add symmetric capabilities")
Cc: stable@dpdk.org
Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: Ankur Dwivedi <adwivedi@marvell.com>

crypto/octeontx: fix out-of-place support

Out of place with linear buffers is supported by octeontx
while not advertised.

Fixes: 0dc1cffa4d33 ("crypto/octeontx: add hardware init routine")
Cc: stable@dpdk.org
Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: Ankur Dwivedi <adwivedi@marvell.com>

doc: update bcmfs guide

Update bcmfs.rst file with supported features and devices.

Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: JP Lee <jongpil.lee@broadcom.com>

app/bbdev: dequeue full ring in offload test

update offload dequeue to retrieve the full ring to be
agnostic of implementation.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>

app/bbdev: reduce duration of throughput test

Reducing number of repetitions from 1000 to 100
to save time. Results are accurate enough with
100 loops.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>

app/bbdev: skip bler test when compression is used

bler test results are not valid when LLR compression
is used or for loopback scenarios. Skipping these.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>
Reviewed-by: Tom Rix <trix@redhat.com>

app/bbdev: define wait for offload

Replacing magic number for default wait time for hw
offload.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>

app/bbdev: include explicit HARQ preloading

Run preloading explicitly for unit tests. Load each code block
by reusing existing input op then restore for the actual test.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Liu Tianjiao <tianjiao.liu@intel.com>

app/bbdev: add explicit check for counters

Adding explicit check in ut that the stats counters
have the expect values.
This was identified as a gap from code coverage.

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>

app/bbdev: add explicit test for latency vs validation

Adding explicit different ut when testing for validation
or latency (early termination enabled or not).

Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Acked-by: Aidan Goddard <aidan.goddard@accelercomm.com>
Acked-by: Dave Burley <dave.burley@accelercomm.com>

doc: recommend latest OpenSSL version

Add recommendation to update to latest OpenSSL version when
using the OpenSSL PMD and to at least version 1.1.1g to avoid
known CVEs.

Signed-off-by: John McNamara <john.mcnamara@intel.com>

common/qat: add missing kmod dependency info

Dependency on kmod needed to manage crypto devices is missing
in qat crypto pmd.

Fixes: 0880c40113ef ("drivers: advertise kmod dependencies in pmdinfo")
Cc: stable@dpdk.org
Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/dpaa2_sec: remove dead code

RTE_LIBRTE_SECURITY_TEST never existed, the variable under this check is
never used.

Fixes: e117c18a1dbe ("crypto/dpaa2_sec: restructure session management")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

fips_validation: fix GCM test

This patch fixes FIPS GCM test of the redundant plaintext string
write and insufficient test case support check for some corner
cases.

Fixes: d09abf2d1007 ("examples/fips_validation: update GCM test")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>

examples/fips_validation: fix missed version line

This patch fixes the missing version line in the response file.

Fixes: 793650184099 ("examples/fips_validation: fix version compatibility")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>

crypto/octeontx2: fix multi-process

During crypto device probe few functions should be called only
for the primary process. This patch fixes this issue.

Fixes: 818d138bcce9 ("crypto/octeontx2: add init sequence in probe")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>

baseband/acc100: remove useless checks

Coverity reported dead code for a few error
checks which are indeed not reachable.

Coverity issue: 363451, 363454, 363455
Fixes: 5ad5060f8f7a ("baseband/acc100: add LDPC processing functions")
Fixes: f404dfe35cc3 ("baseband/acc100: support 4G processing")
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
Reviewed-by: Tom Rix <trix@redhat.com>

baseband/turbo_sw: fix memory leak in error path

In q_setup() allocated memory for the queue data, we should free
it when error happens, otherwise it will lead to memory leak.

Fixes: b8cfe2c9aed2 ("bb/turbo_sw: add software turbo driver")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Reviewed-by: Nicolas Chautru <nicolas.chautru@intel.com>

doc: remove notice about AES-GCM IV and J0

This patch removes information about deprecation of AES-GCM/GMAC
API for IV without J0.

Fixes: fac52fb26a54 ("cryptodev: add option to support both IV and J0 for GCM")
Cc: stable@dpdk.org
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

doc: update VFIO usage in qat crypto guide

This patch marks the old igb-uio driver as unsecure when used
with the QAT PMD and updates all examples to recommend using
VFIO-PCI instead.
It also mentions security issues with the QAT CPM and provides
information about the new VFIO-PCI parameter 'disable_denylist'
available in Linux kernels 5.9 and later.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

doc: fix driver names

Since the built driver filenames have changed in DPDK 20.11, we need to
update the driver doc to match.

Most drivers start their section with the driver filename highlighted in
bold, while a number were missing the highlight. When updating the names,
add the markers for bold text to any missing it, so as to have things more
consistent.

Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

ethdev: include mbuf registration in Tx timestamp API

Previously, the Tx timestamp field and flag were registered in testpmd,
as described in mlx5 guide.
For consistency between Rx and Tx timestamps,
managing mbuf registrations inside the driver, as properly documented,
is a simpler expectation.

The only driver to support this feature (mlx5) is updated
as well as the testpmd application.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: add Tx timestamp registration helper

The function rte_mbuf_dyn_tx_timestamp_register()
can be used to register the required field and flag.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: remove deprecated timestamp field

As announced in the deprecation note, the field timestamp
is removed to give more space to the dynamic fields.
The related offload flag PKT_RX_TIMESTAMP is also removed.

This is how the mbuf layout looks like (pahole-style):

word  type                              name                byte  size
 0    void *                            buf_addr;         /*   0 +  8 */
 1    rte_iova_t                        buf_iova          /*   8 +  8 */
      /* --- RTE_MARKER64               rearm_data;                   */
 2    uint16_t                          data_off;         /*  16 +  2 */
      uint16_t                          refcnt;           /*  18 +  2 */
      uint16_t                          nb_segs;          /*  20 +  2 */
      uint16_t                          port;             /*  22 +  2 */
 3    uint64_t                          ol_flags;         /*  24 +  8 */
      /* --- RTE_MARKER                 rx_descriptor_fields1;        */
 4    uint32_t             union        packet_type;      /*  32 +  4 */
      uint32_t                          pkt_len;          /*  36 +  4 */
 5    uint16_t                          data_len;         /*  40 +  2 */
      uint16_t                          vlan_tci;         /*  42 +  2 */
 5.5  uint64_t             union        hash;             /*  44 +  8 */
 6.5  uint16_t                          vlan_tci_outer;   /*  52 +  2 */
      uint16_t                          buf_len;          /*  54 +  2 */
 7    uint64_t                          dynfield0[1];     /*  56 +  8 */
      /* --- RTE_MARKER                 cacheline1;                   */
 8    struct rte_mempool *              pool;             /*  64 +  8 */
 9    struct rte_mbuf *                 next;             /*  72 +  8 */
10    uint64_t             union        tx_offload;       /*  80 +  8 */
11    struct rte_mbuf_ext_shared_info * shinfo;           /*  88 +  8 */
12    uint16_t                          priv_size;        /*  96 +  2 */
      uint16_t                          timesync;         /*  98 +  2 */
12.5  uint32_t                          dynfield1[7];     /* 100 + 28 */
16    /* --- END                                             128      */

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

ethdev: add doxygen comment for Rx timestamp API

The offload flag DEV_RX_OFFLOAD_TIMESTAMP had no documentation.
After switching to dynamic mbuf flag and field,
it becomes even more important to explicit the feature behaviour.

A doxygen comment for the timesync API was mentioning
the deprecated timestamp field, so it is also updated.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

examples/rxtx_callbacks: switch timestamp to dynamic field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

app/testpmd: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/pcap: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/octeontx2: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

The registration of field and flag is done in both
otx2_nix_dev_start() and otx2_nix_timesync_enable().

The dynamic offset and flag are stored in struct otx2_timesync_info
to favor cache locality.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/nfb: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/mlx5: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

The dynamic offset and flag are stored in struct mlx5_rxq_data
to favor cache locality.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/mlx5: fix dynamic mbuf offset lookup check

The functions rte_mbuf_dynfield_lookup() and rte_mbuf_dynflag_lookup()
can return an offset starting with 0 or a negative error code.

In reality the first offsets are probably reserved forever,
but for the sake of strict API compliance,
the checks which considered 0 as an error are fixed.

Fixes: efa79e68c8cd ("net/mlx5: support fine grain dynamic flag")
Fixes: 3172c471b86f ("net/mlx5: prepare Tx queue structures to support timestamp")
Fixes: 0febfcce3693 ("net/mlx5: prepare Tx to support scheduling")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/dpaa2: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net/ark: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related dynamic mbuf flag is set, although was missing previously.

The timestamp is set if configured for at least one device.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

latency: switch Rx timestamp to dynamic mbuf field

The mbuf timestamp is moved to a dynamic field
in order to allow removal of the deprecated static field.
The related mbuf flag is also replaced with the dynamic one.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: add Rx timestamp flag and helpers

There is already a dynamic field for timestamp,
used only for Tx scheduling with the dedicated Tx offload flag.
The same field can be used for Rx timestamp filled by drivers.

A new dynamic flag is defined for Rx usage.
A new function wraps the registration of both field and Rx flag.
The type rte_mbuf_timestamp_t is defined for the API users.

After migrating all Rx timestamp usages, it will be possible
to remove the deprecated timestamp field.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eventdev: remove software Rx timestamp

This a revert of the commit 569758758dcd ("eventdev: add Rx timestamp").
If the Rx timestamp is not configured on the ethdev port,
there is no reason to set one.
Also the accuracy  of the timestamp was bad because set at a late stage.
Anyway there is no trace of the usage of this timestamp.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

regex/octeontx2: fix driver name

Following the recent alignment of all driver names,
this new driver get unaligned:
librte_regex_octeontx2_regex.so

The 'fmt_name' must be "octeontx2_regex", and if not provided,
is taken from the 'name' variable.
But the variable 'name' should not be overwritten,
to keep the automatic value from the directory name.

The library name will be composed of the class directory
and the driver directory name:
librte_regex_octeontx2.so

Reported-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>

event/dlb: add timeout ticks entry point

Adds the timeout ticks conversion function.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>