app/eventdev: simplify signal handling and teardown

Remove rte_*_dev calls from signal handler callback as signal handlers
are supposed to be light weight.

Split ethdev teardown into Rx and Tx sections, wait for
workers to finish processing after disabling Rx to allow workers
to complete processing currently held packets.

Verified SW event device on ARM64 using the following command:

./build/app/dpdk-test-eventdev -l 7-23 -s 0xf00 --vdev=event_sw0
 -a 0002:02:00.0 -- --prod_type_ethdev --nb_pkts=0 --verbose 2
 --test=pipeline_queue --stlist=o --wlcores 16-23

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

event/cnxk: move post-processing to separate function

Move event post-processing to a separate function.
Do complete event post-processing in tear-down functions to prevent
incorrect memory free.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/cnxk: add checks in release operation

Add additional checks while performing RTE_EVENT_OP_RELEASE to
ensure that there are no pending SWTAGs and FLUSHEs in flight.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

eventdev/timer: add telemetry

Adds telemetry support to get timer adapter info and timer adapter
statistics.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

event/cnxk: fix out of bounds access in test

Fix out of bounds array access reported in coverity scan.

Coverity issue: 375817
Fixes: 2351506401e ("event/cnxk: add SSO selftest and dump")
Cc: stable@dpdk.org
Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

event/dlb2: allow CQ depths up to 1024

Updated to allow overriding the default CQ depth of 32.  Since there are
only 2048 DLB history list entries, increasing the CQ depth decreases
the number of available LDB ports to 2048/max_cq_depth. Resource query
will take this into account and return the correct maximum number of
LDB ports.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>

event/cnxk: fix base pointer for SSO head wait

Function roc_sso_hws_head_wait() expects a base as input pointer, and it
will itself get tag_op from the base. By passing tag_op instead of base
pointer to this function will add SSOW_LF_GWS_TAG register offset twice,
which will lead to accessing wrong register.

Fixes: 1f5b3d55c041 ("event/cnxk: store and reuse workslot status")
Cc: stable@dpdk.org
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

eventdev/eth_rx: fix telemetry Rx stats reset

Caught by covscan:

1. dpdk-21.11/lib/eventdev/rte_event_eth_rx_adapter.c:3279:
logical_vs_bitwise: "~(*__ctype_b_loc()[(int)*params] & 2048 /*
(unsigned short)_ISdigit */)" is always 1/true regardless of the values
of its operand. This occurs as the logical second operand of "||".
2. dpdk-21.11/lib/eventdev/rte_event_eth_rx_adapter.c:3279: remediation:
Did you intend to use "!" rather than "~"?

While isdigit return value should be compared as an int to 0,
prefer ! since all of this file uses this convention.

Fixes: 814d01709328 ("eventdev/eth_rx: support telemetry")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>

doc: fix build with sphinx 4.5

Latest Sphinx checks C language syntax more aggressively.
Fix the following warning by correcting C language syntax.

doc/guides/prog_guide/event_ethernet_rx_adapter.rst:243:
WARNING: Could not lex literal_block as "c". Highlighting skipped.

Fixes: 3c838062b91f ("eventdev: introduce event vector Rx capability")
Cc: stable@dpdk.org
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>

net/mlx5: support ESP SPI match and RSS hash

In packets with ESP header, the inner IP will be encrypted, and
its fields cannot be used for RSS hashing. So, ESP packets
can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient,
because the fields used by the hash functions are only the outer IPs,
causing all traffic belonging to all tunnels between a given
pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPsec packets.

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: fix no-green metering with RSS

When a meter with RSS action being used, there might be several
sub-flows using different sub-policies in the flow splitting stage.
If there's no green action, there's an error that will always use the
same sub-policy for all sub-flows, some resources will be
overwritten and cannot be released, leading assert during port close.

This patch fixes this issue by checking both green and yellow queue
index during getting a blank sub-policy, to avoid the incorrect
resource overwrite.

Fixes: b38a12272b3a ("net/mlx5: split meter color policy handling")
Cc: stable@dpdk.org
Signed-off-by: Shun Hao <shunh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/bnxt: skip wait for link up on port start

Invoking bnxt_link_update_op() with wait_for_completion set would
result in the driver waiting for 10s in case the port link is down to
complete port initialization (dev_start_op()).
Change it by not waiting for the completion when invoking it in
dev_start_op()

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix freeing VNIC filters

In bnxt_free_all_filters(), all the filters attached to a vnic are removed.
But each of these filters hold a backreference ptr to the vnic and they
need to be reset to NULL now. Otherwise, during a normal testpmd quit, as
part of dev_close_op(), first bnxt_free_all_filters() is invoked in
dev_stop, followed by bnxt_free_filter_mem() from bnxt_uninit_resources(),
which finds a filter with a vnic back reference ptr and now
bnxt_hwrm_clean_up_l2_filter() also tries to remove the filter from the
vnic's filter list which was already done as part of
bnxt_free_all_filters().

Fixes: f0f6b5e6cf94 ("net/bnxt: fix reusing L2 filter")
Cc: stable@dpdk.org
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: recheck FW readiness if in reset process

If Firmware is still in reset process and returns the error
HWRM_ERR_CODE_HOT_RESET_PROGRESS, retry VER_GET command.
We have to do it in bnxt_handle_if_change_status().

Fixes: 0b533591238f ("net/bnxt: inform firmware about IF state changes")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix link status when port is stopped

Driver forces link down during port stop. But device is not obliged
link down in certain scenarios, even when forced. In that case,
subsequent link queries returns link as up.
Fixed to return link status as down when port is stopped.
Driver is already doing that for VF/NPAR/MH functions.

Fixes: c09f57b49c13 ("net/bnxt: add start/stop/link update operations")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: force PHY update on certain configurations

Device is not obliged link down in certain scenarios, even
when forced. When FW does not allow any user other than the BMC
to shutdown the port, bnxt_get_hwrm_link_config() call always
returns link up. Force phy update always in that case,
else user configuration for speed/autoneg would not get applied
correctly.

Fixes: 7bc8e9a227cc ("net/bnxt: support async link notification")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: fix speed autonegotiation

The "active_fec_signal_mode" in HWRM_PORT_PHY_QCFG response
does not return correct value till the link is up. Driver cannot
rely on active_fec_signal_mode while setting autoneg speed.

While setting autoneg speed, driver is currently checking only
"auto_link_speed_mask". Fixed to check "auto_pam4_link_speed_mask"
as well. Also, while setting auto mode and setting speed mask,
driver will have to set both NRZ and PAM4 mask.

Fixes: c23f9ded0391 ("net/bnxt: support 200G PAM4 link")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: avoid unnecessary endianness conversion

The "active_fec_signal_mode" in HWRM_PORT_PHY_QCFG response is uint8_t.
So no need of endianness conversion while parsing response.
Also, signal_mode is the first 4bits of "active_fec_signal_mode".

Fixes: c23f9ded0391 ("net/bnxt: support 200G PAM4 link")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: handle queue stop during RSS flow create

The programming of the RSS table was not taking into account if
any of the queues in the set were stopped prior to the flow
creation, hence leading to a vnic RSS config cmd failure thrown by
the FW.
Fix by programming only the active queues in the RSS action queue
set.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: check duplicate queue IDs

Currently driver does not have a check for duplicate queue ids.
User must either specify all Rx queues created or no queues in the
flow create command. Repeating the queue index is invalid.

Also, moved the check for invalid queue to the beginning of the function.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: fix ring group on Rx restart

When an Rx queue is stopped and restarted, as part of that workflow,
for cards that have ring groups, we free and reallocate the ring group.
This new ring group is not communicated to the VNIC though via
HWRM_VNIC_CFG cmd.
Fix to issue HWRM_VNIC_CFG cmd on all adapters now in this scenario.

Fixes: ed0ae3502fc9 ("net/bnxt: update ring group after ring stop start")
Cc: stable@dpdk.org
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: fix RSS action

Specifying a subset of Rx queues created by the application in
the "flow create" command is invalid.
User must either specify all Rx queues created or no queues.

Also removed a wrong comment as RSS action will not be supported
if user or application specifies MARK or COUNT action.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove count action support

'Count' action was never really implemented in the legacy/AFM model.
But there was some place holder code, remove it so that the user
will see a failure when a flow with 'count' action is being
created.

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix tunnel stateless offloads

The HW only supports tunnel header parsing globally for supported tunnel
types. When a function uses one default VNIC to receive both the tunnel
and non-tunnel packets, applying the same stateless offload operation to
both tunnel and non-tunnel packets can cause problems in certain scenarios.
To workaround these problems, the firmware advertises no tunnel header
parsing capabilities to the driver using the HWRM_FUNC_QCAPS.
The driver must check this flag setting and accordingly not advertise
tunnel packet stateless offload capabilities to the stack.

If the device supports VXLAN, GRE, IPIP and GENEVE tunnel parsing,
then reports RX_OFFLOAD_OUTER_IPV4_CKSUM, RX_OFFLOAD_OUTER_UDP_CKSUM
and TX_OFFLOAD_OUTER_IPV4_CKSUM in the Rx/Tx offload capabilities of
the device.
Also, advertise tunnel TSO capabilities based on FW support.

Fixes: 0a6d2a720078 ("net/bnxt: get device infos")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: fix Rx configuration

We are currently not handling RX/RSS modes correctly.
After launching testpmd with multiple RXQs, if the user tries to set
the number of RXQs to 1, driver is not updating the "hash_type"
and "hash_mode" values of the VNICs. As a result, driver issues
bnxt_vnic_rss_configure() unnecessarily and the FW command fails.

Fixed bnxt_mq_rx_configure() to update VNIC RSS fields unconditionally.

Fixes: 4191bc8f79a8 ("net/bnxt: handle multi queue mode properly")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: remove unused macro

BNXT_FLAG_UPDATE_HASH is redundant now, remove it.

Fixes: 1ebb765090a6 ("net/bnxt: fix config RSS update")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: fix device capability reporting

1. Added two functions bnxt_get_tx_port_offloads() and
   bnxt_get_rx_port_offloads() to report the device
   tx/rx offload capabilities to the application.
2. This avoids few duplicate code in the driver and make
   VF-rep capability the same as VF.
3. This will help in selectively reporting offload capabilities
   based on FW support.

Fixes: 0a6d2a720078 ("net/bnxt: get device infos")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: update HWRM structures

Brought in the latest hsi_struct_def_dpdk.h.
HWRM API is now updated to version "1.10.2.83".

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: fix reordering in NEON Rx

Rx descriptor contains a valid bit which indicates readiness of the rest
of descriptor words. Hence, the word contains valid bit must be read
prior to other words.

In NEON vector path, two contiguous 8B descriptor are loaded to a single
NEON register. Given vector load ensures no 16B atomicity, read of the
word that includes valid bit could be reordered after read of other words.
In this case, data could be invalid.

Reloaded lower 64b after read barrier. This ensures what fetched is
correct.

Also fixed comments that not pertains to Arm platform architecture.

Fixes: deae85145c64 ("net/bnxt: handle multiple packets per loop in vector Rx")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove redundant ifdefs

NEON vector path is built only when Arm platform is 64-bit.
The ifdefs in NEON path are of no use, hence remove.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: defer completion index update

When no packet is received, there is no need to update completion raw cons.
Moved update down to remove unnecessary store in this case.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

common/cnxk: support per-port RQ in inline device

Add support for per port RQ in inline device thereby using
Aura/Pool attributes from that port specific first RQ.
When inline device is used with channel masking, it will
fallback to single RQ for all ethdev ports.

Also remove clamping up of CQ size for LBK ethdev when
inline inbound is enabled as now backpressure is supported
even on LBK ethdevs.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix hotplug detach for first device

Fix hotplug detach sequence to handle case where first PCI
device that is hosting NPA LF is being destroyed while in use.

Fixes: 5a4341c84979 ("net/cnxk: add platform specific probe and remove")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix multi-segment extraction in vwqe path

Fix multi-seg extraction in vwqe path to avoid updating mbuf[]
array until it is used via cq0 path.

Fixes: 7fbbc981d54f ("event/cnxk: support vectorized Rx event fast path")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: perform early MTU setup for event mode

Perform early MTU setup for event mode path in order
to update the Rx/Tx offload flags before Rx adapter setup
starts.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support flow control for outbound inline

Add support for flow control in outbound inline path using
FC updates from CPT.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support security statistics

Enabled rte_security stats operation based on the configuration
of SA options set while creating session.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: add capabilities for IPsec options

Added supported capabilities for various IPsec SA options.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: add capabilities for IPsec crypto algos

Added supported crypto algorithms for inline IPsec
offload.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: update L3/L4 checksum offload in mbuf

When the packet is processed with inline IPsec offload,
the ol_flags were updated only with RTE_MBUF_F_RX_SEC_OFFLOAD.

But the hardware can also update the L3/L4 csum offload flags.
Hence, ol_flags are updated with RTE_MBUF_F_RX_IP_CKSUM_GOOD,
RTE_MBUF_F_RX_L4_CKSUM_GOOD, etc based on the microcode completion
codes.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: optimize Rx fast path for security offload

Optimize Rx fast path for security packets by preprocessing
most of the operations such as sa pointer compute,
inner WQE pointer fetch and microcode completion translation
before the pkt is characterized as inbound inline pkt.

Preprocessed info will be discarded if packet is not
found to be security pkt. Also fix fetching of CQ word5
for vector mode. Get ucode completion code from CPT parse
header and RLEN from IP4v/IPv6 decrypted packet as it is
in same 64B cacheline as CPT parse header in most of
the cases. By this method, we avoid accessing an extra
cacheline

Fixes: c062f5726f61 ("net/cnxk: support IP reassembly")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support decrement TTL for inline IPsec

Added support for decrementing TTL(IPv4)/hoplimit(IPv6)
while doing inline IPsec processing if the security session
SA options is enabled with dec_ttl.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: reset offload flag if reassembly is disabled

The rx offload flag need to be reset if IP reassembly flag
is not set while calling reassembly_conf_set.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: update environment variable for debug IV

Changed environment variable name for specifying
debug IV for unit testing of inline IPsec offload
with known test vectors.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: update inline device in ethdev telemetry

Inline PF_FUNC is updated in ethdev_tel_handle_info(),
when inline device is attached to any dpdk process

Signed-off-by: Rakesh Kudurumalla <rkudurumalla@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix roundup size with transport mode

For transport mode, roundup needs to be based on L4 data
and shouldn't include L3 length.

By including L3 length, rlen that is calculated and put in
send hdr would cross the final length of the packet in some
scenarios where padding is necessary.

Also when outer and inner checksum offload flags are enabled,
get the l2_len and l3_len from il3ptr and il4ptr.

Fixes: 55bfac717c72 ("net/cnxk: support Tx security offload on cn10k")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: disable default inner checksum for outbound inline

Disable default inner L3/L4 checksum generation for outbound inline
path and enable based on SA options or RTE_MBUF flags as per
the spec. Though the checksum generation is not impacting much
performance, it is overwriting zero checksum for UDP packets
which is not always good.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: add barrier after meta batch free in scalar

Add barrier after meta batch free in scalar routine when
LMT lines are exactly full to make sure that next LMT line user
in Tx only starts writing the lines only when previous stoerl's
are complete.

Fixes: 4382a7ccf781 ("net/cnxk: support Rx security offload on cn10k")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: update LBK ethdev link info

Update link info of LBK ethdev i.e AF's VF's as always up
and 100G. This is because there is no phy for the LBK interfaces
and driver won't get a link update notification for the same.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support loopback mode on AF VF

Support internal loopback mode on AF VF's using ROC by setting
Tx channel same as Rx channel.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: use aggregate level RR priority from mbox

Use aggregate level Round Robin Priority from mbox response instead of
fixing it to single macro. This is useful when kernel AF driver
changes the constant.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: convert warning to debug print

Inbound SA SPI if not in min-max range specified in devargs,
was marked as a warning. But this is not converted to debug
print because if the entry is found to be duplicate in the mask,
it will give another error print. Hence, warning print is not needed
and is now converted to debug print.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: fix soft expiry disable path

Fix issues in mode where soft expiry is disabled in ROC.
When soft expiry support is not enabled in inline device,
memory is not allocated for the ring base array and should
not be accessed.

Fixes: bea5d990a93b ("net/cnxk: support outbound soft expiry notification")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: skip probing SoC environment for CN9K

SoC run platform file is not present in CN9k so probing
is done for CN10k devices

Signed-off-by: Rakesh Kudurumalla <rkudurumalla@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: fix SQ flush sequence

Fix SQ flush sequence to issue NIX RX SW Sync after SMQ flush.
This sync ensures that all the packets that were in-flight are
flushed out of memory.

This patch also fixes NULL return issues reported by
static analysis tool in Traffic Manager and sync's mailbox
to that of the kernel version.

Fixes: 05d727e8b14a ("common/cnxk: support NIX traffic management")
Fixes: 0b7e667ee303 ("common/cnxk: enable packet marking")
Cc: stable@dpdk.org
Signed-off-by: Satha Rao <skoteshwar@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: support timestamp PKIND in CPT

Add new API to configure the SA table entries with new CPT PKIND
when timestamp is enabled.

Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add new PKIND for CPT when ts is enabled

With timestamp enabled, time stamp will be added to second pass packets
from CPT. NPC needs different configuration to parse second pass packets
with and without timestamp.
New PKIND is defined for CPT when time stamp is enabled on NIX.
CPT should use this PKIND for second pass packets when TS is enabled for
corresponding ethdev port.

Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: add receive channel backpressure for SDP

The SDP interfaces also need to be configured for NIX receive channel
backpressure for packet receive.

Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: support multi channel for SDP send queues

Currently only base channel number is configured as default
channel for all the SDP send queues. Due to this, packets
sent on different SQ's are landing on the same output queue
on the host. Channel number in the send queue should be
configured according to the number of queues assigned to the
SDP PF or VF device.

Signed-off-by: Subrahmanyam Nilla <snilla@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support IPv6 fragment flow pattern item

Support matching IPv6 fragment extension header
with RTE_FLOW_ITEM_TYPE_IPV6_FRAG_EXT flow pattern item.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: enable 3DES-CBC capability

Enable 3DES-CBC capability of inline crypto device.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support custom SA index

Adding cnxk device driver support to configure custom SA index.
Custom SA index can be configured as part of the session create
as SPI, and later original SPI can be updated using session update.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support parsing custom SA action

Adding devargs support to parse custom SA action.
Devargs can be specified in the following way.
-a 0002:02:00.0,custom_sa_act=1

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: support parsing custom SA action

Adding ROC Flow changes to parse custom SA action for cnxk device.
When custom sa action is enabled, VTAG actions are not allowed.
And custom SA index will be calculated based on SA_HI and SA_LO
values. This allows the potential for a MCAM entry to match
many SAs, rather than only match a single SA.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix out of bounds accesses

Fix out of bounds access in cmd array which was reported in coverity
scan.

Coverity issue: 375245, 375246, 375260, 375263, 375264, 375271
Coverity issue: 375278, 375279, 375273, 375274, 375275, 375276
Coverity issue: 375280, 375281, 375283, 375286
Fixes: c5b97e98837 ("net/cnxk: add cn10k template Tx functions to build")
Fixes: dd8c20eee47 ("net/cnxk: add cn9k template Tx functions to build")
Cc: stable@dpdk.org
Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix uninitialized variables

Fix uninitialized scalar variable reported in coverity scan.

Coverity issue: 371876, 371877
Coverity issue: 371872, 371880
Fixes: 39dc567c195 ("net/cnxk: add Tx burst for CN9K")
Fixes: 19f3af2371a ("net/cnxk: add Tx burst for CN10K")
Cc: stable@dpdk.org
Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: fix sizeof not portable

Fix sizeof not portable issue reported in coverity scan.

Coverity issue: 376538
Fixes: 7e9a94909ee ("common/cnxk: realloc inline device XAQ AURA")
Cc: stable@dpdk.org
Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: fix null pointer dereference

Fix null pointer dereference reported in coverity scan.

Coverity issue: 372065
Fixes: 665b6a7400b ("common/cnxk: add NPC helper API")
Cc: stable@dpdk.org
Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: fix unaligned access to device memory

Use mbox_memcpy() instead of memcpy() to prevent unaligned access.

Fixes: 1f66919817e ("common/cnxk: improve MCAM entries management")
Cc: stable@dpdk.org
Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: add message on flow parsing failure

Set the error message with rte_flow_error_set() API
when flow parsing fails.

Fixes: 8c009b4505e ("net/cnxk: support flow API")
Cc: stable@dpdk.org
Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Reviewed-by: Kiran Kumar K <kirankumark@marvell.com>

net/cnxk: use ROC errata API

Used errata APIs roc_errata_nix_has_no_drop_re() and
roc_errata_nix_has_cq_min_size_4k() instead of direct
ROC model check.

Signed-off-by: Rahul Bhansali <rbhansali@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add ROC errata list

Created roc_errata.h to list the errata handled in userspace drivers.
Added no_drop_re, cq_min_size_4k, no_fc_stype_ststp, no_drop_aging,
no_vwqe_flush_op etc erratas.

Signed-off-by: Rahul Bhansali <rbhansali@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: fix crash during hotplug detach operation

hot_plug application does not perform any port setup
configuration via rte_eth_dev_configure() API. All the probed
Ethernet ports do not contain any Rx and Tx queues.

While detaching a device via rte_eal_hotplug_remove(), CNXK
driver expects Rx and Tx queues structures populated during
reset of PFC. So application gets crashed as data->rx_queues
and data->tx_queues are NULL.

Fixes: 9544713564f5 ("net/cnxk: support priority flow control")
Cc: stable@dpdk.org
Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add null check for inline device

Inline device could be null in cases when it was not bound or missing.
Added check to prevent null pointer access.

Fixes: fe5846bcc07 ("net/cnxk: add devargs for min-max SPI")
Cc: stable@dpdk.org
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>

common/cnxk: fix QinQ ROC item mismatch

ROC code is assuming presence of vlan extension headers in
case of QinQ, because of this, there is incompatibility
between the driver and ROC. Fixed this in ROC by treating
QINQ as multiple VLAN pattern items for DPDK (as opposed to
treating QINQ as separate pattern item).

Fixes: b8ac8b089ce ("common/cnxk: support matching VLAN existence")
Cc: stable@dpdk.org
Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: correct ROC naming conventions

Corrected some names to follow ROC naming convention in ROC NPC code.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support mark pattern item type

Added support for RTE_FLOW_ITEM_TYPE_MARK. This item type
can be used to create ingress flow rules to match packets
from CPT's second pass packets.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: support CPT second pass flow rules

Added support to create flow rules to match packets
from CPT's second pass packets. With this change, ingress
rules will be created with bits 10 and 11 of channel field
in the MCAM ignored by default. For rules specific to
second pass packets, the CPT channel bits will be set
in the MCAM.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

app/testpmd: fix MTU verification

The macro RTE_ETHER_MIN_LEN isn't the minimum value of MTU. But testpmd
used it when execute 'port config mtu 0 xx' cmd. This patch fixes it.

Fixes: 1bb4a528c41f ("ethdev: fix max Rx packet length")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@xilinx.com>

app/testpmd: check statistics query before printing

In function 'fwd_stats_display', if function 'rte_eth_stats_get' fails,
'stats' is uncertainty value. The display result will be abnormal.

This patch check the return value of 'rte_eth_stats_get' to avoid
display abnormal stats.

Fixes: 53324971a14e ("app/testpmd: display/clear forwarding stats on demand")
Cc: stable@dpdk.org
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Aman Singh <aman.deep.singh@intel.com>

net/hns3: remove unnecessary RSS switch

Whether the RSS is enabled depends on RTE_ETH_MQ_RX_RSS_FLAG and packet
tuple are enabled. So the RSS switch is unnecessary.

Fixes: 5e782bc2570c ("net/hns3: fix configuring RSS hash when rules are flushed")
Fixes: fd8196838763 ("net/hns3: fix configuring device with RSS enabled")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

ethdev: fix RSS update when RSS is disabled

The RTE_ETH_MQ_RX_RSS_FLAG flag is a switch to enable RSS. If the flag
is not set in dev_configure, RSS will be not configured and enabled.
However, RSS hash and reta can still be configured by ethdev ops to
enable RSS if the flag isn't set. The behavior is inconsistent.

Fixes: 99a2dd955fba ("lib: remove librte_ prefix from directory names")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@xilinx.com>

net/hns3: remove redundant RSS tuple field

The 'rss_tuple_fields' in struct struct hns3_rss_conf::rss_tuple_sets is
redundant. Because the enabled RSS tuple in PMD is already managed by
the 'types' in struct hns3_rss_conf::conf. This patch removes this
redundant variable.

Fixes: c37ca66f2b27 ("net/hns3: support RSS")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix rollback on RSS hash update

The RSS tuple isn't restored when RSS key length is invalid or setting
algo key failed. This patch fixes it.

Fixes: c37ca66f2b27 ("net/hns3: support RSS")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix RSS disable

Currently, hns3 PMD disable RSS by resetting redirection table when user
set rss_hf to 0 so as to all packets go to queue 0. The implementation
may cause following problems:
1) the same type packet may go to different queue on the case of
   disabling all tuples and partial tuples. The problem is determined by
   hardware design.
2) affect the configuration of redirection table and user experience.

For hns3 hardware, the packets with RSS disabled are always go to the
queue corresponding to first entry of the redirection table. Generally,
disable RSS should be implemented by disabling all tuples, This patch
fix the implementation.

Fixes: c37ca66f2b27 ("net/hns3: support RSS")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix mbuf free on Tx done cleanup

Currently, the hns3 PMD may free more mbufs than free_cnt parameter,
this is an incorrect implementation. This patch fixes it.

Fixes: 0b77e8f3d364 ("net/hns3: optimize Tx performance")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix pseudo-sharing between threads

Some fields in the end of 'struct hns3_rx_queue' and
'struct hns3_tx_queue' are not accessed in the I/O path.
But these fields may be accessed in other threads, which may lead to the
problem of cache pseudo-sharing of IO threads. This patch add a
cacheline alignment to avoid it.

Fixes: 9261fd3caf1f ("net/hns3: improve IO path data cache usage")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix MAC and queues HW statistics overflow

The MAC and queues statistics are 32-bit registers in hardware. If
hardware statistics are not obtained for a long time, these statistics
will be overflow.
So PF and VF driver have to periodically obtain and save these
statistics. Since the periodical task and the stats API are in different
threads, we introduce a statistics lock to protect the statistics.

Fixes: 8839c5e202f3 ("net/hns3: support device stats")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix order of clearing imissed register in PF

Clearing imissed registers in PF hardware depends on the
'drop_stats_mode' in struct hns3_hw. The variable is initialized after
the "hns3_get_configuration". But, in current code, the clearing
operation runs before the function.
So this patch fixes this order. In addition, this patch extracts a
public function to initialize and uninitialize statistics to improve the
maintainability of these codes.

Fixes: 3e9f3042d7c8 ("net/hns3: add imissed packet stats")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

ethdev: fix build with vtune option

When VTune profile option is enabled, compilation error is seen.

Bugzilla ID: 1004
Fixes: f9bdee267ab8 ("ethdev: hide internal structures")
Cc: stable@dpdk.org
Signed-off-by: Aman Singh <aman.deep.singh@intel.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix interrupt handler freeing

rte_pmd_tun/tap_probe() allocates pmd->intr_handle in eth_dev_tap_create()
and it should not be freed until rte_pmd_tap_remove() is called.

Inspection of tap_rx_intr_vec_set() shows that the call to
tap_tx_intr_vec_uninstall() was calling rte_intr_instance_free() but
tap_tx_intr_vec_install() can then be immediately called, and this then
uses pmd->intr_handle without it being reallocated.

Move rte_intr_instance_free() call from tap_tx_intr_vec_uninstall()
to rte_pmd_tap_remove().

Fixes: d61138d4f0e2 ("drivers: remove direct access to interrupt handle")
Cc: stable@dpdk.org
Signed-off-by: Quentin Armitage <quentin@armitage.org.uk>
Reviewed-by: David Marchand <david.marchand@redhat.com>

net/bonding: fix slave stop and remove on port close

All slaves will be stopped and removed when closing a bonded port.
But the while loop can not end if both rte_eth_dev_stop and
rte_eth_bond_slave_remove fails, runs infinitely.
This is because the skipped slave port counted in both function failures
but it should be counted only one.

Fixing by not continue to process in the loop after first failure.

Fixes: fb0379bc5db3 ("net/bonding: check stop call status")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/bonding: fix stopping non-active slaves

When stopping a bonded port, all slaves should be stopped. But only
active slaves are stopped.
So fix by stopping all slave ports and later do "deactivate_slave()" for
active slaves.

Fixes: 0911d4ec0183 ("net/bonding: fix crash when stopping mode 4 port")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/ice: support power management in DCF mode

Implement support for the power management API by implementing a
'get_monitor_addr' function that will return an address of an RX ring's
status bit.

Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support DCF queue information query

Add below ops,
rxq_info_get
txq_info_get

Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support extended statistics in DCF mode

Add implementation of xstats() functions in DCF PMD.

Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support DCF VLAN filter and offload

Below PMD ops are supported in this patch:
    .vlan_filter_set  = dcf_dev_vlan_filter_set
    .vlan_offload_set = dcf_dev_vlan_offload_set

Signed-off-by: Alvin Zhang <alvinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support DCF MAC configuration

Below PMD ops are supported in this patch:
    .mac_addr_add      = dcf_dev_add_mac_addr
    .mac_addr_remove   = dcf_dev_del_mac_addr
    .set_mc_addr_list  = dcf_set_mc_addr_list
    .mac_addr_set      = dcf_dev_set_default_mac_addr

Signed-off-by: Alvin Zhang <alvinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support DCF promiscuous configuration

Support configuration of unicast and multicast promisc on dcf.

Signed-off-by: Alvin Zhang <alvinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: add supported packet types query to DCF mode

Add API "dev_supported_ptypes_get" to DCF, that DCF pmd can get
ptypes through the new API.

Signed-off-by: Jie Wang <jie1x.wang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>