eal: calibrate TSC only in primary process

This ensures secondary processes never have to calculate the TSC rate
themselves, which can be noticeable in VMs that don't have access to
arch-specific detection mechanism (such as CPUID leaf 0x15 or MSR 0xCE
on x86).

Since rte_mem_config is now internal to the EAL library, we can add
tsc_hz without ABI breakage concerns.

Reduces rte_eal_init() execution time in a secondary process from 165ms
to 66ms on my test system.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

rcu: fix spurious thread unregister

Thread unregister returns success while unregister not been performed.
This is due to incorrect thread registration status check.
Fix this issue by correcting bitmap check.

Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

service: avoid false sharing on core state

For a valid service, the core mask of the service
is checked against the current core and the corresponding
entry in the active_on_lcore array is set or reset.

Upto 8 cores share the same cache line for their
service active_on_lcore array entries since each entry is a uint8_t.
Some number of these entries also share the cache line with
the internal_flags member of struct rte_service_spec_impl,
hence this false sharing also makes the service_valid() check
expensive.

Eliminate false sharing by moving the active_on_lcore array to
a per-core data structure. The array is now indexed by service id.

Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Acked-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

timer: remove useless check on x86 TSC reliability

This code was added 7+ years ago in
commit fb022b85bae4 ("timer: check TSC reliability")
presumably when variant TSCs were still somewhat common.

But this code doesn't do anything except print a warning,
and the warning doesn't give any kind of advice to the user,
so let's just remove it.

While the warning has no functional meaning, the /proc/cpuinfo
parsing consumes a non-trivial amount of time which is especially
noticeable in secondary processes.
On my test system, it consumes 21ms out of the 66ms total execution
time for rte_eal_init() in a secondary process.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

eal/ppc: add SPDX license tag

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: David Christensen <drc@linux.vnet.ibm.com>

eal/ppc: fix 64-bit atomic exchange operation

The rte_atomic64_exchange operation for ppc_64 incorrectly linked
back to a 32 bit generic operation (__atomic_exchange_4) rather than
the 64 bit generic operation (__atomic_exchange_8).  As a result,
applications that used rte_eth_link_get_nowait() would only receive
the link speed, they would not receive the link state, link duplex,
or link autoneg properties.

Fixes: ff2863570fcc ("eal: introduce atomic exchange operation")
Cc: stable@dpdk.org
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

mbuf: add a copy routine

This is a commonly used operation that surprisingly the
DPDK has not supported. The new rte_pktmbuf_copy does a
deep copy of packet. This is a complete copy including
meta-data.

It handles the case where the source mbuf comes from a pool
with larger data area than the destination pool. The routine
also has options for skipping data, or truncating at a fixed
length.

This patch also introduces internal inline to copy the
metadata fields of mbuf.

Add a test for this new function, based of the clone tests.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: deinline clone function

Cloning mbufs requires allocations and iteration
and therefore should not be an inline.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mbuf: deinline linearize function

This copy part of this function is too big to be put inline.
The places it is used are only in special exception paths
where a highly fragmented mbuf arrives at a device that can't handle it.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

test/mbuf: fix forged mbuf in clone test

The test for cloning changed mbuf would generate an mbuf whose length
and segments count were invalid.
This would cause a crash if test was run with mbuf debugging enabled.

Fixes: 4ccd2bb3a9e2 ("app/test: enhance mbuf refcnt check")
Fixes: af75078fece3 ("first public release")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mempool: clarify default populate function

No functional change.
Clarify the populate function to make future changes easier to
understand.

Rename the variables:
- to avoid negation in the name
- to have more understandable names

Remove useless variable (no_pageshift is equivalent to pg_sz == 0).

Remove duplicate affectation of "external" variable.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

kni: support allmulticast mode set

This patch adds support to allow users enable/disable allmulticast mode for
kni interface.

This requirement comes from bugzilla 312, more details can refer to:
https://bugs.dpdk.org/show_bug.cgi?id=312

Bugzilla ID: 312

Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

service: use log for error messages

EAL should always use rte_log instead of putting errors to
stderr (which maybe redirected to /dev/null in a daemon).

Also checks for null before rte_free are unnecessary.
Minor code consistency improvements.

Fixes: 21698354c832 ("service: introduce service cores concept")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

eal: fix mapping leak in secondary process

Have rte_eal_config_reattach clean up the mapped address which is a valid
address but not the one intended.

Coverity issue: 343439
Fixes: 4e8854ae89fa ("eal: do not panic on shared memory init")
Fixes: b149a7064261 ("eal/freebsd: add config reattach in secondary process")
Cc: stable@dpdk.org
Signed-off-by: Arnon Warshavsky <arnon@qwilt.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

vfio: fix leak with multiprocess

The code checks both rte_mp_request_sync() return code and that the number
of messages in the reply equals 1.  If rte_mp_request_sync() succeeds but
there was more than one message, those messages would get leaked.

Found via code review by Anatoly Burakov of patches that used the vhost
code as a template for using rte_mp_request_sync().

Fixes: 83a73c5fef66 ("vfio: use generic multi-process channel")
Cc: stable@dpdk.org
Reported-by: Anatoly Burakov <anatoly.burakov@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

build: avoid overlinking

A while ago telemetry was added in 57ae0ec6 and it also added as-needed
to config/meson.build. This seems no more needed these days as due to other
build changes the ordering in buildlogs is:
  [...] -lrte_telemetry [...] -Wl,--no-as-needed [...]
Which means telemetry no more benefits from --no-as-needed anyway.

Overlinking problems get triggered by the meson generated pkgconfig which
will have:
   [...] -Wl,--no-as-needed <somelibsusedbydpdk>
This will overlink <somelibs> and in addition anything that follows
as it also doesn't wrap back to --as-needed. So if a projects includes
dpdk libs + <other> it will also consider <other> with --no-as-needed.

Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1841759
Fixes: 57ae0ec62620 ("build: add dependency on telemetry to apps with meson")
Cc: stable@dpdk.org
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Acked-by: Luca Boccassi <bluca@debian.org>

bpf: hide internal program argument type

RTE_BPF_ARG_PTR_STACK is used as internal program
arg type. Rename to RTE_BPF_ARG_RESERVED to
avoid exposing internal program type.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add branch operation

Add branch and call operations.

jump_offset_* APIs used for finding the relative offset
to jump w.r.t current eBPF program PC.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add atomic-exchange-and-add operation

Implement XADD eBPF instruction using STADD arm64 instruction.
If the given platform does not have atomics support,
use LDXR and STXR pair for critical section instead of STADD.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add load and store operations

Add load and store operations.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add byte swap operations

add le16, le32, le64, be16, be32 and be64 operations.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add logical operations

Add OR, AND, NEG, XOR, shift operations for immediate
and source register variants.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add basic arithmetic operations

Add mov, add, sub, mul, div and mod arithmetic
operations for immediate and source register variants.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add prologue and epilogue

Add prologue and epilogue as per arm64 procedure call standard.

As an optimization the generated instructions are
the function of whether eBPF program has stack and/or
CALL class.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf/arm: add build infrastructure

Add build infrastructure and documentation
update for arm64 JIT support.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

app/eventdev: add options for mbuf and packet sizes

Add options to set mbuf size and max packet size which allow the user to
enable jumbo frames and Rx/Tx scatter gather.
Arrange `struct evt_options` based on ascending order of data type to
make it more readable.

Packet mbuf size can be modified by using `--mbuf_sz=N`.
Max packet size can be modified by using `--max_pkt_sz=N`.
These options are only applicable `pipeline_atq` and `pipeline_queue`
tests.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

test/event: enable dpaa2 self test

This patch add the support to include dpaa2 event test
from the test framework.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa2: add self test

This patch add support for testing dpaa2 eventdev self test
for basic sanity for parallel and atomic queues.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa2: add retry break in packet enqueue

The patch adds the break in the TX function, if it is failing
to send the packets out. Previously the system was trying
infinitely to send packet out.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>

event/dpaa2: support destroy

This patch add support to destroy the event device

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa2: remove conditional compilation

This patch removes the conditional compilation for
cryptodev event support from RTE_LIBRTE_SECURITY flag.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

event/dpaa2: fix default queue configuration

Test vector expect only one type of scheduling as default.
The old code is provide support scheduling types instead of default.

Fixes: 13370a3877a5 ("eventdev: fix inconsistency in queue config")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

test/event_crypto: check adapter config failure

Return error when the adapter creation fails.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

test/event_crypto: change cipher algo

The existing code uses NULL as the cipher algo
for testing crypto event adapter.
DPAA1/DPAA2 do not support NULL algo. Hence changing
it to the most common algo AES-CBC, which is supported
by all crypto drivers implementing event crypto adapter.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

test/event_crypto: fix mempool name

The longer mempool name size is causing error in
rte_mempool_create_empty for dpaa1

ret = snprintf(mz_name, sizeof(mz_name), RTE_MEMPOOL_MZ_FORMAT, name);
This patch reduce the size of mempool name string

Fixes: 24054e3640a2 ("test/crypto: use separate session mempools")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

event/octeontx2: fix Rx adapter capabilities

Octeontx2 SSO co-processor allows multiple ethernet device Rx queues
connected to a single Event device queue.
Fix the Rx adapter capabilities to allow application to configure
Rx queueus in n:1 ratio to event queues by adding
`RTE_EVENT_ETH_RX_ADAPTER_CAP_MULTI_EVENTQ` as a capability.

Fixes: 37720fc1fba8 ("event/octeontx2: add Rx adapter")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/sw: fix xstats reset value

The sw PMD implements xstats reset by having the xstat get operations
return a value to the statistic's value at the last reset. The value at the
last reset is maintained in the per-xstat reset_value field, but the PMD
was setting reset_value = current - reset_value instead of reset_value =
current.

Fixes: c1ad03df7ad5 ("event/sw: support xstats")
Cc: stable@dpdk.org
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

test/event_crypto: no service core when HW support available

If HW support is available, service core shall not come
into play by default. This shall be for both FWD/NEW modes.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

test/event_crypto: check session init failure

Mismatch in algo or sec capability can cause session to fail.
This patch handle it and return error timely.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

net/mlx5: fix direct call to rdma-core library

The routine mlx5dv_query_devx_port() was called directly
instead of using the mlx5 glue thunk.

Fixes: d5c06b1b10ae ("net/mlx5: query vport index match mode and parameters")

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: fix device scan within switch domain

In LAG configuration the devices in the same switch domain
might be spawned on the base of different PCI devices, so
we should check all devices backed by mlx5 PMD whether they
belong to specified switch domain. When the new devices are
being created it is not possible to detect whether the
sibling devices created in the current probe() loop belong
to the driver, driver field is not filled yet (it will be
done on returned success of current probe()). This patch
updates the device scanning, allowing extra match on
current backing PCI device, is being used to create siblings.

Fixes: f7e95215ac7c ("net/mlx5: extend switch domain searching range")

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: adjust inline setting for large Tx queue sizes

The hardware may have limitations on maximal amount of
supported Tx descriptors building blocks (WQEBB). Application
requires the Tx queue must accept the specified amount of packets.
If inline data feature is engaged the packet may require more WQEBBs
and overall amount of blocks may exceed the hardware capabilities.
Application has to make a trade-off between Tx queue size and maximal
data inline size.

In case if the inline settings are not requested explicitly with
devarg keys the default values are used. This patch adjusts the
applied default values if large Tx queue size is requested and
default inline settings can not be satisfied due to hardware
limitations.

The explicitly requested inline setting may be aligned (enlarging
only) by configurations routines to provide better WQEBB filling,
this implicit alignment is the subject for adjustment either.

The warning message is emitted to the log if adjustment happens.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/cxgbe: fix races on flow API operations

When rules are being inserted from multiple cores, there are several
race conditions during rte_flow operations.

For example, when inserting rules from 2 cores simultaneously, both
the cores try to fetch a free available filter entry and they both
end up fetching the same entry. Both of them start overwriting the
same filter entry before sending to firmware, which results in wrong
rule being inserted to hardware.

Fix the races by adding spinlock to serialize the rte_flow operations.

Fixes: ee61f5113b17 ("net/cxgbe: parse and validate flows")
Fixes: 9eb2c9a48072 ("net/cxgbe: implement flow create operation")
Fixes: da23bc9d33f4 ("net/cxgbe: implement flow destroy operation")
Fixes: 8d3c12e19368 ("net/cxgbe: implement flow query operation")
Fixes: 86910379d335 ("net/cxgbe: implement flow flush operation")
Cc: stable@dpdk.org
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/bnxt: improve CPR handling in vector path

Reduce overhead of CPR descriptor validity checking in vector
receive and transmit functions.

Preserve raw cpr consumer index in vector transmit completion
function.

Remove an unneeded prefetch (per benchmarking) from vector
transmit completion function.

Fixes: bc4a000f2f53 ("net/bnxt: implement SSE vector mode")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: advertise scatter Rx offload capability

Scattered receive is supported but not included in receive offload
capabilities. Fix by adding it and including in scattered receive
calculation.

Fixes: 9c1507d96ab8 ("net/bnxt: switch to the new offload API")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix default Rx queue for Thor

Use first receive queue assigned to VNIC as the default receive queue
when configuring Thor VNICs. This is necessary e.g. in order for flow
redirection to a specific receive queue to work correctly.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix stats context calculation

The required number of statistics contexts is computed as the sum
of the number of receive and transmit rings plus one for the async
completion ring. A statistics context is not actually required for
the async completion ring, so remove it from the calculation.

Fixes: bd0a14c99f65 ("net/bnxt: use dedicated CPR for async events")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: use common NQ ring

Thor queue scaling is currently limited by the number of NQs that
can be allocated. Fix by using a common NQ for all receive/transmit
rings instead of allocating a separate NQ for each ring.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: support CoS classification

Class of Service (CoS) is a way to manage multiple types of
traffic over a network to offer different types of services
to applications. CoS classification (priority to cosqueue) is
determined by the user and configured through the PF driver.
DPDK driver queries this configuration and maps the cos queue
ids to different VNICs. This patch adds this support.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support LRO on Thor adapters

Add support for LRO for adapters based on Thor (BCM57508).

Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix ring alignment for Thor-based adapters

When using transmit/receive queue sizes smaller than 256, alignment
requirements are not being met for Thor-based adapters. Fix by
forcing memory addresses used for transmit/receive/aggregation ring
allocations to be on 4K boundaries.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: increase TQM entry allocation

The current TQM backing store size isn't sufficient to allow 512
transmit rings. Fix by correcting TQM SP queue size calculation.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/atlantic: add FW mailbox guard mutex

Driver uses the Firmware mailbox to read statistics and configure
some features.
This patch introduces a mutex to provide consistent access to the
FW mailbox to prevent potential data corruption.

Fixes: 86d36773bd42 ("net/atlantic: implement firmware operations")
Cc: stable@dpdk.org
Signed-off-by: Pavel Belous <pavel.belous@aquantia.com>
Signed-off-by: Igor Russkikh <igor.russkikh@aquantia.com>

net/atlantic: fix reported flow control mode

Driver reports current flow control mode based on internal flow control
settings. Currently this logic works incorrectly.

Fixes: 921eb6b8ce31 ("net/atlantic: fix flow control by sync settings on Rx")
Cc: stable@dpdk.org
Signed-off-by: Pavel Belous <pavel.belous@aquantia.com>
Signed-off-by: Igor Russkikh <igor.russkikh@aquantia.com>

net/atlantic: exclude MACsec counters from xstats

Currently, driver always return full set of xstats counters, including
MACSEC counters. But this driver also supports AQC100 chips, which
does not have MACSEC feature.
This fix adds checking for MACSEC availability (based on FW capability
bits) and returns xstats without MACSEC counters if MACSEC feature
is not available.

Fixes: 09d4dfa85359 ("net/atlantic: implement MACsec statistics")
Cc: stable@dpdk.org
Signed-off-by: Pavel Belous <pavel.belous@aquantia.com>
Signed-off-by: Igor Russkikh <igor.russkikh@aquantia.com>

net/octeontx2: fix CQE ring prefetch on wrap around

When computing the head of CQE ring of prefetch, use qmask to point to
the correct head index on wrap around case.

Fixes: cc4d7693f2d9 ("net/octeontx2: support Rx")
Cc: stable@dpdk.org
Signed-off-by: Kommula Shiva Shankar <kshankar@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/bnx2x: update to latest FW 7.13.11

Use latest firmware 7.13.11.

7.13.11 FW changelog:
    - Packets from a VF with pvid configured which were sent with a
      different vlan were transmitted instead of being discarded.
    - In some multi-function configurations, inter-PF and inter-VF
      Tx switching is incorrectly enabled.
    - Wrong assert code in FLR final cleanup in case it is sent not
      after FLR.
    - Chip may stall in very rare cases under heavy traffic with FW GRO
      enabled.
    - VF malicious notification error fixes.
    - Default gre tunnel to IPGRE which allows proper RSS for IPGRE
      packets, L2GRE traffic will reach single queue.
    - Removes unnecessary internal mem config, latest FW performs this
      autonomously.

Update the PMD version to 1.1.0.1.

Signed-off-by: Rasesh Mody <rmody@marvell.com>

net/bnx2x: update HSI

Update hardware software common base driver code in preparation to
update the firmware to version 7.13.11.

Signed-off-by: Rasesh Mody <rmody@marvell.com>

net/bnx2x: update and reorganize HW registers

Update and reorganize HW registers in preparation to update the firmware
to version 7.13.11.
Move HW_INTERRUT_ASSERT_SET_0 out from ecore_reg.h to bnx2x.h.

Signed-off-by: Rasesh Mody <rmody@marvell.com>

net/qede: limit Rx ring index read for debug

Caught by clang, this idx value is only used for a debug message when
the mbufs allocation fails.
No need to use idx as a temporary storage.

Fixes: 8f2312474529 ("net/qede: fix performance bottleneck in Rx path")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Rasesh Mody <rmody@marvell.com>

net/octeontx2: support GRE TSO offload

Extends existing TSO support to GRE tunnel on the
same SoC revisions.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/octeontx2: support TSO offload

Add support to below TCP segmentation offloads for
96XX A1 onwards and 95xx B0 onwards.
- TCPv4, TCPv6
- VXLAN[v4 | v6][v4 | v6]
- GENEVE[v4 | v6][v4 | v6]

This patch also modifies a fastpath function to be forced
inline due to performance reasons for multi-seg mode.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/octeontx2: extract NVGRE as ltype

Adding change to sync RTE Flow with KPU profile to extract
NVGRE as ltype.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/octeontx2: support Tx descriptor status

Adding support for tx descriptor status dev ops for octeontx2.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/octeontx2: update KPU parser profile

Update LB ltypes and use the updated ones so as replace
LB_STAG and LB_QINQ by single LB_STAG_QINQ ltype.

Signed-off-by: Vivek Sharma <viveksharma@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/octeontx2: allow VFs to enable back pressure

Allow VFs to enable backpressure for performance reasons.
The backpressure control is with kernel AF driver that will enable
backpressure even if one PF/VF requests it and disable it only
after all the PFs/VFs request for disable.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/octeontx2: fix checksum flag translation

NPC errors were incorrectly translated to ol_flag as
error code enum was not in sync with NPC profile.

Fixes: 371d3212cbed ("common/octeontx2: add build infrastructure and HW definition")
Cc: stable@dpdk.org
Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net: add missing rte prefix for ESP tail

This structure has been missed during the big rework.

Fixes: 5ef254676752 ("net: add rte prefix to ESP structure")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bnxt: fix flow flush handling

We are not freeing all the flows when a flow_flush is called.
Iterate through all the flows belonging to all the VNICs in use and
free the filters.

Fixes: 4ace85a7da25 ("net/bnxt: allocate rings and groups")
Cc: stable@dpdk.org
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: fix multicast filter programming

Fixed multicast filter programming and allmulti programming.
Fixed to skip programming multicast macs if the user requests
allmulti mode.

Also removed a comment in bnxt_hwrm_cfa_l2_set_rx_mask() which is
no longer valid now.

Fixes: d69851df12b2 ("net/bnxt: support multicast filter and set MAC addr")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix VLAN filtering

Currently, when hw-vlan-filter is enabled on testpmd, driver is
receiving all vlan packets. Instead, it should only receive untagged
packets and vlan packets for which the VLAN filter is programmed.
This is because, the default rule to match on MAC is not getting
deleted, when hw-vlan-filter is ON.

This patch fixes the problem, by deleting the default MAC rule and
programming a new rule to receive only untagged packets, when
hw-vlan-filter is enabled & another rule for each vlan, as and when
that vlan is configured on that port.

Fixes: 246c5cc5f05e ("net/bnxt: use correct flags during VLAN configuration")
Cc: stable@dpdk.org
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: drop untagged frames when specified

When a drop action for L2 filters is specified, support it.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: synchronize between flow related functions

Currently, there are four flow related functions, namely
bnxt_flow_create, bnxt_flow_destroy, bnxt_flow_validate,
bnxt_flow_flush. All these functions are not multi-thread safe.

This patch fixes it by synchronizing these functions with a lock.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: handle cleanup if flow creation fails

If flow creation fails because of an HWRM command failure or
or some other reason, reset the vnic and rxq info set earlier.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: validate RSS hash key length

In bnxt_rss_hash_update_op, driver is proceeding with
bnxt_hwrm_vnic_rss_cfg even though RSS hashkey length is invalid.

This patch fixes the problem by returning -EINVAL when RSS hashkey
length is invalid.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: check for VNIC ID in rollback

If driver init/probe fails as part of cleanup/rollback, we may end
up invoking this HWRM cmd even on an invalid vNIC which will
unnecessarily log an error message as the cmd will fail.
Check for invalid ID before issuing the HWRM cmd

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: check if device is started before flow creation

Check device is started before flow creation.
Since the vnic data structures aren't created until device start,
the driver dereferences NULL vnic if flow creation is attempted before
device is started,

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

net/bnxt: handle ring cleanup in case of error

Initialize all rings to INVALID_HW_RING_ID.
This can be used to determine the rings to free if allocation fails.

Signed-off-by: Rahul Gupta <rahul.gupta@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

net/bnxt: allow only unicast MAC address filter creation

Check if the application is trying to create filters using
broadcast and multicast MAC address and reject it.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>

net/bnxt: fix setting default MAC address

Driver was incorrectly programming the MAC with the already
configured one instead of the newly requested MAC by user.

Also, fix to restore the old mac address back to the default
vnic filter if the mac update operation fails.

Fixes: 68f589f2c728 ("net/bnxt: fix setting primary MAC address")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>

net/bnxt: cleanup VNIC after flow validate

When an application issues flow validate, we free the temporary
filter that is created. But the vnic is not freed up. This can
potentially interfere with subsequent flow creation. So free the vnic.

Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: delete and flush L2 filters cleanly

Once the last filter associated with a VNIC is deleted when using
RSS action or the Queue action free the VNIC. Also free the RSS
context if the VNIC is using it.

Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: parse priority attribute for flow creation

Parse priority attribute during flow creation.
This information will be used to give a hint to the FW to
place the flow rule accordingly in the CFA tables.

Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Lance Richardson <lance.richardson@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support RSS action

Add support for RSS action during flow creation.

group id should not be 0 when RSS action is specified. Driver will
return an error for such a flow.

If a group id is used to create a filter with “n” RSS queues, it cannot
be used to create a filter with a different number of RSS queues till
all the flows using that combination are deleted.

While creating a flow if a group id groups a certain Rx queue ids for
RSS, the same group id shall not create a flow with a different group of
Rx queue ids till all the flows belonging to the group ids are deleted.

While creating a flow if a group id groups a certain Rx queue ids for
RSS, the same queue ids shall not be used with a different group id till
all flows created with that group id are deleted.

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

net/bnxt: support creating SMAC and inner DMAC filters

We are currently creating only outer DMAC filters.
Create SMAC and inner DMAC filters using HWRM_CFA_L2_FILTER_ALLOC.
For this the HWRM_CFA_L2_FILTER_ALLOC has already been updated.

Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: allow flow creation when RSS is enabled

Currently flow creation is allowed with queue action only
when RSS is disabled. Remove this restriction. Flows can be
created when RSS is enabled.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

net/bnxt: allow dynamic creation of VNIC

Refactor code to allow dynamic creation of VNIC for RSS
or Queue Action during flow create.

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Santoshkumar Karanappa Rastapur <santosh.rastapur@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>
Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

net/bnxt: return standard error codes for HWRM command

If the FW returns an error for an HWRM request, it does not necessarily
return standard error codes.
Convert these HWRM errors to standard errno.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/szedata2: fix dependency check

The library libsze2 provides a pkg-config file: libsze2.pc.
Looking for this .pc file - with dependency() - is preferred
than looking for the library - with cc.find_library().

If the library is not installed in a standard path,
it can be found thanks to PKG_CONFIG_PATH variable.
The previous solution required to use CFLAGS and LDFLAGS
environment variables.

Fixes: 508cfe6be9f1 ("net/szedata2: add to meson build")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Jan Remes <remes@netcope.com>

net/nfb: fix dependency check

The library libnfb is part of netcope-common which provides
a pkg-config file: netcope-common.pc.
Looking for this .pc file - with dependency() - is preferred
than looking for the library - with cc.find_library().

If the library is not installed in a standard path,
it can be found thanks to PKG_CONFIG_PATH variable.
The previous solution required to use CFLAGS and LDFLAGS
environment variables.

Fixes: 6435f9a0ac22 ("net/nfb: add new netcope driver")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Jan Remes <remes@netcope.com>

net/qede: re-add to meson

qede has been dropped from the drivers/net meson.
Add it again.

Fixes: 9a8864c8b5da ("net/octeontx2: add build and doc infrastructure")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/dpaa2: add soft parser driver

Signed-off-by: Sunil Kumar Kori <sunil.kori@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support soft parser in MC

Signed-off-by: Sunil Kumar Kori <sunil.kori@nxp.com>
Reviewed-by: Sachin Saxena <sachin.saxena@nxp.com>

net/dpaa2: support separate MC portal per process

Earlier, there was a single MCP handle which was available across
complete DPAA2 driver as well as part of the dev_private which was
shared by the secondary process.

For secondary, that is not valid and it would require to open its
own handle for the MC. This is eventually used as part of the DPNI
configuration.

By using the process_private member of the rte_eth_dev, it is
possible to keep separate handles per process. Without worry of
overwriting when secondary process accesses the dev_private.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Sachin Saxena <sachin.saxena@nxp.com>

bus/fslmc: support multi VFIO group

DPAA2 support VFIO device passthrough in VM.
However in this case, each device is associated with different vfio group.

This code required different container id for each group.
On using the same container fd the second time,
ioctl calls are returning error.

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: check for DMA map in primary process only

DMA mapping is a property of primary process - SMMU population done
once by primary doesn't need to be populated again in secondary

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Sachin Saxena <sachin.saxena@nxp.com>

bus/fslmc: restrict address translation to PA mode

The address translation support for PA->VA is required only in
case of PA mode operation of DPDK. This was causing warning to
be reported on running any DPAA2 application in VA mode:

    Add: Incorrect entry for PA->VA Table(xxxxxxxxxx)
    Add: Lowest address: xxxxxxxxxxxx

This was caused by call to update the DPAAX table when VA mode
was enabled, in which case the VA==IOVA address.

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Reviewed-by: Sachin Saxena <sachin.saxena@nxp.com>

net/dpaa2: add PTP driver

This patch adds the support for PTP driver for
DPAA2 devices.

To enable set
CONFIG_RTE_LIBRTE_IEEE1588=y in
config/defconfig_arm64-dpaa2-linuxapp-gc

Signed-off-by: Priyanka Jain <priyanka.jain@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: add DPRTC sub-module

This patch add support for dprtc
(Data Path Real Time Clock) sub-module
based on MC 10.16.0 flib.

It provides interface for PTP stack software
to access physical IEEE 1588 Real Time Clock.

Signed-off-by: Priyanka Jain <priyanka.jain@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support timestamp

IEEE1588 driver needs timestamp of packets.
For DPAA2, the timestamp of TX packets is
stored in annotation area of corresponding
TX confirmation packet.

This patch enables timestamp fields in
annotation area and TX confirmation mode if
CONFIG_RTE_LIBRTE_IEEE1588 is set in
config/defconfig_arm64-dpaa2-linuxapp-gc

Signed-off-by: Priyanka Jain <priyanka.jain@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>