net/cnxk: add Rx burst for CN9K

Add Rx burst scalar version for CN9K.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: support queue start and stop

Add Rx/Tx queue start and stop callbacks for
CN9K and CN10K.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: support packet type

Add support for packet type lookup on Rx to translate HW
specific types to  RTE_PTYPE_* defines

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add Tx queue setup and release

aDD tx queue setup and release for CN9K and CN10K.
Release is common while setup is platform dependent due
to differences in fast path Tx queue structures.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add Rx queue setup and release

Add Rx queue setup and release op for CN9K and CN10K
SoC. Release is completely common while setup is platform
dependent due to fast path Rx queue structure variation.
Fastpath is platform dependent partly due to core cacheline
size difference.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: support link status update

Add link status update callback to get current
link status.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add device configuration operation

Add device configuration op for CN9K and CN10K. Most of the
device configuration is common between two platforms except for
some supported offloads.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: support device infos query

Add support to retrieve dev infos get for CN9K and CN10K.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add common devargs parsing

Add various devargs parsing command line arguments
parsing functions supported by CN9K and CN10K.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add platform specific probe and remove

Add platform specific probe and remove callbacks for CN9K
and CN10K which use common probe and remove functions.
Register ethdev driver for CN9K and CN10K.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: add build infra and common probing

Add build infrastructure and common probe and remove for cnxk driver
which is used by both CN10K and CN9K SoC.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

common/cnxk: support VLAN push/pop flow actions

Add roc API to configure VLAN tag addition and removal.

This patch also adds 98xx support for increased MCAM
entries for rte flow.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Reviewed-by: Kiran Kumar K <kirankumark@marvell.com>

common/cnxk: remove camel case from model API

Change model check API's to not use Camel case in function
names.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

common/cnxk: add provision to enable RED on RQ

Send RED pass/drop levels based on rq configurations to kernel.
Fixed the aura and pool shift value calculation.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

common/cnxk: allocate LMT region in userspace

As per the new LMTST design, userspace shall allocate LMT region,
setup the DMA translation and share the IOVA with kernel via MBOX.
Kernel will convert this IOVA to physical memory and update the
LMT table entry with the same.
With this new design also shared mode (i.e. all pci funcs sharing
the LMT region allocated by primary/base pci func) is intact.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: support flow action mark and flag

Add roc API to get mark action.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>

common/cnxk: support flow entry dump

Add NPC support API to dump created flow entries.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>

common/cnxk: fix batch alloc completion poll logic

The instruction generation was not correct due to
fact that volatile suppose to use with ccode variable
as well.

Change the logic to use gcc atomic builtin to
simplify and avoid explicit volatile from the code.

Fixes: 81af26789316 ("common/cnxk: support NPA batch alloc/free")
Cc: stable@dpdk.org
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com>

common/cnxk: support locking NIX RQ contexts

This patch will consider device argument to lock RSS table
in NIX.

This patch also adds few misc fixes such as disabling NIX Tx
vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ
for Tx completions and TM related stats API.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

test/mbuf: fix virtual address conversion

Seen with address sanitizer.

rte_mempool_virt2iova() can only be used on mempool elements. In this case,
it is incorrect, and rte_mem_virt2iova() has to be used.

Bugzilla ID: 737
Fixes: 7b295dceea07 ("test/mbuf: add unit test cases")
Cc: stable@dpdk.org
Reported-by: Zhihong Peng <zhihongx.peng@intel.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

test: remove hack for private header inclusion

This hack was needed with the make build system.
With meson, any private header from a library is visible as long as a
dependency to this library is expressed.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

bus/pci: fix leak for unbound devices

For devices not bound to any Linux kernel module, we leak a pci object
since it is never added to the PCI bus device list.

Fixes: c79a1c67465d ("bus/pci: optimize bus scan")
Cc: stable@dpdk.org
Reported-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: David Marchand <david.marchand@redhat.com>

bus/pci: update files description

Some files were starting with some outdated introductions.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David Marchand <david.marchand@redhat.com>

raw/cnxk_bphy: support BPHY self test

Add support for performing selftest.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support registering BPHY IRQ handlers

Custom IRQ handlers may be registered/removed on demand.
Since registration and removal are related they are in the
same patch.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support retrieving BPHY device memory

Allow user to retrieve baseband phy memory resources.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support reading number of BPHY IRQs

Add support for retrieving maximum number of interrupts.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support interrupt init and cleanup

Add support for interrupt initialization and cleanup. Internally
interrupt initialization performs low level setup that allows
custom interrupt handler registration later on.

Interrupt initialization and cleanup are related hence they
are in the same patch.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support BPHY dequeue operation

Add support for dequeueing responses to previously
enqueued messages.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support BPHY enqueue operation

Add preliminary support for enqueue operation.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support reading BPHY queue count

Add support for reading number of available queues from baseband
phy. Currently only single queue is supported.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support reading BPHY queue configuration

Add support for reading baseband phy queue configuration.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: add baseband PHY skeleton driver

Add baseband phy skeleton driver. Baseband phy is a hardware subsystem
accelerating 5G/LTE related tasks. Note this driver isn't involved into
any sort baseband protocol processing. Instead it just provides means
for configuring hardware.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: register BPHY IRQ

Add support for registering user supplied baseband PHY IRQ handler.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: clear BPHY IRQ handler

Add support for clearing previously register baseband PHY IRQ handler.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: set BPHY IRQ handler

Add support for setting custom baseband PHY IRQ handler.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: remove BPHY IRQ stack

Add support for removing existing IRQ stack.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: retrieve BPHY IRQ stack

Add support for retrieving IRQ stack.
If stack does not exist then it gets created.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: check BPHY IRQ availability

Add support for checking whether given IRQ is available.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add BPHY IRQ setup

Add support for initializing baseband PHY IRQs. While at it
also add support for reverting back to the default state.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add BPHY device init and fini

Add support for device init and fini. It merely saves
baseband phy state container in a globally accessible
resource chest.

Signed-off-by: Jakub Palider <jpalider@marvell.com>
Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support CGX self test

Add support for performing selftest operation.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support CGX dequeue operation

Add support for dequeueing responses to previously
enqueued messages.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support CGX enqueue operation

Add support for enqueueing messages.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support reading CGX queue count

Add support for reading number of available queues i.e number
of available logical macs (LMACs).

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: support reading CGX queue configuration

Add support for reading queue configuration. Single queue represents
a logical MAC available on RPM/CGX.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

raw/cnxk_bphy: add BPHY CGX/RPM skeleton driver

Add baseband PHY CGX/RPM skeleton driver which merely probes a matching
device. CGX/RPM are Ethernet MACs hardwired to baseband subsystem.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: start and stop BPHY LMAC

Add support for starting or stopping specific lmac.
Start enables Rx/Tx traffic while stop does the opposite.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: set BPHY link state

Add support for setting link up or down.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: set BPHY link mode

Add support for setting link mode.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: enable and disable BPHY PTP mode

Add support for enabling or disablig PTP mode.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: enable and disable BPHY internal loopback

Add support for enabling or disabling internal loopback.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: get BPHY link information

Add support for retrieving link information.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add BPHY communication with atf

Messages can be exchanged between userspace software and firmware
via set of two dedicated registers, namely scratch1 and scratch0.

scratch1 acts as a command register i.e message is sent to firmware,
while scratch0 holds response to previously sent message.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

common/cnxk: add BPHY CGX/RPM initialization and cleanup

Add support for low level initialization and cleanup of baseband
PHY CGX/RPM blocks.

Initialization and cleanup are related hence are in the same patch.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: Jakub Palider <jpalider@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

bus/auxiliary: introduce auxiliary bus

Auxiliary bus [1] provides a way to split function into child-devices
representing sub-domains of functionality. Each auxiliary device
represents a part of its parent functionality.

Auxiliary device is identified by unique device name, sysfs path:
  /sys/bus/auxiliary/devices/<name>

Devargs legacy syntax of auxiliary device:
  -a auxiliary:<name>[,args...]
Devargs generic syntax of auxiliary device:
  -a bus=auxiliary,name=<name>/class=<class>/driver=<driver>[,args...]

[1] kernel auxiliary bus document:
https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

devargs: add common key definition

Add common devargs key definition for "bus", "class" and "driver".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

eal: save error in string copy

The string copy api rte_strscpy() did not set rte_errno during failures,
instead it just returned negative error number.

Set rte_errrno if the destination buffer is too small.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

examples/l2fwd: remove mac-updating option

The "mac-updating" option can be removed since the associated mac_updating
variable is set to 1 by default.

Signed-off-by: Chenglian Sun <sunchenglian@loongson.cn>
Reviewed-by: David Marchand <david.marchand@redhat.com>

examples/l2fwd: fix [no-]mac-updating options

For l2fwd, --no-mac-updating and --mac-updating are treated as invalid
arguments. Rework long options parsing to let --no-mac-updating and
--mac-updating options work well.

Fixes: fa19eb20d212 ("examples/l2fwd: add forwarding port mapping option")
Cc: stable@dpdk.org
Signed-off-by: Chenglian Sun <sunchenglian@loongson.cn>
Reviewed-by: David Marchand <david.marchand@redhat.com>

examples/l3fwd: remove useless reloads in LPM main loop

Number of rx queue and number of rx port in lcore config are constants
during the period of l3 forward application running. But compiler has
no this information.

Copied values from lcore config to local variables and used the local
variables for iteration. Compiler can see that the local variables are
not changed, so qconf reloads at each iteration can be eliminated.

The change showed 1.8% performance uplift in single core, single port,
single queue test on N1SDP platform with MLX5 NIC.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

examples/l3fwd: remove useless calculations in NEON LPM

Both L2 and L3 headers will be used in forward processing. And these
two headers are in the same cache line. It has the same effect for
prefetching with L2 header address and prefetching with L3 header
address.

Changed to use L2 header address for prefetching. The change showed
no measurable performance improvement, but it definitely removed
unnecessary instructions for address calculation.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

app/test: fix IPv6 header initialization

Fix two issues found when writing PMD unit tests for HW ptype and
L4 checksum offload:

   - The version field in the IPv6 header was being set to zero,
     which prevented hardware from recognizing it as IPv6. The
     IP version field is now set to six.
   - The payload_len field was being initialized using host byte
     order, which (among other things) resulted in incorrect L4
     checksum computation. The payload_len field is now set using
     network (big-endian) byte order.

Fixes: 92073ef961ee ("bond: unit tests")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

bus/pci: support IOVA as VA in PowerVM LPARs

Add IOMMU detection logic for PowerVM LPARs.

PowerNV $ cat /proc/cpuinfo
...
timebase     : 512000000
platform     : PowerNV
model        : 8335-GTW

PowerVM LPAR $ cat /proc/cpuinfo
...
timebase     : 512000000
platform     : pSeries
model        : IBM,9009-22A
machine      : CHRP IBM,9009-22A
MMU          : Hash

PowerNV KVM Guest $ cat /proc/cpuinfo
...
timebase     : 512000000
platform     : pSeries
model        : IBM pSeries (emulated by qemu)
machine      : CHRP IBM pSeries (emulated by qemu)
MMU          : Radix

Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Thinh Tran <thinhtr@linux.vnet.ibm.com>

bus/pci: fix IOVA as VA support for PowerNV

Fix the IOMMU detection logic that looks for the "platform" field of
/proc/cpuinfo on POWER systems.

Fixes: 905215731833 ("bus/pci: support IOVA as VA on PowerNV systems")
Cc: stable@dpdk.org
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

eal/arm: remove unused type

Data types Elf32_auxv_t and Elf64_auxv_t are used by OS Linux
auxiliary vector read, and not used by arch specific cpu flag
API implementations. Hence remove them from Arm file.

Reported-by: James Grant <j.grant@qub.ac.uk>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

devtools: recommend new logtype helpers

Following commit eeded2044af5 ("log: register with standardized names"),
the new helpers should be preferred so that we can maintain a consistent
naming for logtypes.

Signed-off-by: David Marchand <david.marchand@redhat.com>

common/mlx5: fix Netlink port name padding in probing

On some kernels the string attributes within Netlink
reply messages might be not padded with zeroes (in cases
when string length is aligned with 4-byte boundary).
While device probing, the physical port name was wrongly recognized,
causing a probing failure.

Fixes: 30a86157f6d5 ("net/mlx5: support PF representor")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: convert meta register to big-endian

Metadata were stored in the CPU order (little-endian format on x86),
while all the packet header fields are stored in the network order.
That caused wrong results whenever we tried to use metadata value
in the modify_field action: bytes were swapped as a result.

Convert the metadata value into big-endian format before storing it
in the Mellanox NIC to achieve consistent behaviour.

Fixes: 641dbe4fb053 ("net/mlx5: support modify field flow action")
Cc: stable@dpdk.org
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix modify field action order for MAC

MAC addresses are split into 2 parts inside Mellanox NIC:
bits 0-15 are separate from bits 16-47. That makes a copy
from another packet field tricky because any other field
is aligned to 32 bits, not 16. This causes unexpected
results when using the MODIFY_FIELD action with MAC addresses.
Track crossing MAC addresses boundary and arrange a proper
order for the MODIFY_FIELD action involving MAC addresses.

Fixes: 641dbe4fb053 ("net/mlx5: support modify field flow action")
Cc: stable@dpdk.org
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix IPIP multi-tunnel validation

A flow rule must not include multiple tunnel layers.
An attempt to create such a rule, for example:
testpmd> flow create .../ vxlan / eth / ipv4 proto is 4 / end <actions>
results in an unclear error.

In the current implementation there is a check for
multiple IPIP tunnels, but not for combination of IPIP
and a different kind of tunnel, such as VXLAN. The fix
is to enhance the above check to use MLX5_FLOW_LAYER_TUNNEL
that consists of all the tunnel masks. The error message
will be "multiple tunnel not supported".

Fixes: 5e33bebdd8d3 ("net/mlx5: support IP-in-IP tunnel")
Cc: stable@dpdk.org
Signed-off-by: Lior Margalit <lmargalit@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: fix Rx queue timestamp format

The timestamp format was not configured correctly for the
receiving queues created via DevX calls. It caused non-UTC
timestamps in CQEs  for real time configurations.

Fixes: d61381ad46d0 ("net/mlx5: support timestamp format")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: fix switchdev mode recognition

The new kernels might add the switch_id attribute to the
Netlink replies and this caused the wrong recognition
of the E-Switch presence. The single uplink device was
erroneously recognized as master and it caused the
extending match for source vport index on all installed
flows, including the default ones, and adding extra hops
in the steering engine, that affected the maximal
throughput packet rate.

The extra check for the new device name format (it supposes
the new kernel) and the device is only one is added. If this
check succeeds the E-Switch presence is considered as wrongly
detected and overridden.

Fixes: 30a86157f6d5 ("net/mlx5: support PF representor")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: fix aging counter deallocation

When a counter is destroyed and used for aging action, the driver should
remove the counter object from the age-out list if it is there.

The counter memory of the list entry and of the counter shared
information is shared because, currently, shared counter cannot be used
for aging.

When the support for counter action in action handle API was added, the
counter shared information was reused and moved to be used also for
non-shared case. Wrongly, it is used for aging case too.

Remove the usage of shared information in case of aging.

Fixes: f3191849f2c2 ("net/mlx5: support flow count action handle")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Bing Zhao <bingz@nvidia.com>

net/mlx5: fix meter policy creation failure handling

When an error appears in the policy creation,
the IDs mapping between the user policy ID to
the driver policy ID is skipped.

Wrongly, the driver tried to clean the mapping in
this case what caused an error.

Skip the clearance in this case.

Fixes: afb4aa4f122 ("net/mlx5: support meter policy operations")
Cc: stable@dpdk.org
Signed-off-by: Li Zhang <lizh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

net/mlx5: allow copy from one tag to another

The modify field implementation in mlx5 driver has a check to
prevent a copy from a field to the same field. But the level
is not taken into account which prevents a copy from different
tags. Check the level and allow a copy from one tag to another.

Fixes: 641dbe4fb05 ("net/mlx5: support modify field flow action")
Cc: stable@dpdk.org
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix RSS pattern expansion

Flow rule pattern may be implicitly expanded by the PMD if the rule
has RSS flow action. The expansion adds network headers to the
original pattern. The new pattern lists all network levels that
participate in the rule RSS action.

The patch fixes expanded pattern for cases when original pattern
included meta items like MARK, TAG, META.

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")
Cc: stable@dpdk.org
Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: remove barrier for memory region cache

'dev_gen' is a variable to trigger all cores to flush their local caches
once the global MR cache has been rebuilt.

This is due to MR cache's R/W lock can maintain synchronization between
threads:

1. dev_gen and global cache updating ordering inside the lock protected
section does not matter. Because other threads cannot take the lock
until global cache has been updated. Thus, in out of order platform,
even if other agents firstly observe updated dev_gen but global does
not update, they also have to wait the lock. As a result, it is
unnecessary to add a wmb between global cache rebuilding and updating
the dev_gen to keep the memory store order.

2. Store-Release of unlock provides the implicit wmb at the level
visible by software. This makes 'rebuilding global cache' and 'updating
dev_gen' be observed before local_cache starts to be updated by other
agents. Thus, wmb after 'updating dev_gen' can be removed.

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx4: remove barrier for memory region cache

'dev_gen' is a variable to trigger all cores to flush their local caches
once the global MR cache has been rebuilt.

This is due to MR cache's R/W lock can maintain synchronization between
threads:

1. dev_gen and global cache updating ordering inside the lock protected
section does not matter. Because other threads cannot take the lock
until global cache has been updated. Thus, in out of order platform,
even if other agents firstly observe updated dev_gen but global does
not update, they still have to wait the lock. As a result, it is
unnecessary to add a wmb between global cache rebuilding and updating
the dev_gen to keep the memory store order.

2. Store-Release of unlock provides the implicit wmb at the level
visible by software. This makes 'rebuilding global cache' and 'updating
dev_gen' be observed before local_cache starts to be updated by other
agents. Thus, wmb after 'updating dev_gen' can be removed.

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

tests/eal: fix memory leak

The directory steam was not closed when the hugepage action was
HUGEPAGE_CHECK_EXISTS. This caused a memory leak in some parts of
the unit tests.

Fixes: 45f1b6e8680a ("app: add new tests on eal flags")
Cc: stable@dpdk.org
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Reviewed-by: David Marchand <david.marchand@redhat.com>

tests/cmdline: fix memory leaks

Fixes for a few memory leaks in the cmdline_autotest unit test.

All of the leaks were related to not freeing the commandline struct
after testing had completed.

Fixes: dbb860e03eb1 ("cmdline: tests")
Cc: stable@dpdk.org
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

rib: fix max depth IPv6 lookup

ASAN found a stack buffer overflow in lib/rib/rte_rib6.c:get_dir.
The fix for the stack buffer overflow was to make sure depth
was always < 128, since when depth = 128 it caused the index
into the ip address to be 16, which read off the end of the array.

While trying to solve the buffer overflow, I noticed that a few
changes could be made to remove the for loop entirely.

Fixes: f7e861e21c46 ("rib: support IPv6")
Cc: stable@dpdk.org
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

flow_classify: fix leaking rules on delete

Rules in a classify table were not freed if the table
had a delete function.

Fixes: be41ac2a330f ("flow_classify: introduce flow classify library")
Cc: stable@dpdk.org
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

kni: fix crash on userspace VA for segmented packets

When IOVA=VA, address translation for segmented packets is wrong, it
assumes the address in the mbuf->next is physical address, not VA
address.

Fixing the address translation to work both PA & VA mode.

Fixes: e73831dc6c26 ("kni: support userspace VA")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

kni: fix mbuf allocation for kernel side use

In kni_allocate_mbufs(), we alloc mbuf for alloc_q as this code.
allocq_free = (kni->alloc_q->read - kni->alloc_q->write - 1) \
& (MAX_MBUF_BURST_NUM - 1);
The value of allocq_free maybe zero, for example :
The ring size is 1024. After init, write = read = 0. Then we fill
kni->alloc_q to full. At this time, write = 1023, read = 0.

Then the kernel send 32 packets to userspace. At this time, write
= 1023, read = 32. And then the userspace receive this 32 packets.
Then fill the kni->alloc_q, (32 - 1023 - 1) & 31 = 0, fill nothing.
...
Then the kernel send 32 packets to userspace. At this time, write
= 1023, read = 992. And then the userspace receive this 32 packets.
Then fill the kni->alloc_q, (992 - 1023 - 1) & 31 = 0, fill nothing.

Then the kernel send 32 packets to userspace. The kni->alloc_q only
has 31 mbufs and will drop one packet.

Absolutely, this is a special scene. Normally, it will fill some
mbufs everytime, but may not enough for the kernel to use.

In this patch, we always keep the kni->alloc_q to full for the kernel
to use.

Fixes: 49da4e82cf94 ("kni: allocate no more mbuf than empty slots in queue")
Cc: stable@dpdk.org
Signed-off-by: Cheng Liu <liucheng11@huawei.com>
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/virtio: add MAC device config getter and setter

This patch uses the new device config ops to get and set
the MAC address if supported.

If a valid MAC address is passed as devarg of the
Virtio-user PMD, the driver will try to store it in the
device config space. Otherwise the one provided in
the device config space will be used, if available.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

net/virtio: add device config support to vDPA

This patch introduces two virtio-user callbacks to get
and set device's config, and implements it for vDPA
backends.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

net/virtio: keep device and frontend features separated

This patch is preliminary rework to add support for getting
and setting device's config space.

In order to get or set a device config such as its MAC address,
we need to know whether the device itself support the feature,
or if it is emulated by the frontend.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: allocate and free packets in bulk in Tx split

Same idea as commit a287ac28919d ("vhost: allocate and free packets
in bulk in Tx packed"), allocate and free packets in bulk.
Also remove the unused function virtio_dev_pktmbuf_alloc.

Signed-off-by: Balazs Nemeth <bnemeth@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix kernel set features for multi-queue device

Restore the original code, where VHOST_SET_FEATURES is applied to
all vhostfds of the device.

Fixes: cc0151b34dee ("net/virtio: add virtio-user features ops")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost/crypto: check request pointer before dereference

Use vc_req only after it was checked not to be NULL.

Fixes: 2d962bb736521 ("vhost/crypto: fix possible TOCTOU attack")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

devtools: fix file listing in maintainers check

When having multiple working trees, the main one has a .git directory
while attached trees have a .git file.
Thus the git check should work for both file and directory.

In the case there is no working tree (.git not readable), the command
"find" is used and should be able to list paths with wildcards.
Wildcards work only as shell expansion in the case of file paths,
so the quotes must be removed.

Fixes: 27c2ce563216 ("maintainers: start a Linux-style file")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

config/arm: check SVE CPU flag

If compiled with SVE feature (e.g. "-march=armv8.2-a+sve'), the binary
could not run on non-SVE platform else it will encounter illegal
instruction [1].

This patch fixes it by adding 'RTE_CPUFLAG_SVE' to compile_time_cpuflags,
so that rte_cpu_is_supported() will print meaningful log under above
situation.

[1] http://mails.dpdk.org/archives/dev/2021-May/209124.html

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Ruifeng Wang <ruifeng.wang@arm.com>

eal/windows: cleanup interrupt resources

Interrupt manager in Windows EAL allocates on IOCP and starts
a control thread that runs indefinitely. At DPDK cleanup
this thread was not stopped and IOCP handle was not closed.

Gracefully stop interrupt-handling in rte_eal_cleanup().
The thread already closes IOCP handle before exiting.

Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Jie Zhou <jizh@microsoft.com>
Tested-by: Jie Zhou <jizh@microsoft.com>

eal/windows: fix interrupt thread handle leakage

Each time a work was scheduled in the interrupt thread,
usually an alarm, a handle was opened but not closed.

Opening a handle is a system call, which harms alarm precision.
Instead of opening and closing a handle each time, open it
when interrupt thread starts and close it when the thread finishes.

Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Tested-by: Pallavi Kadam <pallavi.kadam@intel.com>

eal/windows: fix interrupt thread ID

Interrupt thread ID retained its value after interrupt thread finish.
Other interrupt routines could then operate on the wrong thread.
Clear interrupt thread ID before thread termination.

Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

raw/ioat: fix missing ring pointer reset

In the event of a device reconfigure, "hdls_avail" is not being reset. This
can lead to miscalculations in rte_ioat_completed_ops(), causing the
function to report an incorrect amount of completed operations. This patch
fixes the issue by resetting "hdls_avail" during the device configure.

Fixes: 74464005a2af ("raw/ioat: rework SW ring layout")
Cc: stable@dpdk.org
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

raw/ioat: fix memory leak in device configure

During device configure, memory is allocated for "hdl_ring_flags". In the
event of another call to the device configure function (reconfigure), a
memory leak would occur. This patch fixes the memory leak by free'ing the
memory before reallocating it.

Fixes: 245efe544d8e ("raw/ioat: report status of completed jobs")
Cc: stable@dpdk.org
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

buildtools: allow string constant padding

Size of string constant symbol may be larger than its length
measured up to NUL terminator. In this case pmdinfogen included padding
bytes after NUL terminator in generated source, yielding incorrect code.

Always trim string data to NUL terminator while reading ELF.
It was already done for COFF because there's no symbol size.

Bugzilla ID: 720
Fixes: f0f93a7adfee ("buildtools: use Python pmdinfogen")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

vfio: add stdbool include

This became visible by backporting the following for the 19.11 stable tree:
 c13ca4e8 "vfio: fix DMA mapping granularity for IOVA as VA"

The usage of type bool in the vfio code would require "#include
<stdbool.h>", but rte_vfio.h has no direct paths to stdbool.h.
It happens that in eal_vfio_mp_sync.c it comes after "#include
<rte_log.h>".

And rte_log.h since 20.05 includes stdbool since this change:
 241e67bfe "log: add API to check if a logtype can log in a given level"
and thereby mitigates the issue.

It should be safe to include stdbool.h from rte_vfio.h itself
to be present exactly when needed for the struct it defines using that
type.

Fixes: c13ca4e81cac ("vfio: fix DMA mapping granularity for IOVA as VA")
Cc: stable@dpdk.org
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>