net/bnx2x: fix packet drop

Patch "8bd31421c593 ("net/bnx2x: fix ramrod timeout")"
introduced a regression where sc->scan_fp flags is
set for unexpectedly long time. So the slow path completion
handler flow is run unnecessarily which walks over receive
descriptor ring of fast path and drops the data packets while looking
for slow path completion descriptor out of fast path ring.

This issue is seen under heavy traffic with link events happening
in background.

Fixes: 8bd31421c593 ("net/bnx2x: fix ramrod timeout")
Cc: stable@dpdk.org
Signed-off-by: Shahed Shaikh <shshaikh@marvell.com>
Acked-by: Rasesh Mody <rmody@marvell.com>

net/ena: fix assigning NUMA node to IO queue

Previous solution was using memzones in invalid way in hope to assign
IO queue to the appropriate NUMA zone.

The right way is to use socket_id from the rx/tx queue setup function
and then pass it to the IO queue.

Fixes: 3d3edc265fc8 ("net/ena: make coherent memory allocation NUMA-aware")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

app/testpmd: create mbuf based on max supported segments

Configuring buffer size based following parameters:
- max-pkt-len
- max supported segments per MTU

Buffer size are configured as given below:
- If platform supports infinite segments per packet then default
  buffer size is used.
- If platform supports nb_mtu_seg_max segments then buffer size
  is configured as (max-pkt-len / nb_mtu_seg_max) + headroom

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: add default value for max segment

rte_eth_dev_info structure exposes, nb_seg_max & nb_mtu_seg_max
to provide maximum number of supported segments for a given platform.

Defining UINT16_MAX as default value of above mentioned variables to
expose support of infinite/maximum segments.

Based on above values, application can decide best size for buffers
while creating mbuf pool.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/af_xdp: remove unused struct member

Fixes: f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/af_xdp: support multi-queue

This patch adds two parameters `start_queue` and `queue_count` to
specify the range of netdev queues used by AF_XDP pmd.

Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/af_xdp: enable zero copy by external mbuf

Implement zero copy of af_xdp pmd through mbuf's external memory
mechanism to achieve high performance.

This patch also provides a new parameter "pmd_zero_copy" for user, so they
can choose to enable zero copy of af_xdp pmd or not.

To be clear, "zero copy" here is different from the "zero copy mode" of
AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk
application.

Suggested-by: Vipin Varghese <vipin.varghese@intel.com>
Suggested-by: Tummala Sivaprasad <sivaprasad.tummala@intel.com>
Suggested-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/failsafe: fix reported device info

The failsafe driver device info had several issues in the
info it reported in dev_info_get:
  - it cleared dev_info->device set in rte_eth_dev_info_get
  - many fields (for example max_rx_queue) should be the minimum
    of all sub devices
  - it reported tx capa for the active transmit device, but
    the device may change.

There was enough messed up that ended up reworking the info_get
handler. There is no need to save current values or have a
template for defaults.

Fixes: 4e31ee26ed51 ("net/failsafe: report actual device capabilities")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
Acked-by: Gaetan Rivet <gaetan.rivet@6wind.com>

net/i40e: fix uninitialized value

Initializes mark_spec pointer to NULL.

Coverity issue: 341075
Fixes: 0bbcfc706a2b ("net/i40e: support MARK and RSS flow action")

Signed-off-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>

net/bnxt: enable RSS for thor-based controllers

Make changes needed to support rss for thor-based controllers.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: enable completion coalescing for thor

Enable completion coalescing for Thor-based adapters.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support thor controller

This commit adds support to the bnxt PMD for devices
based on the BCM57508 "thor" Ethernet controller.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>

net/bnxt: refactor ring allocation

Reduce code duplication and prepare for supporting hardware with
different ring allocation requirements by refactoring ring
allocation code.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: refactor doorbell handling

Reduce code duplication and prepare for newer controllers that
use different doorbell protocols by refactoring doorbell handling
code.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support extended HWRM request sizes

Enable support for extended request sizes.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: reset function earlier in initialization

Move function reset to beginnng of initialization sequence.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: use consistent values for VNIC RSS rule

Use consistent values for vnic->rss_rule. No functional change,
these all equate to uint16_t 0xffff.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix variable width in endian conversion

Use 32-bit conversion width when converting to 32-bit values.

Fixes: 6371b91fb66d ("net/bnxt: add ring alloc/free")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix ring type macro name

Use consistent macro names for ring type values. (There is no
functional change, the "alloc" and "free" values are identical.)

Fixes: 6371b91fb66d ("net/bnxt: add ring alloc/free")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix endianness in ring macros

Descriptor fields in CP ring are in little-endian form, convert
to CPU endian before performing arithmetic operations.

Also use more general comparison when checking for ring
index wrap.

Fixes: f2a768d4d186 ("net/bnxt: add completion ring")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/sfc: advertise offload capabilities by Tx datapaths

Tx datapath feature bits were useful on migration from the old offload API
to the new one. However, right now it just adds indirection which
complicates code reading and understanding. Also addition of a new
offloads requires addition of a new feature bits and makes patches longer
and harder to understand. So, remove feature bits which correspond to Tx
offloads and simply advertise device and per-queue offloads directly.
Generic code could still mask some offloads if running HW or FW does not
support it.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Igor Romanov <igor.romanov@oktetlabs.ru>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

net/sfc: advertise offload capabilities by Rx datapaths

Rx datapath feature bits were useful on migration from the old offload API
to the new one. However, right now it just adds indirection which
complicates code reading and understanding. Also addition of a new
offloads requires addition of a new feature bits and makes patches longer
and harder to understand. So, remove feature bits which correspond to Rx
offloads and simply advertise device and per-queue offloads directly.
Generic code could still mask some offloads if running HW or FW does not
support it.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Igor Romanov <igor.romanov@oktetlabs.ru>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

ethdev: add a check on mempool during RxQ setup

We currently have no check on the mempool pointer passed to
rte_eth_rx_queue_setup.
Let's avoid a plain crash when dereferencing it.

Suggested-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jens Freimann <jfreimann@redhat.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/ena: fix Rx checksum errors statistics

Rx checksum flags and input errors shouldn't be updated on Tx, as it
would work only for packets forwarding.

The ierrors statistic should be updated on Rx, right after checking
Rx checksum flags if the Rx checksum offload is enabled.

Fixes: 1173fca25af9 ("ena: add polling-mode driver")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>

net/ena: fix Tx statistics

Instead of counting number of used NIC Tx bufs just count number
of Tx packets.

Fixes: 45b6d86184fc ("net/ena: add per-queue software counters stats")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/memif: introduce memory interface PMD

Shared memory packet interface (memif) PMD allows for DPDK and any other
client using memif (DPDK, VPP, libmemif) to communicate using shared
memory. The created device transmits packets in a raw format. It can be
used with Ethernet mode, IP mode, or Punt/Inject. At this moment, only
Ethernet mode is supported in DPDK memif implementation. Memif is Linux
only.

Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/virtio: remove useless pointer checks

This patch removes useless checks on 'prev' pointer, as it
is always set before with a valid value.

Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix segment length in mergeable packed Rx

Head segment data_len field is wrongly summed with the length
of all the segments of the chain, whereas it should be the
length of the first segment only.

Fixes: a76290c8f1cf ("net/virtio: implement Rx path for packed queues")
Cc: stable@dpdk.org
Reported-by: Yaroslav Brustinov <ybrustin@cisco.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix mergeable Rx with segmented packet

After having dequeued a burst of descriptors, there may be a
need to dequeue a few more if the last packet was segmented
and not complete. When it happens, the extra segments were
not properly attached to the mbuf chain, and so were lost.

Also, head segment data_len field is wrongly summed with
the length of all the segments of the chain.

This patch fixes both the mbuf chaining and head segment's
data_len field

Fixes: bcac5aa207f8 ("net/virtio: improve batching in mergeable path")
Cc: stable@dpdk.org
Reported-by: Yaroslav Brustinov <ybrustin@cisco.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix in-order Rx with segmented packet

After having dequeued a burst of descriptors, there may be a
need to dequeue a few more if the last packet was segmented
and not complete. When it happens, the extra segments were
not properly attached to the mbuf chain, and so were lost.

Also, head segment data_len field is wrongly summed with
the length of all the segments of the chain.

This patch fixes both the mbuf chaining and head segment's
data_len field.

Fixes: e5f456a98d3c ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org
Reported-by: Yaroslav Brustinov <ybrustin@cisco.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/vhost: release port upon close

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private
resources for the port can be freed by rte_eth_dev_close().

Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

eal/x86: force inlining of all memcpy and mov helpers

Some helpers in the header file are forced inlined other are
only inlined, this patch forces inline for all.

It will avoid it to be embedded as functions when called multiple
times in the same object file. For example, when we added packed
ring support in vhost-user library, rte_memcpy_generic got no
more inlined.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

vhost: simplify descriptor buffer prefetching

Now that we have a single function to map the descriptors
buffers, let's prefetch them there as it is the earliest
place we can do it.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: do not inline unlikely fragmented buffers code

Handling of fragmented virtio-net header and indirect descriptors
tables was implemented to fix CVE-2018-1059. It should never
happen with healthy guests and so is already considered as
unlikely code path.

This patch moves these bits into non-inline dedicated functions
to reduce the I-cache pressure.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: do not inline packed and split functions

At runtime either packed Tx/Rx functions will always be called,
or split Tx/Rx functions will always be called.

This patch removes the forced inlining in order to reduce
the I-cache pressure.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: un-inline dirty pages logging functions

In order to reduce the I-cache pressure, this patch removes
the inlining of the dirty pages logging functions, that we
can consider as cold path.

Indeed, these functions are only called while doing live
migration, so not called most of the time.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

net/virtio: remove useless check on mempool

This .rx_queue_setup devop is called after ethdev already dereferenced
the mempool pointer.
No need to check and we can remove this rte_exit.

Fixes: 48cec290a3d2 ("net/virtio: move queue configure code to proper place")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/i40e: eliminate weak symbols in data path

Use of weak symbols can hide makefile errors especially when
custom makefiles are used.  Removing the use of weak symbols
to avoid a stub function being linked in production code.

Signed-off-by: David Harton <dharton@cisco.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

net/af_xdp: fix remove path

When users call rte_eth_dev_close() and rte_dev_remove(), the af_xdp
pmd return -1 (EPERM) due to eth_dev == NULL.

Since the af_xdp pmd driver advertises RTE_ETH_DEV_CLOSE_REMOVE, all
the resources are freed on rte_eth_dev_close().  rte_dev_remove() tries
to detach device and subsequently calls rte_pmd_af_xdp_remove() that
tries to free already freed resources and fails.
Fix it by return success.

Fixes: f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")
Cc: stable@dpdk.org
Reported-at: https://patchwork.ozlabs.org/patch/1106528/
Suggested-by: Ilya Maximets <i.maximets@samsung.com>
Signed-off-by: William Tu <u9012063@gmail.com>
Acked-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/mlx5: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ixgbe: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/i40e: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/enic: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ena: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/e1000: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/cxgbe: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bonding: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bnxt: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Lance Richardson <lance.richardson@broadcom.com>

net/axgbe: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ark: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/atlantic: remove unnecessary cast

The device private pointer (dev_private) is of type void *
therefore no cast is necessary in C.

Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bnxt: update HWRM API to version 1.10.0.74

Update HWRM API to version 1.10.0.74

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>

net/bnxt: update HWRM API to version 1.10.0.48

Update HWRM version to 1.10.0.48

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>

net/bnxt: update HWRM API to version 1.10.0.19

Update HWRM API to version 1.10.0.19

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Lance Richardson <lance.richardson@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>

net/bnxt: fix RSS RETA indirection table ops

We are trying to update the indirection table for all the VNICs.
We should update the table only for the default vnic0.

Fix the reta update function to only update table entries that are
selected by the update mask. Translate queue number to firmware
group ID when updating an entry.

Fix reta query op to only return table entries as identfied by the
provided mask. Translate firmware group IDs to queue numbers.

Removed extraneous code from bnxt_reta_query_op().

Fixes: d819382543f3 ("net/bnxt: add RSS redirection table operations")
Cc: stable@dpdk.org
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Rahul Gupta <rahul.gupta@broadcom.com>

net/bnxt: implement SSE vector mode

Introduce SSE vector mode support for the bnxt pmd.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: compute and store scattered Rx status

In preparation for a bnxt vector-mode driver, compute and store
scattered_rx status for the device when started.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/bnxt: move Tx bd checking to header file

To allow sharing of tx_bds_in_hw() and bnxt_tx_avail() between
vector-mode and non-vector transmit functions, move these functions
into bnxt_txr.h.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/bnxt: update release notes

Update release doc briefly describing updates to bnxt PMD done during
19.05 release, including transmit optimization changes in the commits
identified by the "Fixes:" tags below.

Fixes: 5ef3592c97b9 ("net/bnxt: support bulk free of Tx mbufs")
Fixes: 220de9869bc3 ("net/bnxt: optimize Tx batching")

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>

net/mlx5: fix event handler uninstall

When device is being closed and tries to unregister interrupt callback,
there is a chance the handler is still active (called in context of
eal_intr_thread_main thread). If so the rte_intr_callback_unregister
returns -EAGAIN and keeps the handler registered, causing crash when
underlaying resourse is gone away.

This race condition may happen if event handling in application takes
a long time. We should check the return code of unregistering routine
and try again to unregister the handler. The diagnostic messages are
shown once a second, while trying to unregister.

Fixes: 028b2a28c3cb ("net/mlx5: update event handler for multiport IB devices")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>

net/mlx5: fix order of items in NEON scatter

Previous patch added handling of metadata for multi-segment packet.
Function txq_scatter_v in file mlx5_rxtx_vec_neon.h was updated
incorrectly, items were inserted into WQE in wrong order.

This patch fixes the issue, inserting items into WQE correctly.

Fixes: 7f4019d370f6 ("net/mlx5: fix Tx metadata for multi-segment packet")
Cc: stable@dpdk.org
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

examples/rxtx_callbacks: support HW timestamp

Use rxtx callback to demonstrate a way to use rte_eth_read_clock to
convert the hardware timestamps to an amount of cycles.

This allows to get the amount of time the packet spent since its entry
in the device. While the regular latency only shows the latency from
when it entered the software stack.

Signed-off-by: Tom Barbette <barbette@kth.se>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/mlx5: support reading clock

Implements support for read_clock for the mlx5 driver. mlx5 supports
hardware timestamp offload, setting packets timestamp field to the
device clock. rte_eth_read_clock allows to read the device's current
clock value and therefore compare values on similar time base.

See rxtx_callbacks for an example.

Signed-off-by: Tom Barbette <barbette@kth.se>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

ethdev: add API to read device clock

Add rte_eth_read_clock to read the raw clock of a device.

The main use is to get the device clock conversion co-efficients to be
able to translate the raw clock of the timestamp field of the pkt mbuf
to a local synced time value.

This function was missing to allow users to convert the Rx timestamp
field to real time without the complexity of the rte_timesync* facility.
One can derivate the clock frequency by calling twice read_clock and
then keep a common time base.

Signed-off-by: Tom Barbette <barbette@kth.se>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/tap: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 02f96a0a82d1 ("net/tap: add TUN/TAP device PMD")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>

net/szedata2: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: abef3dd62e7a ("szedata2: add new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ring: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: e1e4017751f1 ("ring: add new driver")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/pcap: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 4c173302c307 ("pcap: add new driver")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/null: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: c743e50c475f ("null: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/mlx5: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 87011737b715 ("mlx5: add software counters")
Fixes: 9f9a48eb2978 ("net/mlx5: fix Tx stats error counter definition")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

net/mlx4: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 7fae69eeff13 ("mlx4: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

net/kni: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 75e2bc54c018 ("net/kni: add KNI PMD")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/cxgbe: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 856505d303f4 ("cxgbe: add port statistics")
Fixes: a0a344a8f728 ("net/cxgbe: add VF port statistics")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bnxt: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 577d3dced0dc ("net/bnxt: refactor the query stats")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/avp: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 5a5abe2de94b ("net/avp: add device statistics operations")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Allain Legacy <allain.legacy@windriver.com>

net/af_packet: fix RxQ errors stat

Transmit errors must not be reported in q_errors[] which is for
reception.

Fixes: 364e08f2bbc0 ("af_packet: add PMD for AF_PACKET-based virtual devices")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

acl: fix build with some arm64 compiler

Some compilers reporting the following error, though the existing
code doesn't have any uninitialized variable case.
Just to make compiler happy, initialize the int32x4_t variable
one shot using vdupq_n_s32.

lib/librte_acl/acl_run_neon.h: In function 'search_neon_4'
lib/librte_acl/acl_run_neon.h:230:12: error:
  'input' may be used uninitialized in this function
  int32x4_t input;

Fixes: 34fa6c27c156 ("acl: add NEON optimization for ARMv8")
Cc: stable@dpdk.org
Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Aaron Conole <aconole@redhat.com>

hash: simplify signature compare with NEON

Replaced multiple neon instructions with single equivalent instruction.
This made simpler code and a bit higher performance.
Hash bulk lookup had 0.1% ~ 3% performance gain in tests on ARM A72
platforms.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

build: generate Windows exports file

Rather than having a separate version.map file for linux/BSD and an
exports definition file for windows for each library, generate the
latter from the former automatically at build time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

test: skip meson dependency checks if not required

Don't need to check dependencies if test apps will not be built anyway.

Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>

crypto/aesni_mb: fix build with gcc 4.8.5

On CentOS/RHEL 7, we get build errors reported due to a variable being
possibly uninitialized. Setting a default init value fixes this issue.

Fixes: c68d7aa354f6 ("crypto/aesni_mb: use architecture independent macros")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

bpf: fix pseudo calls for program loaded from ELF

clang 6.0 and onwards, for the external function call generates
BPF_PSEUDO_CALL instruction:
call pseudo +-off -> call another bpf function.
More details about that change: https://lwn.net/Articles/741773/
DPDK BPF implementation right now doesn't support multiple BPF
functions per module.
To overcome that problem, and preserve existing functionality
(ability to call allowed by user external functions),
bpf_elf_load() clears EBPF_PSEUDO_CALL value.
For details how to reproduce the issue:
https://bugs.dpdk.org/show_bug.cgi?id=259

Fixes: 5dba93ae5f2d ("bpf: add ability to load eBPF program from ELF object file")
Cc: stable@dpdk.org
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

bpf: remove use of weak functions

Weak functions don't work well with static libraries and require the use of
"whole-archive" flag to ensure that the correct function is used when
linking.  Since the weak function is only used as a placeholder within this
library alone, we can replace it with a non-weak version protected using
preprocessor ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

acl: remove use of weak functions

Weak functions don't work well with static libraries and require the use of
"whole-archive" flag to ensure that the correct function is used when
linking. Since the weak functions are only used as placeholders within
this library alone, we can replace them with non-weak functions using
preprocessor ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

examples/multi_process: fix FreeBSD build

Caught on FreeBSD 12:

/usr/include/netinet/ip.h:71:17: error: field 'ip_src' has incomplete type
  struct in_addr ip_src,ip_dst; /* source and dest address */
                 ^~~~~~

On FreeBSD, netinet/ip.h is not auto sufficient like on Linux.
But actually, this header is not used in the example, just remove it.

Fixes: 764bf26873b9 ("add FreeBSD support")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

rcu: fix format specifier in dump

Use %u to fix argument type mismatch in fprintf.

Coverity issue: 340074
Fixes: 64994b56cfd7 ("rcu: add RCU library supporting QSBR mechanism")
Cc: stable@dpdk.org
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

test/rcu: make global variable per core

Each hash entry has a pointer to one uint32 memory location.
However, all the readers increment the same location causing
race conditions. Allocate memory for each thread so that each
thread will increment its own memory location.

Fixes: b87089b0bb19 ("test/rcu: add API and functional tests")
Cc: stable@dpdk.org
Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
Tested-by: David Marchand <david.marchand@redhat.com>

doc: fix Linux guide for arm64 cross-compilation

libnuma.so is needed to augment the cross toolchain with NUMA support.
This fixed meson cross compiling issue.

Command used:
meson arm64-build --cross-file config/arm/arm64_armv8_linux_gcc
ninja -C arm64-build

Compiling error:
.../aarch64-linux-gnu/bin/ld: lib/librte_eal.so.10.1: version node
not found for symbol numa_run_on_node_mask@@libnuma_1.2
.../aarch64-linux-gnu/bin/ld: failed to set dynamic section sizes:
Bad value
collect2: error: ld returned 1 exit status
[58/1370] Compiling C object 'lib/76b5a35@@rte_cmdline@sta/
librte_cmdline_cmdline_parse_string.c.o'.
ninja: build stopped: subcommand failed.

Fixes: 01add9da25cd ("doc: add cross compiling guide")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

doc: update quick start guide for meson

The build-sdk-meson.txt file is a little out of date, so update it with
information on the latest build requirements, and remove any content
no longer needed.

Since the cross-compilation file quoted in the document is now considerably
longer and more complex than previous, replace the contents of the file
with a summary of it instead. This is shorter and more maintainable, and
the original file is available as part of the repo anyway if the user wants
to view it.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

ipc: handle unsupported IPC in async request

Currently, IPC API will silently ignore unsupported IPC.
Fix the API call to explicitly handle unsupported IPC cases.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ipc: handle unsupported IPC in sync request

Currently, IPC API will silently ignore unsupported IPC.
Fix the API call and its callers to explicitly handle
unsupported IPC cases.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ipc: handle unsupported IPC in sendmsg

Currently, IPC API will silently ignore unsupported IPC.
Fix the API call and its callers to explicitly handle
unsupported IPC cases.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ipc: do not unregister action if IPC unsupported

Currently, unregister will be attempted even if IPC wasn't
supported in the first place. It is harmless, but for
consistency reasons, update the unregister API call to
exit early when IPC is not supported.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ipc: handle unsupported IPC in action register

Currently, IPC API will silently ignore unsupported IPC.
Fix the API call and its callers to explicitly handle
unsupported IPC cases.

For primary processes, it is OK to not have IPC because
there may not be any secondary processes in the first place,
and there are valid use cases that disable IPC support, so
all primary process usages are fixed up to ignore IPC
failures.

For secondary processes, IPC will be crucial, so leave all
of the error handling as is.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ipc: handle unsupported IPC in init

Currently, IPC API will silently ignore unsupported IPC.
Fix the API call and its callers to explicitly handle
unsupported IPC cases.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

ring: remove unnecessary forward declaration

As memzone.h is introduced by
commit 38c9817ee1d8 ("mempool: adjust name size in related data types"),
forward declaration for rte_memzone is no longer needed.

Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

mem: ease init in a docker container

move_pages() is only used to get the numa node id, but this function
is not allowed by default in docker (it needs CAP_SYS_NICE and an update of
the seccomp profile).
get_mempolicy() also requires CAP_SYS_NICE but doesn't need any change in
the default seccomp profile.

Note that the returned value of move_pages() was not checked, thus some
errors could be hidden (if the requested id was 0).

Fixes: 582bed1e1d1d ("mem: support mapping hugepages at runtime")
Cc: stable@dpdk.org
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Didier Pallard <didier.pallard@6wind.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

examples/ip_fragmentation: fix Tx queues init

The application tries to configure queue ids larger
than the maximum allowed by MAX_TX_QUEUE_PER_PORT. This
causes the startup error:

"
  ...
  Initializing port 0 on lcore 0... Address:7C:FE:90:12:23:0D
  txq=0,0 txq=1,1 txq=2,2 txq=3,3 txq=4,4 txq=5,5 txq=6,6 txq=7,7 txq=8,8
  txq=9,9 txq=10,10 txq=11,11 txq=12,12 txq=13,13 txq=14,14 txq=15,15
  txq=16,16 Invalid TX queue_id=16

  EAL: Error - exiting with code: 1
    Cause: rte_eth_tx_queue_setup: err=-22, port=0
"

The error reproduces when lcores aren't set, and when the
machine has more than 16 cores.

Fixes: af75078fece3 ("first public release")
Cc: stable@dpdk.org
Signed-off-by: Ali Alnubani <alialnu@mellanox.com>

examples/multi_process: build with meson

This enables building the example multiprocess applications in
the subdirectory multi_process.

Signed-off-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Luca Boccassi <bluca@debian.org>

examples: use child directory as name

This would allow correctly naming an application residing
in a subdirectory. For example, if the example is set to 'path/to/app',
then the name would be 'app'.
This doesn't affect the naming of an example that isn't in a subdirectory.

Signed-off-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Luca Boccassi <bluca@debian.org>