mbuf: support dynamic fields and flags

Many features require to store data inside the mbuf. As the room in mbuf
structure is limited, it is not possible to have a field for each
feature. Also, changing fields in the mbuf structure can break the API
or ABI.

This commit addresses these issues, by enabling the dynamic registration
of fields or flags:

- a dynamic field is a named area in the rte_mbuf structure, with a
  given size (>= 1 byte) and alignment constraint.
- a dynamic flag is a named bit in the rte_mbuf structure.

The typical use case is a PMD that registers space for an offload
feature, when the application requests to enable this feature.  As
the space in mbuf is limited, the space should only be reserved if it
is going to be used (i.e when the application explicitly asks for it).

The registration can be done at any moment, but it is not possible
to unregister fields or flags.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

net/ice: add GTP tunnel type

Add GTP tunnel type in ice pmd tunneling parameters parse
to enable Tx checksum offload.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

mbuf: add GTP tunnel type

Add GTP tunnel type flag in mbuf for future use in GTP
Tx checksum offload.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

app/testpmd: increase RSS max queue size

Max queue indices ACTION_RSS_QUEUE_NUM , which limits testpmd actions
rss queue size, has worked for a long time.

As more powered PMD is merged, more RSS queues are supported(e.g. Intel
ice driver FDIR supports up to 128 RSS queue numbers).

The original max queue indices number doesn't satisfy PMD's requirement
and it needs to become bigger.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/enetc: enable dpaax library

dpaaX is used to maintain a local copy of PA->VA translations.

Using the rte_mem_virt2iova or rte_mem_virt2phy is expensive. This
library is an attempt to reduce the overall cost associated with
this translation.

This patch enables this dpaaX library by populating a dpaaX's
table for PA to VA translation. This change will also help the
caam JR driver in PA to VA translation when used with enetc driver
over enetc supported SoCs.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: print MAC address logs at notice level

To display random MAC address as notice,
a log level NOTICE is added.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: set random MAC in case no MAC for SI

for SGMII interfaces, there can be 0 value
written on MAC registers.
This patch set the random MAC address for those
interfaces.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: fix BD ring alignment

enetc BD rings should be aligned to 128
instead of RTE_CACHE_LINE_SIZE.

Fixes: 469c6111a799 ("net/enetc: enable Rx and Tx")
Cc: stable@dpdk.org
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/mlx5: fix DevX event registration timing

The DevX counter management triggers an asynchronous event to get back
the new counters values from the HW.

The counter management doesn't trigger 2 parallel events for the same
pool, hence, the pool cannot be updated again in the event waiting time.

When the port is stopped, the DevX event mechanism wrongly was
destroyed what remained all the waiting pools in waiting state forever.

As a result, the counters of the stuck pools were never updated again.

Separate the DevX interrupt installation from the dev installation and
remove the DevX interrupt unregistration\registration from the
stop\start operations.

Now, the DevX interrupt should be installed in probe and uninstalled in
close.

Cc: stable@dpdk.org
Fixes: f15db67df09c ("net/mlx5: accelerate DV flow counter query")

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/ice: fix queue index for switch filter

When calculate queue index for queue action in
switch filter, vsi base queue should be included.

Fixes: 57c4f2693588 ("net/ice: enable switch filter")
Cc: stable@dpdk.org
Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/af_xdp: support unaligned umem chunks

This patch enables the unaligned chunks feature for AF_XDP which allows
chunks to be placed at arbitrary places in the umem, as opposed to them
being required to be aligned to 2k. This allows for DPDK application
mempools to be mapped directly into the umem and in turn enable zero copy
transfer between umem and the PMD.

This patch replaces the zero copy via external mbuf mechanism introduced
in commit e9ff8bb71943 ("net/af_xdp: enable zero copy by external mbuf").
The pmd_zero copy vdev argument is also removed as now the PMD will
auto-detect presence of the unaligned chunks feature and enable it if so
and otherwise fall back to copy mode if not detected.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

ethdev: add HIGIG2 key field to flow API

Add new rte_flow_item_higig2_hdr in order to match higig2 header.
It is a layer 2.5 protocol and used in Broadcom switches.
Header format is based on the following document.
http://read.pudn.com/downloads558/doc/comm/2301468/HiGig_protocol.pdf

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

drivers/net: enable promiscuous and multicast by default

The promiscuous and multicast fields are now initialised as enabled for
some virtual PMDs. This allows the devices to be used when running
applications that attempt to enable promiscuous or multicast mode.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

ethdev: check device promiscuous state

The promiscuous enable and disable functions now check the
promiscuous state of the device before checking if the dev_ops
function exists for the device.

This change is necessary to allow sample applications run on
virtual PMDs, as previously -ENOTSUP returned when the promiscuous
enable function was called. This caused the sample application to
fail unnecessarily.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/af_packet: improve Tx statistics accuracy

When sendto call fails and ENOBUFS/EAGAIN error is being set
some of the packets are actually successfully transmitted.
There is no available count of those packets, so in order to
make the statistics more accurate, all the previously enqueued
packets will be considered successful, even though this is not
entirely correct.

Statistics numbers before this update:

Pktgen:
   Total Rx Pkts:               1360084
         Tx Pkts:               2000000
testpmd:
   RX-packets: 1408346  RX-missed: 0       RX-bytes:  84503418
   TX-packets: 526486   TX-errors: 881851  TX-bytes:  31589724

Statistics numbers after this update:

Pktgen:
   Total Rx Pkts:               1329872
         Tx Pkts:               2000000
testpmd:
   RX-packets: 1389156  RX-missed: 0       RX-bytes:  83349360
   TX-packets: 1389156  TX-errors: 0       TX-bytes:  83349360

Fixes: 74b7fc0a0ff1 ("net/af_packet: fix packet bytes counting")
Cc: stable@dpdk.org
Signed-off-by: Flavia Musatescu <flavia.musatescu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/mlx5: fix crash on hash Rx queue handling for drop

When to create hrxq for the drop, it could fail on creating qp and goto
the error handle which will release created ind_table by calling drop
release function, which takes rte_ethdev as the only parameter and uses
the priv->drop_queue.hrxq as input to release.

Unfortunately, at this point, the hrxq is not allocated and
priv->drop_queue.hrxq is still NULL, which leads to a segfault.

This patch fixes the above by allocating the hrxq at first place and
when the error happens, hrxq is released as the last one.

This patch also release other allocated resources by the correct order,
which is missing previously.

Fixes: 78be885295b8 ("net/mlx5: handle drop queues as regular queues")
Cc: stable@dpdk.org
Reported-by: Zengmo Gao <gaozengmo@jd.com>
Signed-off-by: Xiaoyu Min <jackmin@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/ice: enable advanced RSS

This patch supports:

- symmetric hash by rte_flow RSS action.
- input set change by rte_flow RSS action.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: add RSS configuration for GTPU/PPPoE

This patch adds rss support for gtpu with input set teid and
pppoe/pppod with input set source mac and session id.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: support flow director on AVX Rx

This commit adds FDIR ID support to the AVX2 based receive
path routine. Support for both 16B and 32B descriptors is
implemented.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/i40e: support flow director on SSE Rx

This commit adds an implementation to the SSE vector implementation of
RX routine and moves some common defines from a c file to the header
file.

I40e can have 16 and 32 byte descriptors, and the Flow
Director ID data and indication-bit are in different locations
for each size descriptor. The support is implemented in two
separate functions as they require vastly different operations.

The 16B descriptor re-purposes the "filter-status" u32 field
to indicate FDIR ID when the FLM bit is set. No extra loads
are required, however we do have to store to mbuf->fdir.hi,
which is not stored to in the RX path before this patch.

The 32B descriptor requires loading the 2nd 16 bytes of each
descriptor, to get the FLEXBH_STAT and FD Filter ID from
qword3. The resulting data must also be stored to mbuf->fdir.hi,
same as the 16B code path.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/i40e: cache flow director enable value in Rx queue

This commit adds a fdir_enable flag in a uint8_t sized hole
the rx queue structure The flag enables the rx code path to
easily identify if fdir is active. This can be used to skip
fdir id processing when it is not required.

The flag is zero by default (as rxq is zmalloc-ed at startup),
and the flag is set to 1 on configuration of a flow director rule.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/axgbe: fix double unlock

One issue caught by Coverity 340835
*unlock: axgbe_phy_set_mode unlocks pdata->phy_mutex
*double_unlock: axgbe_phy_sfp_detect unlocks pdata->phy_mutex
while it is unlocked.

In axgbe_phy_sfp_detect()/axgbe_phy_set_redrv_mode(),
axgbe_phy_get_comm_ownership() and axgbe_phy_put_comm_ownership()
are invoked subsequently.

Currently in axgbe_phy_get_comm_ownership(), during one of the case
'phy_data->comm_owned' is not protected and before returning 0, lock is
not called and unlock is called in axgbe_phy_put_comm_ownership()
directly which is incorrect.

Ideally, the variable 'phy_data->comm_owned' needs to be protected.
During success scenario, lock is called in
axgbe_phy_get_comm_ownership() followed by unlock in
axgbe_phy_put_comm_ownership().  In failure case, unlock is invoked in
axgbe_phy_get_comm_ownership() itself appropriately.

The fix is to protect 'phy_data->comm_owned' in the identified case
ensuring locks/unlocks properly exist.

Coverity issue: 340835
Fixes: a5c7273771e8 ("net/axgbe: add phy programming APIs")
Cc: stable@dpdk.org
Signed-off-by: Pallantla Poornima <pallantlax.poornima@intel.com>
Acked-by: Ravi Kumar <ravi1.kumar@amd.com>

net/iavf: enable AVX2 for iavf

This patch enables AVX data path for iavf PMD.

Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/mlx5: add flow match on GENEVE item

This commit adds support for matching flows on Geneve headers.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: query HCA for enabled FLEX parser protocols

This commit add querying the HCA which FLEX protocols are already
enabled.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: retry on link update failure

mlx5_link_update immediately returns when called with no-wait parameter
and its call for retrieving the link status returns with EAGAIN error.
This is too harsh on busy systems where a first call fails with EAGAIN
from time to time.
This patch adds a (very limited) retry on such cases in order to allow
retrieving the link status.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: fix build with strict alignment enabled

This patch converts some of the casts to unaligned integer types.
The memcpy call is replaced with explicit copying because it
may require type qualifiers in some environments.

Fixes: 18a1c20044c0 ("net/mlx5: implement Tx burst template")
Cc: stable@dpdk.org
Reported-by: Jeremy Plsek <jplsek@iol.unh.edu>
Signed-off-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: fix Tx packets statistics

The transmitter packets counter was not updated
correctly in the loop inside the tx_burst routines.

Fixes: f32a3f5216a3 ("net/mlx5: fix completion queue overflow for large burst")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx4: remove dependency on libmnl in meson

There is no dependency on this library for mlx4.

Fixes: 1dd7c7e38c19 ("net/mlx4: support meson build")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>

app/testpmd: fix help for loop topology option

The testpmd --help option did not show all possible choices for port
topology previously. The loop topology option is now added.

Fixes: 3e2006d6186c ("app/testpmd: add loopback topology")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net: add missing endianness annotations

OVS currently maintains a copy of those headers with the right endianness
annotations so that sparse checks can pass.

We introduced rte_beXX_t for better readibility in v17.08.
Let's make use of them, OVS then only needs to override those rte_beXX_t
types by exposing a tweaked rte_byteorder.h header.

Other existing dpdk users won't be affected since rte_beXX_t types are
mapped to uintXX_t types.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

doc: fix a common typo in NIC guides

'CRC striping' should be spelled 'CRC stripping'.

Fixes: 3eee1f067e7c ("fm10k: add guide")
Fixes: 7a4d9f6676d7 ("doc: add liquidio")
Fixes: f820b5896631 ("doc: add octeontx ethdev driver documentation")
Fixes: 920717e4d8ba ("net/octeontx2: add device start operation")
Fixes: f994cecafdcf ("doc: add ThunderX nicvf")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

maintainers: add next-net-brcm sub-tree

Adding vendor specific sub-tree for Broadcom drivers.
The next-net-brcm sub-tree will be sub-tree of the next-net.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

net/octeontx2: support multicast filter

Patch adds mc filter support for otx2 eth devices.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/octeontx2: upgrade mbox definition to version 2

Sync mail box data structures to version 0x0002.
This patch checks for mismatch in mail box revision and
avoids initializing octeontx2 pci device if
there is a mismatch.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/ice: cleanup RSS/FDIR profile on device init

When app restarts, RSS/FDIR profile ID will increase which may
run out of all the profile resources.

The patch cleanup RSS/FDIR profile resources when device init
to fix this issue.

Fixes: d7d150b93070 ("net/ice: enable RSS when device init")
Cc: stable@dpdk.org
Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director GTPU tunnel

Enable FDIR GTPU matching for RTE_FLOW

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director VXLAN tunnel

Enable FDIR vxlan tunnel matching for RTE_FLOW

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: reject duplicated flow for flow director

Enable duplication lookup for existing flow director rule entry.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director counter

This patch add FDIR statistical counter support
and it includes RTE_FLOW count actions support
and query support.

RTE_FLOW count actions support id and shared.
RTE_FLOW query record packet hits by default.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add flow director counter resource init/release

The patch integrates the counter resource init/release into fdir's
init/release scenario

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: enable flow director queue group

FDIR can send packet to a group of queues and distruibte it by RSS.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add flow director create and destroy

Add ice_create_fdir_filter to create a rule. If a flow is matched by
flow director filter, filter rule will be set to HW. For now common
pattern and queue/passthru/drop/mark actions are supported.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: configure HW flow director rule

This patch adds a HW FDIR rule to the FDIR HW table
without adding a FDIR filter.

Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: enable flow director engine

Enable flow director engine, including initialization and teardown.
 - Control VSI create and release.
 - Queue pair allocated, set up and release.
 - Programming packet create and release.
 - FDIR profile create and release.

Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

app/testpmd: support extended RSS offload types

This patch adds cmdline support for extended rss types configuration.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

ethdev: extend RSS offload types

This patch reserves several bits as input set selection from the
high end of the 64 bits. It is combined with exisiting ETH_RSS_*
to represent RSS types. This patch also checks the simultaneous
use of SRC_ONLY and DST_ONLY of the same level.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Ori Kam <orika@mellanox.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

ethdev: decouple flow types and RSS offload types

This patch decouples RTE_ETH_FLOW_* and ETH_RSS_*. The former defines
flow types and the latter defines RSS offload types.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/ixgbe: fix address of first segment

This patch fixes (dereference after null check) coverity issue.
The address of first segmented packets was not set correctly during
reassembling packets which led to this issue.

Coverity issue: 13245
Fixes: 8a44c15aa57d ("net/ixgbe: extract non-x86 specific code from vector driver")
Cc: stable@dpdk.org
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/i40e: fix address of first segment

This patch fixes (dereference after null check) coverity issue.
The address of first segmented packets was not set correctly during
reassembling packets which led to this issue.

Coverity issue: 343422, 343403
Fixes: ca74903b75cf ("net/i40e: extract non-x86 specific code from vector driver")
Cc: stable@dpdk.org
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: invoke callback when link status change

Needs to call _rte_eth_dev_callback_process to run registered
callbacks when link status change.

Fixes: cf911d90e366 ("net/ice: support link update")
Cc: stable@dpdk.org
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: rework switch filter

The patch reworks packet process engine's binary classifier
(switch) for the new framework. It also adds support for new
packet type like PPPoE for switch filter.

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add pattern manifest

The patch adds a manifest for all patterns which can be selected
by low level filter engine to compose their supported list.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: rework for generic flow enabling

The patch reworks the generic flow API (rte_flow) implementation.
It introduces an abstract layer which provides a unified interface
for low-level filter engine (switch, fdir, hash) to register supported
patterns and actions and implement flow validate/create/destroy/flush/
query activities.

The patch also removes the existing switch filter implementation to
avoid compile error. Switch filter implementation for the new framework
will be added in the following patch.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add devargs to control pipeline mode

Added a devarg to control the mode in generic flow API.
We use none-pipeline mode by default.

Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: clean up redundant assignment and indentations

The patch removes redundant code and cleans up some wrong indentations.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

vhost: improve performance by supporting large buffer

The rte_vhost_dequeue_burst supports two ways of dequeuing data.
If the data fits into a buffer, then all data is copied and a
single linear buffer is returned. Otherwise it allocates
additional mbufs and chains them together to return a multiple
segments mbuf.

While that covers most use cases, it forces applications that
need to work with larger data sizes to support multiple segments
mbufs. The non-linear characteristic brings complexity and
performance implications to the application.

To resolve the issue, add support to attach external buffer
to a pktmbuf and let the host provide during registration if
attaching an external buffer to pktmbuf is supported and if
only linear buffer are supported.

Signed-off-by: Flavio Leitner <fbl@sysclose.org>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix descriptor addressed in Tx

Previous fix removes usage of rte_pktmbuf_prepend() to get pointer
to virtio net header which changes mbuf data_off and data_len.
Size of virtio net header is added to segment length when Tx descriptor
is composed, but segment address (calculated using data_off) is not
adjusted to take size of virtio net header into account.

Fixes: 1ae55ad38e5e ("net/virtio: fix mbuf data and packet length mismatch")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add packed ring support to vring related APIs

This patch add packed ring support in two APIs
so user can get the packed ring`.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add APIs to get inflight ring

This patch introduces two APIs. one is for getting inflgiht
ring and the other is for getting base.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add APIs to operate inflight ring

This patch introduces three APIs to operate the inflight
ring. Three APIs are set, set last and clear. It includes
split and packed ring.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: checkout resubmit inflight information

This patch shows how to checkout the inflight ring and construct
the resubmit information also include destroying resubmit info.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: support inflight info sharing

This patch introduces two new messages VHOST_USER_GET_INFLIGHT_FD
and VHOST_USER_SET_INFLIGHT_FD to support transferring a shared
buffer between qemu and backend.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add inflight structures

This patch adds the inflight queue region structure include
the split and packed.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add packed ring into vring struct

This patch add the packed ring in the rte_vhost_vring.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add inflight description

This patch add the inflight message description and
the inflight share fd protocol feature flag.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: prevent zero copy mode if IOMMU is on

The simultaneous use of dequeue_zero_copy and IOMMU is problematic.
Not only because IOVA_VA mode is not supported but also because the
potential invalidation of guest pages while the buffers are in use,
is not handled.

Prevent these two features to be enabled simultaneously.

Fixes: 69c90e98f483 ("vhost: enable IOMMU support")
Cc: stable@dpdk.org
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: convert buffer addresses to GPA for logging

Add IOVA versions of dirty page logging functions.

Note that the API facing rte_vhost_log_write is not modified.
So, make explicit that it expects the address in GPA space.

Fixes: 69c90e98f483 ("vhost: enable IOMMU support")
Cc: stable@dpdk.org
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: translate incoming log address to GPA

When IOMMU is enabled the incoming log address is in IOVA space. In that
case, look in IOTLB table and translate the resulting HVA to GPA.

If IOMMU is not enabled, the incoming log address is already a GPA so no
transformation is needed.

Fixes: 69c90e98f483 ("vhost: enable IOMMU support")
Cc: stable@dpdk.org
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix Rx AltiVec path by getting all packets

The loop to read packets does not take all packets as the number of
available packets (nb_used) is decremented in the loop.

Fixes: 52b5a707e6ca ("net/virtio: add Altivec Rx")

Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix Rx stats on AltiVec path

With vectorized functions, only the rx stats for number of packets is
incremented.
Update also the other statistics.

Fixes: 52b5a707e6ca ("net/virtio: add Altivec Rx")

Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: improve perf via one-way barriers on used flag

In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the frontend
and backend are assumed to be implemented in software, that is they can
run on identical CPUs in an SMP configuration.
Thus a weak form of memory barriers like rte_smp_r/wmb, other than
rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers == 1)
and yields better performance.
For the above case, this patch helps yielding even better performance
by replacing the two-way barriers with C11 one-way barriers for used
flags in packed ring.

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: improve perf via one-way barrier on avail flag

In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the frontend
and backend are assumed to be implemented in software, that is they can
run on identical CPUs in an SMP configuration.
Thus a weak form of memory barriers like rte_smp_r/wmb, other than
rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers == 1)
and yields better performance.
For the above case, this patch helps yielding even better performance
by replacing the two-way barriers with C11 one-way barriers for avail
flags in packed ring.

Meanwhile, a read barrier is required to ensure ordering between
descriptor's flags and content reads [1]. With C11, load-acquire can
enforce the ordering instead of rmb barrier.

[1] https://patchwork.dpdk.org/patch/49109/

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ice: support RSS in AVX path

Support 32 bits RSS in FlexMD fields in AVX path.

Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

net/ice: support RSS in SSE path

Support 32 bits RSS in FlexMD fields in SSE path.

Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

net/ice: fix LUT type for RSS

This patch changes RSS lut_type to corresponding macro.

Fixes: ff963bfa7cb1 ("net/ice: support RSS")
Cc: stable@dpdk.org
Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix integer overflow

When configuring i40e rx queue, the temporary variable to store max
packet length is not big enough which leads to integer overflow issue.
This patch fixes the issue by removing the variable and using the
expression directly since the variable is only used once.

Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: fix protocol extraction devargs parsing

If the default dev args 'proto_xtr' is not in the first position, it
will overwrite the parsed queue map value, so use an new variable to
save the default.

And enhance the error message printing to show the right information.

Fixes: 7e124ff12c85 ("net/ice: support protocol extraction per Rx queue")

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: fix memory leak on dev closed

The original protocol extraction memory free is removed by introducing
new design for releasing port upon close.

Fixes: bd513ece3c40 ("net/ice: release port upon close")

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

app/testpmd: fix CRC strip command

This patch fixed the bug that a failure appeared when config
rx_offload crc_strip using command "port config all crc-strip
on|off". The reason is that this command was removed in Commit
e5db17a1e54e. The current command is "port config <port_id>
rx_offload keep_crc on|off" instead.

In this patch, some codes left over about 'crc_strip' are
removed to make the current command clearer.

Fixes: e5db17a1e54e ("app/testpmd: remove duplicated Rx offload commands")
Cc: stable@dpdk.org
Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

app/testpmd: show Rx/Tx burst mode description

Add the 'Burst mode' section into command 'show rxq|txq info <port_id>
<queue_id>' to show the Rx/Tx burst mode description like:
  "Burst mode: Vector AVX2 Scattered"

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support Rx/Tx burst mode info

Retrieve burst mode options according to the selected Rx/Tx burst
function name.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/i40e: support Rx/Tx burst mode info

Retrieve burst mode options according to the selected Rx/Tx burst
function name.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

ethdev: add API for getting burst mode information

Some PMDs have more than one Rx/Tx burst paths, add the ethdev API
that allows an application to retrieve the mode information about
Rx/Tx packet burst such as Scalar or Vector, and Vector technology
like AVX2.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/bnxt: fix error handling in xstats

Add missing return instead of setting the error status in case of error.

Fixes: bfb9c2260be2 ("net/bnxt: support xstats get/reset")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix endianness while retrieving MTU from FW

In bnxt_hwrm_func_qcfg(), use rte_le_to_cpu_16 while getting
the mtu value from command response.

Fixes: 905cd45ce30e ("net/bnxt: use configured MTU during load")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix race between interrupt handler and dev config

Fix a possible that while interrupt handler is in the middle of
processing, dev_configure_op() might come in and try to destroy/realloc
the default completion ring on which interrupt completions arrive.
Synchronize the two so they don't race each other

Fixes: 7bc8e9a227cc ("net/bnxt: support async link notification")
Cc: stable@dpdk.org
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: move macro definitions to header file

Move device and vendor ids macro definition from bnxt_ethdev.c
to bnxt.h.
There is no functionality impact.

Fixes: 882aa9c6e2e1 ("net/bnxt: move PCI device ids to the driver")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: cleanup comments

- updated one comment for more readability and understanding
- fixed typo in comments
- moved a comment to right place
- removed one out-of-date comment
- removed few commented code

Fixes: f2a768d4d186 ("net/bnxt: add completion ring")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: reset Rx allocation state on Rx queue restart

Call bnxt_rxq_vec_setup() to ensure that rxq->rxrearm_nb and
rxq->rxrearm_start are initialized correctly when a receive
queue is restarted.

Fixes: bc4a000f2f53 ("net/bnxt: implement SSE vector mode")
Cc: stable@dpdk.org
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Xiaobing Zhang <xiaobing.zhang@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix dereference before null check

This patch fixes potential null pointer access in bnxt_alloc_ag_data().
Fix to return an error if null check is true.

Fixes: daef48efe5e5 ("net/bnxt: support set MTU")
Cc: stable@dpdk.org
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/ice/base: improve misc code style

Combine a couple of function definitions that can fit on one line.
RCT a variable declaration.

Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: add QFI for flow director

Added the GTP QFI field to the Flow director interface to allow
matching against this field. Since this field only appears in GTP
packets with extension headers, this also requires adding profile
TCAM mask matching capability.  This allows comprehending different
PTYPE attributes by examining flags from the parser. Using this
method, different profiles can be used by examining flag values
from the parser.

Signed-off-by: Dan Nowlin <dan.nowlin@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: fix switch rule programming for all profiles

In switch rule programming, if the caller requested tunnel type of
ICE_SW_TUN_AND_NON_TUN, then the code would incorrectly attempt to
add a tunneled UDP port in the training packet, this would cause the
rule addition to fail. This patch does not attempt to add the UDP
port so that the rule programming will succeed.

Fixes: c3d6ac02c0c3 ("net/ice/base: update switch training packets with open ports")

Signed-off-by: Dan Nowlin <dan.nowlin@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: fix flow raw field vector extraction

Correct the ordering of raw field extraction in the field vector
by taking into account the ordering setting for requesting block.

Fixes: aa1cd410fa64 ("net/ice/base: add flow module")
Cc: stable@dpdk.org
Signed-off-by: Dan Nowlin <dan.nowlin@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: fix unexpected switch rule overwrite

A switch rule with "drop" action will be overwritten by a
rule with same pattern match but with a "to queue" action.
While in an inversed flow creation sequence, the "to queue"
can't be overwritten by the "drop" rule.

The inconsistent behavior is not expected, the patch
fix the issue by preventing rule overwrite in both cases.

Fixes: fed0c5ca5f19 ("net/ice/base: support programming a new switch recipe")
Cc: stable@dpdk.org
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: initialize structures to zero

Some functions create ice_pkg_enum structure, but it seems it's possible
some of the members are used un-initialized.  So we'll initialize all
instantiations of this structure within ice_flex_pipe.c

The patch also fix header comment mismatch issue for
ice_init_prof_result_bm

Signed-off-by: Jeb Cramer <jeb.j.cramer@intel.com>
Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: remove dead error condition

The pointer cmd is set to an address of a structure, which can never be
NULL.  Remove the check-for-NULL lines since it's dead code anyway.

Signed-off-by: Jeb Cramer <jeb.j.cramer@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: improve flow director masking

Currently, 3-tuple FD matching is implemented using masking. However,
this is using up twenty-four of the thirty-two FD masks available. This
patch uses the swap register more efficiently to implement the 3-tuple
matches, which saves all FD masks for other uses.

Added IPV6 versions of DSCP, TTL and Protocol fields for flow director
use.

Signed-off-by: Dan Nowlin <dan.nowlin@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ice/base: support GTPU TEID for flow director

Added the training packet for GTPU TEID field to the Flow director to
allow matching against this field.

Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>