net/qede/base: re-arrange few structures for DDC

This patch rearranges some of the base driver structures which will be
also used by debug data collection (DDC) implementation. It adds a new
file ecore_hsi_func_common.h with Physical, Virtual memory descriptors.

Signed-off-by: Rasesh Mody <rmody@marvell.com>
Signed-off-by: Igor Russkikh <irusskikh@marvell.com>

net/netvsc: detach external buffer on failure

When external buffer is used, driver should detach it if it doesn't make
it successfully to the queue.

Signed-off-by: Long Li <longli@microsoft.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/netvsc: fix underflow when Rx external mbuf

When rte_pktmbuf_attach_extbuf() is used, the driver should not decrease
the reference count in its callback function hn_rx_buf_free_cb, because
the reference count is already decreased by rte_pktmbuf. Doing it twice
may result in underflow and driver may never send an ack packet over
vmbus to host.

Also declares rxbuf_outstanding as atomic, because this value is shared
among all receive queues.

Fixes: 4e9c73e96e83 ("net/netvsc: add Hyper-V network device")
Cc: stable@dpdk.org
Signed-off-by: Long Li <longli@microsoft.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

net/bonding: fix socket ID check

Caught by code review, rte_eth_dev_socket_id() returns -1 on error.
The code should behave the same, but still, do not use LCORE_ID_ANY for
something that is not a lcore id.

Fixes: c15c5897340d ("net/bonding: avoid allocating mempool on unknown socket")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Chas Williams <chas3@att.com>

net/bnxt: avoid hard coded values when reading counters

Instead of using hardcoded values for the byte/pkt value shifts/masks
to read from the HW counters, use the shift/mask values from the device
template params

Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: enable filter ctrl ops for port representor

Inorder to offload flows on the vfrep device, it must be
populated with rte_flow_ops.

This patch enables the same.

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: use VF VNIC if port action is for VF rep

Fix to use the vf's vnic port for ingress flows whose
port action is a vf rep port.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support src and dst port actions

Added support for set transport port source and destination
rewrite action items. This allows changing the tcp or udp
source/destination ports for a given flow.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support NAT action items

Added support for set ipv4 address action items. It allows the source
or destination ip address to be changed for a given flow.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add conditional opcodes for mapper result table

Added support for conditional mapper result opcodes. The conditional
opcodes allows to set the action details in hardware based on the
actions configured for the flow. This allows aggregation of multiple
templates.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: change port config for full offload

Added port configuration changes to support full offload
rules when VF representor ports are used. The direction of
the flow is determined using the configured direction and the
configured match and action ports of the flow create.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: enable PF and VF port action items

Added support for the PF and VF port action items in the flow
create. During flow create the output port action can now be specified
as PF or VF port and those ports are parsed accordingly and converted
to vnic or vport as per the flow direction.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add VLAN tag count in computed field list

Added number of vlan tags in the computed field list so conditional
table execution could be done based on number of vlan tags in the
flow create.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: move VXLAN outer IP protocol ID in encap

The outer ip protocol was not encapsulated in the right location
when ip header is sent by the application. The order of encapsulation
has to be reversed.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove VNIC and vport bits from template match

Removed the vnic and vport bitmaps from template matching. It
is assumed that these will be populated implicitly and based
on the direction the appropriate action property shall be used.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support VLAN push and pop actions

Add support for the vlan push and vlan pop actions

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: cleanup and refactor session management

The return value of some functions is explicitly ignored
in cases where scope id may not be valid for internal EM
entries.
Additional minor refactoring and cleanups
- Change log level for some log messages to DEBUG instead of ERR.
- Check data size conformity and log appropriate message.

Signed-off-by: Michael Wildt <michael.wildt@broadcom.com>
Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add HWRM for global config

Change global cfg from tunneled to non-tunneled
HWRM cmds.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Peter Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: update HWRM structures

Brought in the latest hsi_struct_def_dpdk.h in order to get
the TF global cfg set/get HWRM cmds.
HWRM API is now updated to version 1.10.1.48

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support index opcode constant

Add support for index opcode constant so that
parif configuration could be constant value.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: validate port ID and PHY port parsing

Added validations to the ulp parser to validate the dpdk port id
and phy port index during the flow creation.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support negative conditional opcodes

Added support for negative conditional opcodes in the
mapper processing.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support VLAN header bitmap

Add support for the vlan headers in the matching of the flow
patterns.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: use source property for VXLAN encap/decap

The vxlan encap and decap flows need to allocate the source
record property and populate the action fields during the
flow parsing.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

net/bnxt: support NEON

Add bnxt vector PMD support using NEON SIMD instructions.
Also update the 20.08 release notes with this information.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

net/bnxt: support Rx/Tx burst mode info

Retrieve burst mode options according to the selected Rx/Tx burst
function name.
Update 20.08 release notes with this information.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: disable vector Rx for mark action

The bnxt vector mode receive handler does not support the rte_flow
'mark' action. Since we cannot know in advance whether this action
will be required, add support for dynamically switching from vector
to non-vector receive when the first flow create request with a
mark action is processed.

Fixes: 94eb699bc82e ("net/bnxt: support flow mark action")
Cc: stable@dpdk.org
Suggested-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/dpaa2: support raw flow classification

Add support for raw flow, which can be used for any
protocol rules.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: configure per-class distribution size

TC distribution size is set with dist_queues or

nb_rx_queues % dist_queues in order of TC priority.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support flow API FS miss action configuration

1) dpni_set_rx_hash_dist and dpni_set_rx_fs_dist used for TC
   configuration instead of dpni_set_rx_tc_dist. Otherwise,
   re-configuration of default TC of QoS fails.

2) Default miss action is to drop.  "export
   DPAA2_FLOW_CONTROL_MISS_FLOW=flow_id" is used receive the missed
   packets from flow with flow ID specified.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: modify flow API QoS setup to follow FS setup

In HW/MC logical, QoS setup should follow FS setup.
In addition, Skip QoS setup if MAX TC number of DPNI is set 1.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: add flow data sanity check

Check flow attributions and actions before creating flow.
Otherwise, the QoS table and FS table need to re-build
if checking fails.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support index of queue action for flow

Make more sense to use RXQ index for queue distribution

instead of flow ID.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support distribution size set on multiple TCs

Default distribution size of TC is 1, which is limited by MC. We have to
set the distribution size for each TC to support multiple RXQs per TC.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: differentiate IPv4/IPv6 in IP flow setup

Discriminate between IPv4 and IPv6 in generic IP flow setup.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: add logging of flow extracts and rules

This patch add support for logging the flow rules.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: define size of table entry

If entry size is not bigger than 27, MC alloc one TCAM entry,
otherwise, alloc 2 TCAM entries.
Extracts size by HW must be not bigger than TCAM entry size(27 or 54).
So define the flow entry size as 54.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support QoS or FS table entry indexing

Calculate QoS/FS entry index by group and priority of flow.

1)The less index of entry, the higher priority of flow.
2)Verify if the flow with same group and priority has been added before
  creating flow.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: free flow rule memory

Free rule memory when the flow is destroyed.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: add sanity check for flow extracts

Define extracts support for each protocol and check the fields of each
pattern before building extracts of QoS/FS table.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support key extracts of flow API

1) Support QoS extracts and TC extracts for multiple TCs.

2) Protocol type of L2 extract is used to parse L3.
   Next protocol of L3 extract is used to parse L4.

3) generic IP key extracts instead of IPv4 and IPv6 respectively.

4) Special for IP address extracts:
   Put IP(v4/v6) address extract(s)/rule(s) at the end of extracts array
   to make rest fields at fixed position.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: support dynamic flow control

Dynamic flow used instead of layout defined.

The actual key/mask size depends on protocols and(or) fields
of patterns specified.
Also, the key and mask should start from the beginning of IOVA.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/dpaa: enable set link status

Enabled set link status API to start/stop phy
device from application.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/dpaa: enable link state interrupt

Enable/disable link state interrupt and get link state api is
defined using IOCTL calls from kernel driver

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

drivers/dpaa: optimize thread local storage

Minimize the number of different thread variables

Add all the thread specific variables in dpaa_portal
structure to optimize TLS Usage.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa: update process specific device info

For DPAA devices the memory maps stored in the FMAN interface
information is per process. Store them in the device process specific
area.
This is required to support multi-process apps.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa: support 2.5G

Handle 2.5Gbps ethernet ports as well.

Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/dpaa: enable Tx queue taildrop

Enable congestion handling/tail drop for TX queues.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: rename cinh read functions used for ls1088

This patch changes the qbman I/O function names as they are
only reading from cinh register, but writing to cena registers.

This gives way to add functions which purely work in cinh mode

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: support portal migration

The patch adds support for portal migration by disabling stashing
for the portals which is used in the non-affined threads, or on
threads affined to multiple cores

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: handle portal allocation failure

Add the error handling on failure.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: rework portal allocation to per-thread basis

The patch reworks the portal allocation which was previously
being done on per lcore basis to a per thread basis.
Now user can also create its own threads and use DPAA2 portals
for packet I/O.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: combine thread-specific variables

This is to reduce the thread local storage

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa2: enable timestamp for Rx offload case

This patch enables the packet timestamping
conditionally when Rx offload is enabled for timestamp.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/dpaa: fix FD offset data type

On DPAA fd offset is 9 bits, but we are using uint8_t in the
SG case. This patch fixes the same.

Fixes: 8cffdcbe85aa ("net/dpaa: support scattered Rx")
Cc: stable@dpdk.org
Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

bus/fslmc: fix getting FD error

Fix the incorrect register for getting error

Fixes: 03e36408b9fb ("bus/fslmc: add macros required by QDMA for FLE and FD")
Cc: stable@dpdk.org
Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

ethdev: verify reserved HW ring

Function 'rte_eth_dma_zone_reserve()' returns an existing memzone based
on name match, but other requested attributes are discarded.
This may cause driver using a memzone with wrong size or alignment.

Verify size, alignment and socket_id for matched memzone, and do not use
memzone if any one of the attributes are not justified.

It is possible to free the existing memzone and allocate again with the
requested attributes but it is better caller do the explicit free.

Reported-by: Renata Saiakhova <renata.saiakhova@ekinops.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

app/testpmd: support RSS config in flow query

This patch support RSS action in flow query.
It can display the RSS configuration of the specified rule.

For example:
we can create an RSS rule by command "flow create 0 ingress
pattern eth / ipv4 / tcp / end actions rss types ipv4-tcp
l3-src-only l4-dst-only end queues end func symmetric_toeplitz
 / end" and then query it "flow query 0 0 rss"
the log will be follow
RSS:
 queues: none
 function: symmetric_toeplitz
 types:
  ipv4-tcp
  l3-src-only
  l4-dst-only

Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/i40e: enable flow query RSS

This patch enables flow query function to get the
configuration of the specified rule.

Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>

app/testpmd: replace legacy filter usage

The legacy filter API will be superseded. This patch use
private api to change the implementation of commands
global_config <port_id> gre-key-len <key_len> and
show port fdir <port_id>

Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Qiming Yang <qiming.yang@intel.com>

net/ixgbe: add private APIs for flow filtering

The legacy filter API will be superseded by rte_flow.
There are also several small features which can not be
implemented in rte_flow. This patch re-implemented these
features as private API.
Two APIs are added:
rte_pmd_ixgbe_get_fdir_info.
rte_pmd_ixgbe_get_fdir_stats.

Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>

net/i40e: add private APIs for flow filtering

The legacy filter API will be superseded by rte_flow.
There are also several small features which can not be
implemented in rte_flow. This patch re-implemented these
features as private API.
Three APIs are added:
rte_pmd_i40e_get_fdir_info.
rte_pmd_i40e_get_fdir_stats.
rte_pmd_i40e_set_gre_key_len.

Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>

net/i40e: support cloud filter with L4 port

This patch enables cloud filter for IPv4/6_UDP/TCP/SCTP with
SRC port only or DST port only.
This supports different filter types for the same packet type.
E.g. one IPv4_UDP rules with SRC port only and another IPv4_UDP rule
with DST port only.

Signed-off-by: Guinan Sun <guinanx.sun@intel.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>

net/iavf: support flow director GTPU outer IPv4/IPv6

Add FDir support for MAC_IPV4_GTPU and MAC_IPV6_GTPU type with outer
IPv4/IPv6 address, teid and qfi fields matching. Note that outer IPv4
and IPv6 matching fields here include both SRC & DST of both IPv4 &
IPv6.

Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support flow director GTPU outer IPv4/IPv6

Add FDir support for MAC_IPV4_GTPU and MAC_IPV6_GTPU type with outer
IPv4/IPv6 address, teid and qfi fields matching. Note that outer IPv4
and IPv6 matching fields here include both SRC & DST of both IPv4 &
IPv6.

Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support src MAC filter for flow director

This patch enables FDIR with src mac filter and change the logic for
only supporting ether type field without IP pattern. Application may
set ether type with IP. It is redundant for flow, ignore ether type
except for ICE_FLTR_PTYPE_NON_IP_L2.

Also, remove redundant code for checking next_type.

Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: fix tunnel filtering rule

If we download a NVGRE rule like "eth / ipv4 / nvgre / eth / ipv4 src
is 192.168.1.2 dst is 192.168.1.3 / end actions queue index 3 / end"
the special word will not be added because of rm->n_grp_count = 1,
so we have to change the ice_add_special_words() function before
ice_create_recipe_group(), then the special words will be add
into rm->rg_list.
Btw, the flag match_tun_mask is useless now and no need to add special
words in function ice_add_sw_recipe().

Fixes: eda40e22b858 ("net/ice/base: improve switch chained recipe")
Cc: stable@dpdk.org
Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: fix tunnel type check for PPPoE

This patch fix tunnel type check for PPPoE type of recipe
when search for recipe which has been created before, and
also fix a bug of typo.

Fixes: 418d2563d10b ("net/ice/base: get tunnel type for recipe")

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

common/octeontx2: retry interrupt callback unregister

Interrupt callback unregister can fail with -EAGAIN
when interrupt handler is active in interrupt thread.
Hence retry before reporting a failure or proceeding further.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

vdpa/mlx5: enable status protocol feature

This patch advertises VHOST_USER_PROTOCOL_F_STATUS
support in the MLX5 driver so that that the protocol
feature is negotiated.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vdpa/ifc: enable status protocol feature

This patch advertises VHOST_USER_PROTOCOL_F_STATUS
support in the IFC driver so that that the protocol
feature is negotiated.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: support virtio status message

This patch adds support to the new Virtio device get status
Vhost-user message.

The driver can send this new message to read the device status.

One of the uses of this message is to ensure the feature negotiation has
succeeded.  According to the virtio spec, after completing the feature
negotiation, the driver sets the FEATURE_OK status bit and re-reads it
to ensure the device has accepted the features.

This patch also clears the FEATURE_OK status bit if the feature
negotiation has failed to let the driver know about his failure.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: support virtio status

This patch adds support to the new Virtio device status
Vhost-user protocol feature.

Getting such information in the backend helps to know
when the driver is done with the device configuration
and so makes the initialization phase more robust.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: check vDPA configuration succeed

This patch checks whether vDPA device configuration
succeed and does not set the CONFIGURED flag if it
didn't.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: make some vDPA callbacks mandatory

Some of the vDPA callbacks have to be implemented
for vDPA to work properly.

This patch marks them as mandatory in the API doc and
simplify code calling these ops with removing
unnecessary checks that are now done at registration
time.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: refactor virtio ready check

This patch is a small refactoring, as preliminary work
for adding support to Virtio status support.

No functional change here.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix virtio ready flag check

Before checking whether the device is ready is done
a check on whether the RUNNING flag is set. Then the
READY flag is set if virtio_is_ready() returns true.

While it seems to not cause any issue, it makes more
sense to check whether the READY flag is set and not
the RUNNING one.

Fixes: c0674b1bc898 ("vhost: move the device ready check at proper place")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

net/virtio-user: support reply-ack

This patch adds support reply-ack vhost-user protocol
feature, which is for now only used to ensure
VHOST_USER_SET_MEM_TABLE requests are handled by the
slave, but later will be used for VHOST_USER_SET_STATUS.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

net/virtio: support vhost-user protocol features

This patch adds support for Vhost-user protocol features.
It is required to support protocol features that were not in
initial Vhost-user specification, such as reply-ack, MTU...

Also, this patch prevents Virtio multiqueue feature negotiation
if the slave does not support MQ protocol feature as stated
in Vhost-user specification:
"The multiple queues feature is supported only when the protocol
feature ``VHOST_USER_PROTOCOL_F_MQ`` (bit 0) is set."

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

eal/linux: truncate thread name

pthread_setname_np refuses names larger than 16 bytes (\0 included).
Rather than return an error, truncate the name to this limit in the
rte_thread_setname helper.

Caught with ixgbe which creates control thread with name
"ixgbe-link-handler":

Configuring Port 0 (socket 0)
EAL: Cannot set name for ctrl thread
...
EAL: Cannot set name for ctrl thread

Port 0: link state change event
...
EAL: Cannot set name for ctrl thread

Port 0: link state change event

Note: before this change, the thread would keep its original name, which
meant in my test for the ixgbe handler either "dpdk-testpmd" or
"eal-intr-thread".

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

lpm: hide defer queue handle

There is no need to return the defer queue handle in rte_lpm_rcu_qsbr_add,
since enough flexibility has been provided to configure the defer queue.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

examples/l3fwd-power: select default mode automatically

Currently, the application does support running without the power
library being initialized, but it has to be specifically requested. On
platforms without support for frequency scaling using the power library,
we can just enable interrupt-only mode by default.

Suggested-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Harman Kalra <hkalra@marvell.com>

power: add environment capability probing

Currently, there is no way to know if the power management env is
supported without trying to initialize it. The init API also does
not distinguish between failure due to some error and failure due to
power management not being available on the platform in the first
place.

Thus, add an API that provides capability of probing support for a
specific power management API.

Suggested-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

examples/l3fwd-power: add interrupt-only mode

In addition to existing modes, add a mode which is very similar to
legacy mode, but does not do frequency scaling, and thus does not
depend on the power library.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Yinan Wang <yinan.wang@intel.com>

examples/l3fwd-power: add legacy mode option

Currently, legacy mode is the implicit default, but it is not possible
to directly request using legacy mode. Add the argument to enable
requesting legacy mode, and also make it the default.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

examples/l3fwd-power: fix style and control flow

Make the coding style more consistent, and the init logic control flow
more explicit.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

examples/l3fwd-power: allow only supported environments

Currently, l3fwd-power will attempt to run even if the power env
is set to KVM, which is not supported. Fix this by preventing the
app from initializing unless the env is set to one of the supported
modes.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Yinan Wang <yinan.wang@intel.com>

examples/l3fwd-power: disable interrupts by default

Currently, interrupts are enabled in telemetry and empty poll modes, but
they are not used. Switch to disabling interrupts by default, and only
enable interrupts for modes that require them.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Yinan Wang <yinan.wang@intel.com>

pci: keep API compatibility with mmap values

The function pci_map_resource() returns MAP_FAILED in case of error.
When replacing the call to mmap() by rte_mem_map(),
the error code became NULL, breaking the API.
This function is probably not used outside of DPDK,
but it is still a problem for two reasons:
- the deprecation process was not followed
- the Linux function pci_vfio_mmap_bar() is broken for i40e

The error code is reverted to the Unix value MAP_FAILED.
Windows needs to define this special value (-1 as in Unix).
After proper deprecation process, the API could be changed again
if really needed.

Because of the switch from mmap() to rte_mem_map(),
another part of the API was changed: "int additional_flags"
are defined as "additional flags for the mapping range"
without mentioning it was directly used in mmap().
Currently it is directly used in rte_mem_map(),
that's why the values rte_map_flags must be mapped (sic) on the mmap ones
in case of Unix OS.

These are side effects of a badly defined API using Unix values.

Bugzilla ID: 503
Fixes: 2fd3567e5425 ("pci: use OS generic memory mapping functions")

Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Tested-by: Lihong Ma <lihongx.ma@intel.com>

eal: fix parentheses in alignment macros

Found an issue while using RTE_ALIGN_MUL_NEAR with an
expression, like as passed in estimate_tsc_freq().
RTE_ALIGN_MUL_FLOOR resulted in unexpected value as
parathesis are required to evaluate an expression.

Fixes: 5120203d753f ("eal: add macros to align value to multiple")
Cc: stable@dpdk.org
Signed-off-by: Harman Kalra <hkalra@marvell.com>

doc: add RIB and FIB into the API index

Add RIB/FIB library into the API doxygen index.
Move LPM/LPM6 under separate section "routing".

Fixes: 5a5793a5ffa2 ("rib: add RIB library")
Fixes: f7e861e21c46 ("rib: support IPv6")
Fixes: 39e927248416 ("fib: add FIB library")
Fixes: 40d41a8a7b34 ("fib: support IPv6")
Cc: stable@dpdk.org
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

eal/windows: detect insufficient privileges for hugepages

AdjustTokenPrivileges() succeeds even if no requested privileges have
been granted; this behavior is documented. Check last error code in
addition to return value to detect such case.

Make error messages more specific and add troubleshooting hint.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>

net: fix checksum on big endian CPUs

With current code, the checksum of odd-length buffers is wrong on
big endian CPUs: the last byte is not properly summed to the
accumulator.

Fix this by left-shifting the remaining byte by 8. For instance,
if the last byte is 0x42, we should add 0x4200 to the accumulator
on big endian CPUs.

This change is similar to what is suggested in Errata 3133 of
RFC 1071.

Fixes: 6006818cfb26("net: new checksum functions")
Cc: stable@dpdk.org
Signed-off-by: Hongzhi Guo <guohongzhi1@huawei.com>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

net: fix unneeded replacement of TCP checksum 0

Per RFC768:
If the computed checksum is zero, it is transmitted as all ones.
An all zero transmitted checksum value means that the transmitter
generated no checksum.

RFC793 for TCP has no such special treatment for the checksum of zero.

Fixes: 6006818cfb26 ("net: new checksum functions")
Cc: stable@dpdk.org
Signed-off-by: Hongzhi Guo <guohongzhi1@huawei.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>

vhost: restrict pointer aliasing for packed ring

Restrict pointer aliasing to allow the compiler to vectorize loop
more aggressively.

With this patch, a 9.6% improvement is observed in throughput for
the packed virtio-net PVP case, and a 2.8% improvement in throughput
for the packed virtio-user PVP case. All performance data are measured
on ThunderX-2 platform under 0.001% acceptable packet loss with 1 core
on both vhost and virtio side.

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>

net/virtio: restrict pointer aliasing for NEON

Restrict pointer aliasing to allow the compiler to vectorize loops
more aggressively.

With this patch, a 9.6% improvement is observed in throughput for
the virtio-net PVP case, and a 2.4% perf improvement in throughput
for the virtio-user PVP case. All performance data are measured
on ThunderX-2 platform under the 0.001% acceptable packet loss with
2 cores on the vhost side and 1 core on the virtio side.

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>

introduce restricted pointer aliasing marker

The 'restrict' keyword is recognized in C99, while type qualifier
'__restrict' compiles ok in C with all language levels. This patch
is to replace the existing 'restrict' with '__rte_restrict' which
is a common wrapper supported by all compilers.

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

test/lpm: add RCU performance tests

Add performance tests for RCU integration. The performance
difference with and without RCU integration is very small
(~1% to ~2%) on both Arm and x86 platforms.

Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

test/lpm: add RCU functional tests

Add positive and negative tests for API rte_lpm_rcu_qsbr_add.
Also test LPM library behavior when RCU QSBR is enabled.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>

lpm: implement RCU rule reclamation

Currently, the tbl8 group is freed even though the readers might be
using the tbl8 group entries. The freed tbl8 group can be reallocated
quickly. This results in incorrect lookup results.

RCU QSBR process is integrated for safe tbl8 group reclaim.
Refer to RCU documentation to understand various aspects of
integrating RCU library into other libraries.

To avoid ABI breakage, a struct __rte_lpm is created for lpm library
internal use. This struct wraps rte_lpm that has been exposed and
also includes members that don't need to be exposed such as RCU related
config.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>