net/mlx5: support shared Rx queue

This patch introduces shared RxQ. All shared Rx queues with same group
and queue ID share the same rxq_ctrl. Rxq_ctrl and rxq_data are shared,
all queues from different member port share same WQ and CQ, essentially
one Rx WQ, mbufs are filled into this singleton WQ.

Shared rxq_data is set into device Rx queues of all member ports as
RxQ object, used for receiving packets. Polling queue of any member
ports returns packets of any member, mbuf->port is used to identify
source port.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: remove Rx queue data list from device

Rx queue data list(priv->rxqs) can be replaced by Rx queue
list(priv->rxq_privs), removes it and replaces with universal wrapper
API.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: move Rx queue DevX resource

To support shared RX queue, moves DevX RQ which is per queue resource to
Rx queue private data.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: remove port info from shareable Rx queue

To prepare for shared Rx queue, removes port info from shareable Rx
queue control.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: move Rx queue hairpin info to private data

Hairpin info of Rx queue can't be shared, moves to private queue data.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: move Rx queue reference count

Rx queue reference count is counter of RQ, used to count reference to RQ
object. To prepare for shared Rx queue, this patch moves it from
rxq_ctrl to Rx queue private data.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: split Rx queue into shareable and private

To prepare shared Rx queue, splits RxQ data into shareable and private.
Struct mlx5_rxq_priv is per queue data.
Struct mlx5_rxq_ctrl is shared queue resources and data.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: clean Rx queue code

This patch removes unused Rx queue code.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix Rx queue memory allocation return value

If error happened during Rx queue mbuf allocation, boolean value
returned. From description, return value should be error number.

This patch returns negative error number.

Fixes: 0f20acbf5eda ("net/mlx5: implement vectorized MPRQ burst")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: support receive memory pool

The hardware Receive Memory Pool (RMP) object holds the destination for
incoming packets/messages that are routed to the RMP through RQs. RMP
enables sharing of memory across multiple Receive Queues. Multiple
Receive Queues can be attached to the same RMP and consume memory
from that shared poll. When using RMPs, completions are reported to the
CQ pointed to by the RQ, user index that set in RQ creation time is
carried to completion entry.

This patch enables RMP based RQ, RMP is created when mlx5_devx_rq.rmp is
set.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: support basic receive memory pool

The hardware Receive Memory Pool (RMP) object holds the destination for
incoming packets/messages that are routed to the RMP through RQs. RMP
enables sharing of memory across multiple Receive Queues. Multiple
Receive Queues can be attached to the same RMP and consume memory
from that shared poll. When using RMPs, completions are reported to the
CQ pointed to by the RQ, and this Completion Queue can be shared as
well.

This patch adds DevX supports of PRM RMP object.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: introduce user index field in completion

On ConnectX devices the completion entry provides the dedicated 24-bit
field, that is filled up with some static value assigned at the
Receiving Queue creation moment. This patch declares this field. This is
a preparation step for supporting shared RQs and the field is supposed
to provide actual port index while handling the shared receiving
queue(s).

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: fix Altivec Rx

This patch fixes stale field reference.

Fixes: a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: handle flex item in flows

Provide flex item recognition, validation and translation
in flow patterns. Track the flex item referencing.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: translate flex item pattern into matcher

The matcher is an steering engine entity that represents
the flow pattern to hardware to match. It order to
provide match on the flex item pattern the appropriate
matcher fields should be configured with values and masks
accordingly.

The flex item related matcher fields is an array of eight
32-bit fields to match with data captured by sample registers
of configured flex parser. One packet field, presented in
item pattern can be split between several sample registers,
and multiple fields can be combined together into single
sample register to optimize hardware resources usage
(number os sample registers is limited), depending on field
modes, widths and offsets. Actual mapping is complicated
and controlled by special translation data, built by PMD
on flex item creation.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: translate flex item configuration

RTE Flow flex item configuration should be translated
into actual hardware settings:

  - translate header length and next protocol field samplings
  - translate data field sampling, the similar fields with the
    same mode and matching related parameters are relocated
    and grouped to be covered with minimal amount of hardware
    sampling registers (each register can cover arbitrary
    neighbour 32 bits (aligned to byte boundary) in the packet
    and we can combine the fields with smaller lengths or
    segments of bigger fields)
  - input and output links translation
  - preparing data for parsing flex item pattern on flow creation

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: add flex parser DevX object management

The DevX flex parsers can be shared between representors
within the same IB context. We should put the flex parser
objects into the shared list and engage the standard
mlx5_list_xxx API to manage ones.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: add flex item operations

This patch is a preparation step of implementing
flex item feature in driver and it provides:

  - external entry point routines for flex item
    creation/deletion

  - flex item objects management over the ports.

The flex item object keeps information about
the item created over the port - reference counter
to track whether item is in use by some active
flows and the pointer to underlying shared DevX
object, providing all the data needed to translate
the flow flex pattern into matcher fields according
hardware configuration.

There is not too many flex items supposed to be
created on the port, the design is optimized
rather for flow insertion rate than memory savings.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: update eCPRI flex parser structures

To handle eCPRI protocol in the flows the mlx5 PMD engages
flex parser hardware feature. While we were implementing
eCPRI support we anticipated the flex parser usage extension,
and all related variables were named accordingly, containing
flex syllabus. Now we are preparing to introduce more common
approach of flex item, in order to avoid naming conflicts
and improve the code readability the eCPRI infrastructure
related variables are renamed as preparation step.

Later, once we have the new flex item implemented, we could
consider to refactor the eCPRI protocol support  to move on
common flex item basis.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: fix flex parser DevX creation routine

Add missing modify_field_select, next_header_field_size
field values setting.

Fixes: 38119ebe01d6 ("common/mlx5: add DevX command for flex parsers")
Cc: stable@dpdk.org
Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: extend flex parser capabilities

MLX5 PARSE_GRAPH_NODE is the main data structure used by the Flex
Parser when a new parsing protocol is defined. While software
creates PARSE_GRAPH_NODE object for a new protocol, it must
verify that configuration parameters it uses comply with
hardware limits.

The patch queries hardware PARSE_GRAPH_NODE capabilities and
stores ones in PMD internal configuration structure:

 - query capabilities from parse_graph_node attribute page
 - query max_num_prog_sample_field capability from HCA page 2

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

common/mlx5: refactor HCA attributes query

There is the common part of code querying the HCA attributes
from the device, and this part can be commoditized as
dedicated routine.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/bnxt: fix scalar Rx datapath on Thor

The patch introduced by
commit 657c2a7f1dd4 ("net/bnxt: create aggregation rings when needed")
ended up shortening the return code path from the function thereby
resulting in not executing the line of code at the end of the function
that was resetting the next consumer index to 0.
This would result in an application crash when error recovery or other
port stop/start scenarios were invoked on Thor which is what this
commit 61cd4384fabf ("net/bnxt: fix crash after port stop/start")
was addressing.
Fix it by moving the resetting line of code before the return path
in the case when aggregation rings are not used (default case).

Fixes: 657c2a7f1dd4 ("net/bnxt: create aggregation rings when needed")
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: use enum for bank ID

Instead of integer, using enum tf_sram_bank_id for bank
id in tf_set_sram_policy_parms.

Add index check against the allocation of the meter
instance for meter drop count because there is no
reason to access it if the corresponding meter
entry is not allocated.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Reviewed-by: Steve Rempe <steve.rempe@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: check mismatch of control and physical port

During the parsing of the ingress port ignore for a flow, added
check to match the control port and the physical port that is configured
to be ignored. If they do not match then the configuration to setup the
svif ignore shall fail.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Shahaji Bhosle <sbhosle@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove 2 slice wildcard entries

Remove 2-slice wildcard entries for scale.
The type-5 wildcard IPv6 flows are removed in order to increase
the scale for app-id=3.
The app no longer supports 2-slice wildcard entries.

Signed-off-by: Mike Baucom <michael.baucom@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add Tx TruFlow table config for P4 device

Add TX direction TruFlow table type config to be
compatible with other devices. For P4 device, the TX cfg
is duplicated from RX.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support TruFlow and AFM SRAM partitioning

Implement set/get_sram_policy which support both Rx/Tx
direction truflow type the specific SRAM bank.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add TruFlow API to get SRAM resources

Implement tf_get_sram_resources to return SRAM
partition information, including bank count and
SRAM profile number.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: modify VF representor allocation sequence

When the VF representor interface is created, the VF pair relationship
is established between the VF and it is representor. If the pair
already exists then the pair needs to be deleted before allocation.
This could happen if the application is abruptly killed and restarted.
If the deletion of an existing VF rep is not done then hw pipeline is
not cleaned and a new allocation shall leave the hw in inconsistent
state.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support socket direct feature

Added support for socket direct feature. This feature
allows to ignore the incoming interface and use other fields
in the packet to identify the flow and forward.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: update log messages in TruFlow path

Some of the error level log messages are made debug level messages.
When Truflow is not enabled then Truflow init error messages are
moved to debug level instead.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix type casting

Typecast flow_item type, action_item type and the ENUMs to uint32_t
before comparing.

Bugzilla ID: 821
Fixes: bdf4a3c6316b ("net/bnxt: support tunnel offload")
Signed-off-by: Shahaji Bhosle <sbhosle@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove devargs option for stats accumulation

The accumulation of flow counters is not determined by the application
device arguments anymore. Instead it is now dictated by the platform
capabilities whether to do software based accumulation or not.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: increase flow scale for Thor

* Updated defines and data types to allow 256 VFRs.
* Increased the encap record cache to support 256 to 4K entries. So
  VxLAN connections can be scaled to 4K entries.

Signed-off-by: Shahaji Bhosle <sbhosle@broadcom.com>
Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: get TruFlow version

Implement tf_get_version that returns TruFlow version
numbers and CFA resources capabilities.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Peter Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support inner IP header for GRE tunnel flows

This change allows adding IP header matches for GRE flows that
does not specify outer IP header in the flow match pattern.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: enable wildcard match for ingress flows

Enabled wildcard match support for IPv4 ingress flows.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>

net/bnxt: add suffix to unsigned long flow signatures

The flow signature is of type unsigned long value and needs to be
suffixed with UL. This patch addresses ISO C90 compilation error.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: remove unused functions

Remove unused functions from the code.
Specifically the following functions are being removed.

bnxt_get_iface_mac();
bnxt_get_vnic_id();
bnxt_get_parent_vnic_id();
bnxt_get_bp();
bnxt_get_svif();
bnxt_get_fw_func_id();
bnxt_get_parif();
bnxt_get_phy_port_id();
bnxt_get_vport();
bnxt_get_interface_type();

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: add capability option for socket redirect

Added support for socket redirect feature capability so applications
can enable or disable this feature. This patch contains the template
changes.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support clear on read

Add clear on read stats support for Thor. Currently, the
flow stats are not cleared after they are read from the FW.
This patch adds support for clear on read. Since clear on
read support is added for flow stats in Thor, the flow
accumulation is enabled on Thor as well.

Signed-off-by: Farah Smith <farah.smith@broadcom.com>
Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: fix out of bounds access in hash list

The number of hash bucket list calculation is fixed.
Added check to avoid the out of bounds condition.

Fixes: 0001cc58d362 ("net/bnxt: support generic hash table")
Cc: stable@dpdk.org
Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support multi root capability

Update driver to read the multi root capability and ignore
PCI address check while creating ulp session when multi root
capability is enabled in the hardware. DPDK HSI version updated
from 1.10.2.44 to 1.10.2.68.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

net/bnxt: support NAT for dest IP and port combination

* Added support for NAT action for destination IP and port
  combination for Thor devices.
* Consolidated the encapsulation and NAT entries for scaling flows
  with NAT actions.

Signed-off-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Mike Baucom <michael.baucom@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

vdpa/sfc: set multicast filter during init

Insert unknown multicast filter to allow IPv6 neighbor discovery

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vdpa/sfc: support setting vring state

Implements vDPA ops set_vring_state to configure vring state.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vdpa/sfc: support MAC filter config

Add support for unicast and broadcast MAC filter configuration.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vdpa/sfc: get queue notify area info

Implement the vDPA ops get_notify_area to get the notify
area info of the queue.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vdpa/sfc: support device configure and close

Implement vDPA ops dev_conf and dev_close for DMA mapping,
interrupt and virtqueue configurations.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vdpa/sfc: get VFIO device file descriptor

Implement vDPA ops get_vfio_device_fd to get the VFIO device fd.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vdpa/sfc: get max supported queue count

Implement vDPA ops get_queue_num to get the maximum number
of queues supported by the device.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vdpa/sfc: support device and protocol features queries

Implement vDPA ops get_feature and get_protocol_features.
This patch retrieves device supported features and enables
protocol features.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vdpa/sfc: support device initialization

Add HW initialization and vDPA device registration support.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vdpa/sfc: introduce Xilinx vDPA driver

Add new vDPA PMD to support vDPA operations of Xilinx devices.
This patch implements probe and remove functions.

Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

doc: remove deprecation notices for eventdev

All deprecation notice targeted for v21.11 of eventdev has been
committed in the following commits, remove deprecation notices.

Fixes: 3c838062b91 ("eventdev: introduce event vector Rx capability")
Fixes: 929ebdd5430 ("eventdev/eth_rx: simplify event vector config")
Fixes: 295c053f90f ("eventdev: hide event device related structures")
Fixes: dd451ad1520 ("doc: remove event crypto metadata deprecation note")
Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Jerin Jacob <jerinj@marvell.com>

eventdev: support device maintenance in adapters

Introduce support for event devices requiring calls to
rte_event_maintain() in the Ethernet RX, Timer and Crypto Eventdev
adapters.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Tested-by: Richard Eklycke <richard.eklycke@ericsson.com>

event/dsw: use maintenance facility

Set the RTE_EVENT_DEV_CAP_REQUIRES_MAINT flag, and perform DSW
background tasks on rte_event_maintain() calls.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Tested-by: Richard Eklycke <richard.eklycke@ericsson.com>
Tested-by: Liron Himi <lironh@marvell.com>

eventdev: add port maintenance API

Extend Eventdev API to allow for event devices which require various
forms of internal processing to happen, even when events are not
enqueued to or dequeued from a port.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Tested-by: Richard Eklycke <richard.eklycke@ericsson.com>
Tested-by: Liron Himi <lironh@marvell.com>

event/cnxk: rework enqueue path

Rework SSO enqueue path for CN9K make it similar to CN10K
enqueue interface.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/cnxk: reduce workslot memory consumption

SSO group base addresses are always are always contiguous we
need not store all the base addresses in workslot memory, instead
just store the base address and compute the group address offset
when required.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/cnxk: fix packet Tx overflow

The transmit loop incorrectly assumes that nb_mbufs is always
a multiple of 4 when transmitting an event vector. The max
size of the vector might not be reached and pushed out early
due to timeout.

Fixes: 761a321acf91 ("event/cnxk: support vectorized Tx event fast path")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/cnxk: use common XAQ pool functions

Use the common API to create and free XAQ pool.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

common/cnxk: add SSO XAQ pool create and free

Add common API to create and free SSO XAQ pool.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

test/event: add unit test for Rx adapter

add unit test for rte_event_eth_rx_adapter_queue_stats_get() and
rte_event_eth_rx_adapter_queue_stats_reset() APIs.

Signed-off-by: Naga Harish K S V <s.v.naga.harish.k@intel.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>

eventdev/eth_rx: support telemetry

Added telemetry support for rxa_queue_stats and
rxa_queue_stats_reset to get and reset rx queue
stats respectively.

Signed-off-by: Naga Harish K S V <s.v.naga.harish.k@intel.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>

eventdev/eth_rx: add queue stats API

This patch adds new api ``rte_event_eth_rx_adapter_queue_stats_get`` to
retrieve queue stats. The queue stats are in the format
``struct rte_event_eth_rx_adapter_queue_stats``.

For resetting the queue stats,
``rte_event_eth_rx_adapter_queue_stats_reset`` api is added.

The adapter stats_get and stats_reset apis are also updated to
handle queue level event buffer use case.

Signed-off-by: Naga Harish K S V <s.v.naga.harish.k@intel.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>

doc: add cryptodev table for supported operating systems

Added table to the crypto device drivers documentation
stating the support of each PMD on Linux, FreeBSD and Windows.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/crypto: fix vectors for ZUC-256

Fix the test vectors added for ZUC 256-bit key
Add known vectors form ZUC 256 RFC.

Fixes: fa5bf9345d4e ("test/crypto: add ZUC cases with 256-bit keys")
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

common/qat: fix queue pairs number

This patch fixes incorrect number of queue pairs.

Fixes: 4c0d2ee23c39 ("crypto/qat: remove incorrect usage of bundle number")
Cc: stable@dpdk.org
Signed-off-by: Przemyslaw Zegan <przemyslawx.zegan@intel.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

crypto/qat: add gen-specific implementation

This patch replaces the mixed QAT symmetric and asymmetric
support implementation by separate files with shared or
individual implementation for specific QAT generation.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

crypto/qat: define gen-specific structs and functions

This patch adds the symmetric and asymmetric crypto data
structure and function prototypes for different QAT
generations.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

crypto/qat: unify device private data structure

This patch unifies the QAT symmetric and asymmetric device
private data structures and functions.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

compress/qat: add gen-specific implementation

This patch replaces the mixed QAT compression support
implementation by separate files with shared or individual
implementation for specific QAT generation.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

compress/qat: define gen-specific structs and functions

This patch adds the compression data structure and function
prototypes for different QAT generations.

Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

common/qat: add gen-specific queue implementation

This patch replaces the mixed QAT queue pair configuration
implementation by separate files with shared or individual
implementation for specific QAT generation.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Signed-off-by: Przemyslaw Zegan <przemyslawx.zegan@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

common/qat: add gen-specific queue pair function

This patch adds the queue pair data structure and function
prototypes for different QAT generations.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

common/qat: add gen-specific device implementation

This patch replaces the mixed QAT device configuration
implementation by separate files with shared or
individual implementation for specific QAT generation.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

common/qat: define gen-specific structs and functions

This patch adds the data structure and function prototypes for
different QAT generations.

Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

test/crypto: fix test vectors for ZUC-256

Fix the IV for ZUC-256 test vectors

Fixes: 216125c62d28 ("test/crypto: add ZUC-256 vectors")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

security: add telemetry endpoint for capabilities

Add telemetry endpoint for cryptodev security capabilities.
Details of endpoints added in documentation.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: fix IV length for ZUC-256

Fix supported IV length for ZUC 256
Add support in capability for 4 byte mac len for ZUC 256
Pack the last 8 bytes of IV to 6 bytes by ignoring the 2 msb bits of
each byte.

Fixes: 29742632ac9e ("crypto/cnxk: support ZUC with 256-bit key")
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>

common/cnxk: fix ZUC constants

Use appropriate ZUC constants based on key length and mac length

Fixes: a90db80d7d72 ("common/cnxk: set key length for PDCP algos")
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>

crypto/mlx5: support 1MB data-unit

Add 1MB data-unit length to the capability's bitmap.
Handle 1MB data-unit length in the mlx5 session create operation,
and expose its capability in the mlx5 capabilities.

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

test/crypto: refactor DOCSIS to show hidden cases

In the current implementation, the DOCSIS test cases are running
and being reported as one test, despite the fact that multiple
test cases are hidden inside i.e. "test_DOCSIS_PROTO_all" runs
52 test cases. Each DOCSIS test case should be reported individually
instead.

This commit achieves this by removing the use of the
test_DOCSIS_PROTO_all function and statically listing the test cases
to run when building the test suite, which are then reported to the
user by description.

Signed-off-by: Rebecca Troy <rebecca.troy@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: David Coyle <david.coyle@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: support event vector

Adds event vector support to inline protocol offload mode.
By default vector support is disabled, it can be enabled by
using the option --event-vector.
Additional options to configure vector size and vector timeout are
also implemented and can be used by specifying --vector-size and
--vector-tmo.

Signed-off-by: Srujana Challa <schalla@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: support additional algorithms

Add support for AES-GMAC, AES_CTR, AES_XCBC_MAC,
AES_CCM, CHACHA20_POLY1305

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: add ethdev reset callback

Applications should not quietly ignore an ethdev reset event.
Register an event handler for ethdev reset callback
RTE_ETH_EVENT_INTR_RESET that prints a message and
quits the application.

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: define initial ESN value

New option added to the SA configuration arguments that
allows setting an arbitrary start value for ESN.

For example in the SA below ESN will be enabled and first egress
IPsec packet will have the ESN value 10000:

sa out 15 cipher_algo null auth_algo null mode ipv4-tunnel \
src 172.16.1.5 dst 172.16.2.5 \
esn 10000

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: support telemetry

Add telemetry support to the IPsec GW sample app and add
support for per SA telemetry when using IPsec library.

Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: support UDP encap for inline crypto

Enable UDP encapsulation for both transport and tunnel modes for the
inline crypto offload path.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: update inline session create

Rework create inline session function as to update the session
configuration parameters before create session is called.
Also updated the rss key array size to prevent buffers overflows
with PMDs that copy more than 40 bytes.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

examples/ipsec-secgw: add stats interval argument

Add -t for stats screen update interval, disabled by default.

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

examples/ipsec-secgw: move global array from header

When STATS_INTERVAL is set to a non-zero value the
core_statistics array will be defined in multiple
compilation units and this can trigger a linker error
on particular environments. In order to fix this the
core_statistics definition was moved out of the header file.

Fixes: 1329602b6c8f ("examples/ipsec-secgw: add per-core packet statistics")
Cc: stable@dpdk.org
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

test/compress-perf: remove unused variable

clang-13 rightfully complains that the total_deq_ops
variable in cperf_cyclecount_op_setup is set but not
used, since the final accumulated total_deq_ops
results isn't used anywhere. So just remove the
total_deq_ops variable.

Fixes: 2695db95a147 ("test/compress: add cycle-count mode to perf tool")
Cc: stable@dpdk.org
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

test/crypto-perf: fix memory allocation in asym case

While populating the crypto ops in case of asymmetric, result
is being allocated from stack. This is causing crash in the
application. And operation type is also not being initialized
properly. Adding a fix by allocating the result from global
memory and initialized the operation memory properly.

Fixes: ba588ce3f9339 ("test/crypto-perf: test asymmetric crypto throughput")
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: support IPv6 mixed tunnel mode

Adds IPv6 mixed tunnel mode support for cn9k.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: update auth key size

Update auth key size in capabilities for to support
SHA256_HMAC for cn9k.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

doc: update feature list in CN9K crypto guide

Updated feature list supported with cn9k crypto PMD.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: support ESN and anti-replay on CN9K

Adds ESN and anti-replay support for lookaside IPsec
on CN9K platforms.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>