git.droids-corp.org - dpdk.git/log

net: introduce functions to verify L4 checksums

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() and
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

Since these functions should be used to calculate the checksum to set in
a packet, introduce 2 new helpers for checksum verification. They return
0 if the checksum is valid in the packet.

Use this new helper in net/tap driver.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on TCP packets

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() or
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

This new behavior broke the checksum verification in tap driver for TCP
packets: these packets are marked with PKT_RX_L4_CKSUM_BAD.

Fix this by checking the 2 possible values. A next commit will introduce
a checksum verification helper to simplify this a bit.

Fixes: d5df2ae0428a ("net: fix unneeded replacement of TCP checksum 0")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on IP options packets

When packet type is IPV4_EXT, the checksum is always marked as good in
the mbuf offload flags.

Since we know the header lengths, we can easily call
rte_ipv4_udptcp_cksum() in this case too.

Fixes: 8ae3023387e9 ("net/tap: add Rx/Tx checksum offload support")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/cxgbe: add MAC match-all to track promiscuous traffic

Chelsio T6 ASIC doesn't track Rx promisc traffic dropped due to lack
of Rx buffers and hence the imissed counter doesn't increment. Add
support for RAW MAC filter to insert a wildcard matchall rule at
the end of MPS TCAM to make MPS track the promisc traffic. This
rule will only be added/removed when promisc mode is turned on/off
on the interface.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: use C11-style compiler builtins for atomics

Replace rte_atomic ops with C11 atomics.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/sfc: fix outer L4 checksum Rx

Do not return "GOOD" flag if the status was bad.

Fixes: 9e6e7f479ae9b ("net/sfc: support Rx checksum offload for EF100")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/pfe: remove unnecessary null check

ethdev library does the check of dev before calling the link update.

Fixes: acd4818ea2a45 ("net/pfe: add link status update")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Gagandeep Singh <g.singh@nxp.com>

app/testpmd: move conntrack update after null check

Use out only after it was checked not to be null.

Fixes: 4d07cbefe3ba0 ("app/testpmd: add commands for conntrack")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Aman Deep Singh <aman.deep.singh@intel.com>

net/hns3: support Tx push quick doorbell for performance

Kunpeng 930 support Tx push mode which could improve performance.
It works like below:
1. Add PCIe bar45 which support driver direct write the Tx descriptor
    or tail reg to it.
2. Support three operations: a) direct write one Tx descriptor, b)
    direct write two Tx descriptors, c) direct write tail reg.
3. The original tail reg located at bar23, the above bar45 tail reg
    could provide better bandwidth from the hardware perspective.

The hns3 driver only support direct write tail reg (also have the name
of quick doorbell), the detail:
Considering compatibility, firmware will report Tx push capa if the
hardware support it.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: remove unnecessary blank lines

Delete redundant blank lines to make:
1.Return value judgment follow the function call.
2.No blank lines at the end of a code block defined by braces.

Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix maximum queues on configuration failure

The "cfg_max_queues" maintains configured max queue numbers from user,
and is equal to the maximum of "nb_rx_queues" and "nb_tx_queues" in
"dev->data".

From the ethdev layer framework, "nb_rx/tx_queues" in "dev->data" were set
to zero in rte_eth_dev_configure() if ops.dev_configure in PMD fails to be
executed, In addition, if ops.dev_configure in HNS3 PMD failed, the fake
queues are also cleared on a device that does not support independent Rx/Tx
queues.

Therefore, the "cfg_max_queues" should be also set to zero when
dev_configure fails.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix VLAN strip log

When the current VLAN stripping is set, the log print always prompts
that the enabling fails, bug if may actually be the disabling failure.

Fixes: 411d23b9eafb ("net/hns3: support VLAN")
Cc: stable@dpdk.org
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix fake queue rollback

When the device supports independent Rx/Tx queues, fake queues do not need
to be created in unequal Rx/Tx queues case. However, dev_configure fails
to be executed on the device supported independent Rx/Tx queues, the
current rollback code logic contains the fake queue. As a result, the fake
queue is created. When dev_configure is successfully called again, these
fake queues still exists and are configured to the hardware.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix delay for waiting to stop Rx/Tx

When the primary process executes dev_stop or is being reset, the packet
sending and receiving functions is changed. In this moment, the primary
process requests secondary processes to change their Rx/Tx functions, and
delays a period of time in case of crashes when queues are still in use.
The delay time depends on the number of queues actually used, instead of
the maximum number of queues supported by the device.

Fixes: 23d4b61fee5d ("net/hns3: support multiple process")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: increase VF reset retry maximum

When the device is very busy, VF reset may have to be
retried many times to succeed, leading to the current
max reset fail retry count not enough.

Modify max reset fail retry count to 30 to enhance
the reliability of reset function.

Fixes: 2790c6464725 ("net/hns3: support device reset")
Cc: stable@dpdk.org
Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

eal: handle compressed firmware

Introduce an internal firmware loading helper to remove code duplication
in our drivers and handle xz compressed firmware by calling libarchive.

This helper tries to look for .xz suffixes so that drivers are not aware
the firmware has been compressed.

libarchive is set as an optional dependency: without libarchive, a
runtime warning is emitted so that users know there is a compressed
firmware.

Windows implementation is left as an empty stub.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Igor Russkikh <irusskikh@marvell.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Tested-by: Haiyue Wang <haiyue.wang@intel.com>

net/ice: factorize firmware loading

Both "normal" and "dcf" inits have their copy of some firmware loading
code.

The DSN query is moved in specific parts for the "normal" and "dcf" init.

A common helper ice_load_pkg is then introduced and takes an adapter
pointer as its main input.

This helper takes care of finding the right firmware file and loading
it.
The adapter active_pkg_type field is set by this helper.

The ice_access macro is removed from the osdep.h header: osdep.h should
only hosts wrappers for base driver code.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>

telemetry: add extra log message on socket bind failure

If the library fails to create the needed socket, add an additional
check to report if the error is due to a missing DPDK runtime dir.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Ciara Power <ciara.power@intel.com>

eal: create runtime dir even when shared data is not used

When multi-process is not wanted and DPDK is run with the "no-shconf"
flag, the telemetry library still needs a runtime directory to place the
unix socket for telemetry connections. Therefore, rather than not
creating the directory when this flag is set, we can change the code to
attempt the creation anyway, but not error out if it fails. If it
succeeds, then telemetry will be available, but if it fails, the rest of
DPDK will run without telemetry. This ensures that the "in-memory" flag
will allow DPDK to run even if the whole filesystem is read-only, for
example.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

drivers/net: fix memzone allocations for DMA memory

Caught by code review.

Using a random name for memzone allocations can result in init failures
in the unlikely case that a name collision occurs.
Use a simple sequential generator on 64 bits.

Fixes: 3f50f072ff06 ("i40e: fix memzone freeing")
Fixes: 22b123a36d07 ("net/avf: initialize PMD")
Fixes: 5f0978e96220 ("net/ice/base: add OS specific implementation")
Fixes: 737f30e1c3ab ("net/hns3: support command interface with firmware")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>

net/ice: fix build on RHEL 7

This patch fixed the unmatched integer type issue in the comparison,
which causing compilation failure on RHEL.

drivers/net/ice/ice_dcf_sched.c:353:15: error:
comparison between signed and unsigned integer expressions
if (node_id > tc_nb * hw->num_vfs) {
^

CI reported the failure:
http://mails.dpdk.org/archives/test-report/2021-July/201905.html

Fixes: 3a6bfc37eaf4 ("net/ice: support QoS config VF bandwidth in DCF")

Signed-off-by: Ting Xu <ting.xu@intel.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>

net/sfc: fix outer match in MAE backend

The current code doesn't use match on invalid outer rule ID
in action rules of non-encap. flows. This is wrong and must
be fixed. Do that and explain correct behaviour in the code.

Fixes: dadff137931c ("net/sfc: support encap flow items in transfer rules")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: check ID overflow in action port ID

The container in the action configuration is U32,
but the ID is U16, and overflow check is missing.

Fixes: 1fb65e4dae8a ("net/sfc: support flow action port ID in transfer rules")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support VLAN presence match in transfer rules

Take into account VLAN presence fields in items ETH and VLAN.

Provided that the item ETH does not match on the EtherType,
the pattern behaviour will be as follows:

- ETH (mask->has_vlan = 0) | IPv4 = match both tagged and untagged;
- ETH (mask->has_vlan = 1) | IPv4 = match as per spec->has_vlan;
- ETH (mask->has_vlan = 0) | VLAN | IPv4 = match only tagged.

Similar logic applies to double tagging.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE VLAN presence match bits

Introduce necessary infrastructure for these fields to
be set, validated and compared during class comparison.
Enumeration and mappings envisaged are MCDI-compatible.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>

common/sfc_efx/base: update MCDI headers

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>

doc: add transfer rules VXLAN actions in sfc guide

Actions VXLAN_DECAP and VXLAN_ENCAP need to be listed
among actions supported for transfer flows.

Fixes: 6ab6c40d1e83 ("net/sfc: support action VXLAN decap in transfer rules")
Fixes: 1bbd1ec2348a ("net/sfc: support action VXLAN encap in MAE backend")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

examples/l3fwd: remove useless reloads in EM main loop

This patch aligns the l3fwd EM code with the changes made to LPM in
commit 74fb854a3de6 ("examples/l3fwd: remove useless reloads in LPM
main loop").
This change ensures the compiler knows that the lcore config variables
are constant values and the compiler will then optimize the code
accordingly.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

examples/l3fwd: remove useless reloads in FIB main loop

This patch aligns the l3fwd FIB code with the changes made to LPM in
commit 74fb854a3de6 ("examples/l3fwd: remove useless reloads in LPM
main loop").
This change ensures the compiler knows that the lcore config variables
are constant values and the compiler will then optimize the code
accordingly.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

app/eventdev: add option to enable per port pool

Add option to configure unique mempool for each ethernet device
port. The new option available with `pipeline_atq` and
`pipeline_queue` tests.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

event/octeontx2: configure aura backpressure

In poll mode driver of octeontx2 the RQ is connected to a CQ and it is
responsible for asserting backpressure to the CGX channel.
When event eth Rx adapter is configured, the RQ is connected to a event
queue, to enable backpressure we need to configure AURA assigned to a
given RQ to backpressure CGX channel.
Event device expects unique AURA to be configured per ethernet device.
If multiple RQ from different ethernet devices use the same AURA,
the backpressure will be disabled, application can override this
using devargs:

-a 0002:0e:00.0,force_rx_bp=1

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

mempool/octeontx2: fix shift calculation

Shift is used to generate an 8-bit saturate value from the current
aura used count. The shift value should be derived from the log2 of
block count if it is greater than 256 else the shift should be 0.

Fixes: 7bcc47cbe2fa ("mempool/octeontx2: add mempool alloc op")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/dsw: flag adapters capabilities

Set the appropriate capability flags for the RX, crypto and timer
eventdev adapters to use.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Tested-by: Heng Wang <heng.wang@ericsson.com>

event/cnxk: fix clang build on Arm

clang-10 build issue log:
drivers/event/cnxk/cnxk_tim_worker.h:372:23:
warning: value size does not match register size
specified by the constraint and modifier [-Wasm-operand-widths]
                             : [rem] "=&r"(rem)
                                           ^
cnxk/cnxk_tim_worker.h:365:17: note: use constraint modifier "w"
                             "ldxr %[rem], [%[crem]]  \n"
                                             ^~~~~~
                                             %w[rem]

Changed variable type to match register size, which placates clang.

Fixes: 300b796262a1 ("event/cnxk: add timer arm routine")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

app/eventdev: remove unnecessary barrier from order test

For "order_launch_lcores" function, wmb after that the main lcore
updates the variable "t->err", which represents the end of the test
signal, is unnecessary. Because after the main lcore updates this
signal variable, it will jump out of the launch function loop, and wait
other lcores stop or return error in the main function(evt_main.c).
During this time, there is no storing operation and thus no need for
wmb.

Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

vdpa/mlx5: fix TSO offload without checksum

Packet was corrupted when TSO requested without CSUM update.

Enables CSUM automatically if only TSO requested.

Fixes: 2aa8444b0084 ("vdpa/mlx5: support stateless offloads")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: use DPDK allocations for in-flight data

Inflight metadata are allocated using glibc's calloc.
This patch converts them to rte_zmalloc_socket to take
care of the NUMA affinity.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: allocate all data on same node as virtqueue

This patch saves the NUMA node the virtqueue is allocated
on at init time, in order to allocate all other data on the
same node.

While most of the data are allocated before numa_realloc()
is called and so the data will be reallocated properly, some
data like the log cache are most likely allocated after.

For the virtio device metadata, we decide to allocate them
on the same node as the VQ 0.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: improve NUMA reallocation

This patch improves the numa_realloc() function by making use
of rte_realloc_socket(), which takes care of the memory copy
and freeing of the old data.

Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix NUMA reallocation with multi-queue

Since the Vhost-user device initialization has been reworked,
enabling the application to start using the device as soon as
the first queue pair is ready, NUMA reallocation no more
happened on queue pairs other than the first one since
numa_realloc() was returning early if the device was running.

This patch fixes this issue by reallocating the device metadata
only if the device is running. For the virtqueues, a vring state
change notification is sent to notify the application of its
disablement. Since the callback is supposed to be blocking, it
is safe to reallocate it afterwards.

Fixes: d0fcc38f5fa4 ("vhost: improve device readiness notifications")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing cache logging NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the log cache on the new NUMA node was
not done. This patch fixes this by reallocating it if it has
been allocated already, which means a live-migration is
on-going.

Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing guest pages table NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the guest pages table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: e246896178e6 ("vhost: get guest/host physical address mappings")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing memory table NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the Vhost memory table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
Cc: stable@dpdk.org
Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

net/ice: support default RSS for IP fragment packet

This patch adds default RSS support for IPv4 and IPv6 fragment packet.

Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix multi-process shared data

The rte_eth_devices array is not in share memory, it should not be
referenced by i40e_adapter which is shared by primary and secondary.
Any process set i40e_adapter->eth_dev will corrupt another process's
context.

The patch removed the field "eth_dev" from i40e_adapter.
Now, when the data paths try to access the rte_eth_dev_data instance,
they should replace adapter->eth_dev->data with adapter->pf.dev_data.

Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: improve vector Tx performance

For i40e vector Tx path, if tx_offload is set as FAST_FREE_MBUF mode,
no mbuf fast free operations are executed. To fix this, add mbuf fast
free mode for vector Tx path.

Furthermore, for i40e vector Tx path, if implement FAST_FREE_MBUF mode,
it means per-queue all mbufs come from the same mempool and have
refcnt = 1. Thus we can use bulk free of the buffers when mbuf fast free
mode is enabled.

For vector path in arm platform:
In n1sdp, performance is improved by 18.4%;
In thunderx2, performance is improved by 23%.

For vector path in x86 platform:
No performance changes.

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

net/i40e: improve scalar Tx performance

For i40e scalar Tx path, if implement FAST_FREE_MBUF mode, it means
per-queue all mbufs come from the same mempool and have refcnt = 1.

Thus we can use bulk free of the buffers when mbuf fast free mode is
enabled.

Following are the test results with this patch:

MRR L3FWD Test:
two ports & bi-directional flows & one core
RX API: i40e_recv_pkts_bulk_alloc
TX API: i40e_xmit_pkts_simple
ring_descs_size = 1024;
Ring_I40E_TX_MAX_FREE_SZ = 64;
tx_rs_thresh = I40E_DEFAULT_TX_RSBIT_THRESH = 32;
tx_free_thresh = I40E_DEFAULT_TX_FREE_THRESH = 32;

For scalar path in arm platform with default 'tx_rs_thresh':
In n1sdp, performance is improved by 7.9%;
In thunderx2, performance is improved by 7.6%.

For scalar path in x86 platform with default 'tx_rs_thresh':
performance is improved by 4.7%.

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>

net/ice: add option for setting HW debug mask

The HW debug mask is always zero, so user can't enable the related debug
function like ICE_DBG_XXX etc, add the devarg 'hw_debug_mask' to set the
debug mask log output at runtime.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix flow director input set conflict

Currently, there'll be conflict error when running
the following commands:
1. flow create 0 ingress
     pattern eth / ipv4 / udp src is 32 / end
     actions queue index 2 / end
2. flow destroy 0 rule 0
3. flow create 0 ingress
     pattern eth / ipv4 / udp dst is 32 / end
     actions queue index 2 / end

This patch fixes the input set conflict issue.

Fixes: 42044b69c67d ("net/i40e: support input set selection for FDIR")
Fixes: 4a072ad43442 ("net/i40e: fix flow director config after flow validate")
Cc: stable@dpdk.org
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Tested-by: Lingli Chen <linglix.chen@intel.com>

net/iavf: check Tx packet with correct UP and queue

Add check in the Tx packet preparation function, to guarantee that the
packet with specific user priority is distributed to the correct Tx
queue according to the configured Tx queue TC mapping.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: query QoS capabilities and set queue TC mapping

This patch added the support for VF to config the ETS-based Tx QoS,
including querying current QoS configuration from PF and config queue TC
mapping. PF QoS is configured in advance and the queried info is
provided to the user for future usage. VF queues are mapped to different
TCs in PF through virtchnl.

Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support QoS config VF bandwidth in DCF

This patch supports the ETS-based QoS configuration. It enables the DCF
to configure bandwidth limits for each VF VSI of different TCs. A
hierarchy scheduler tree is built with port, TC and VSI nodes.

Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support DCF link status event handling

When link status changes, DCF will receive virtchnl PF event message.
Add support to handle this event, change link status and update link
info.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: support DCF query port ETS adminq

In the adminq command query port ETS function, the root node teid is
needed. However, for DCF, the root node is not initialized, which will
cause error when we refer to the variable. In this patch, we will check
whether the root node is available or not first.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS
configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS
configuration. If VF and PF both have this flag, then the ETS-based QoS
offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max
bandwidth for each VF per enabled TCs. To make the VSI node bandwidth
configuration work, DCF also needs to configure TC node bandwidth
directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such
as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The
configuration is previously set by DCB and DCF, and now is the potential
QoS capability of VF. VF can take it as reference to configure queue TC
mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and
Rx queues. Queues mapping to one TC should be continuous and all
allocated queues should be mapped.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: add AVX2 offload Rx

Add a specific path for RX AVX2.
In this path, support the HW offload features, like,
checksum, VLAN stripping, RSS hash.
This path is chosen automatically according to the
configuration.

'inline' is used, then the duplicate code is generated
by the compiler.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Leyi Rong <leyi.rong@intel.com>

net/ice: add AVX2 offload Tx

Add a specific path for TX AVX2.
In this path, support the HW offload features, like,
checksum insertion, VLAN insertion.
This path is chosen automatically according to the
configuration.

'inline' is used, then the duplicate code is generated
by the compiler.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Leyi Rong <leyi.rong@intel.com>

net/ice/base: fix first profile mask

Since each PF does not share the same structure space, the first
mask value should start at 0 instead of hw->pf_id * per_pf to avoid
address overflow. Otherwise, address space will overlap when
masks.first + masks.count > ICE_PROF_MASK_COUNT, and it may lead to
unexpected variable assignment, which causes segmentation fault.

Fixes: 9467486f179f ("net/ice/base: enable masking for RSS and FD field vectors")
Cc: stable@dpdk.org
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix L2 payload RSS mask input set

Allow VLAN tag being added to L2 payload packet type RSS input set.

Fixes: ef4c16fd9148 ("net/i40e: refactor RSS flow")
Cc: stable@dpdk.org
Signed-off-by: Alvin Zhang <alvinx.zhang@intel.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>

net/i40e: fix available RSS hash function

i40e can support following rss hash function types: default/toeplitz,
symmetric toeplitz, and simple_xor. However, when filter engine parses
pattern action, it only supports symmetric toeplitz & default.

Add simple_xor and toeplitz hash functions support when parsing pattern
action.

Fixes: ef4c16fd9148 ("net/i40e: refactor RSS flow")
Cc: stable@dpdk.org
Signed-off-by: Steve Yang <stevex.yang@intel.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>

net/ice: fix overflow in maximum packet length config

The len variable, used in the computation of max_pkt_len could overflow,
if used to store the result of the following computation:

ICE_SUPPORT_CHAIN_NUM * rxq->rx_buf_len

Since, we could define the mbuf size to have a large value (i.e 13312),
and ICE_SUPPORT_CHAIN_NUM is defined as 5, the computation mentioned
above could potentially result in a value which might be bigger than
MAX_USHORT.

The result will be that Jumbo Frames will not work properly

Fixes: 1b009275e2c8 ("net/ice: add Rx queue init in DCF")
Cc: stable@dpdk.org
Signed-off-by: Tudor Cornea <tudor.cornea@keysight.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix data path selection in secondary process

The flag use_avx2 and use_avx512 are defined as local variables, they
will not be aware by the secondary process, then wrong data path is
selected. Fix the issue by moving them into struct i40e_adapter.

Fixes: 6ada10deac66 ("net/i40e: remove devarg use-latest-supported-vec")
Fixes: e6a6a138919f ("net/i40e: add AVX512 vector path")
Cc: stable@dpdk.org
Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: support RSS for GRE tunnel packet

Support AVF RSS for inner header of GRE tunnel packet. It supports
RSS based on fields inner IP src + dst address and TCP/UDP src + dst
port.

Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

common/iavf: add GRE header type

Add a virtchnl protocol header type to support AVF FDIR and RSS for GRE.

Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: support flow pattern for GRE

Add GRE pattern support for AVF FDIR and RSS.

Patterns are listed below:
  1. eth/ipv4/gre/ipv4
  2. eth/ipv4/gre/ipv6
  3. eth/ipv6/gre/ipv4
  4. eth/ipv6/gre/ipv6
  5. eth/ipv4/gre/ipv4/tcp
  6. eth/ipv4/gre/ipv6/tcp
  7. eth/ipv4/gre/ipv4/udp
  8. eth/ipv4/gre/ipv6/udp
  9. eth/ipv6/gre/ipv4/tcp
  10. eth/ipv6/gre/ipv6/tcp
  11. eth/ipv6/gre/ipv4/udp
  12. eth/ipv6/gre/ipv6/udp

Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support ESP/NATT flow director to match outer IP

This patch adds IPV4/IPV6 SRC/DST input set for ESP/NAT_T_ESP to
support outer IP match.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: support ESP flow director to match outer IP

Enable FDIR ESP for matching outer IPV4/IPV6 SRC/DST field.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/iavf: support ESP flow director to match outer IP

This patch adds IPV4/IPV6 SRC/DST input set for ESP to support
outer IP match.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/octeontx_ep: fix null pointer access

mz is known to be NULL. Do not use it to print a memzone name.

Fixes: 242e18c056890 ("net/octeontx_ep: add Rx queue setup and release")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>

net/cnxk: support multi-segment vector Tx

Add multi segment Tx vector routine.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: enable TSO processing in vector Tx

Enable TSO offload in vector Tx burst function.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: enable PTP processing in vector Tx

Enable PTP offload in vector Tx burst function. Since, we can
no-longer use a single LMT line for burst of 4, split the LMT
into two and transmit twice.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: enable VLAN processing in vector Tx

Enable VLAN offload in vector Tx burst function.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: enable PTP processing in vector Rx

Enable PTP offload in vector Rx burst function, use vector path
for processing mbufs and finally switch to scalar when extracting
timestamp.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/cnxk: support multi-segment vector Rx

Add multi-segment Rx vector routine, form the primary mbufs using
vector path switch to scalar path when extracting segments.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/octeontx2: support link status

Adding a new callback for reading the link status. PF can read it's
link status and can forward the same to VF once it comes up.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/octeontx2: send link event to VF

Currently link event is only sent to the PF by AF as soon as it comes
up, or in case of any physical change in link. PF will broadcast
these link events to all its VFs as soon as it receives it.
But no event is sent when a new VF comes up, hence it will not have
the link status.
Adding support for sending link status to the VF once it comes up
successfully.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/octeontx2: use runtime LSO format indices

Currently LSO formats setup initially are expected to be
compile time constants and start from 0.

Change the logic in slow and fast path so that LSO format indexes
are only determined runtime.

Fixes: 3b635472a998 ("net/octeontx2: support TSO offload")
Cc: stable@dpdk.org
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>

net/octeontx2: fix flow creation limit on CN98xx

CN96xx and CN98xx have 4096 and 16384 MCAM entries respectively.
Aligning the code with the same numbers.

Fixes: 092b3834185 ("net/octeontx2: add flow init and fini")
Cc: stable@dpdk.org
Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/octeontx2: support fast free for inline IPsec

Add support for DEV_TX_OFFLOAD_MBUF_FAST_FREE for inline IPsec path

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>

net/cnxk: support marking and VLAN tagging

This patch adds support for mark, flag, VLAN pop and
push flow actions.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>

net/cnxk: support multicast filter

Patch adds multicast filter support for cn9k and cn10k platforms.

CGX DMAC filter table(32 entries) is divided among all LMACs
connected to it i.e. if CGX has 4 LMACs then each LMAC can have
up to 8 filters. If CGX has 1 LMAC then it can have up to 32
filters.

Above mentioned filter table is used to install unicast and multicast
DMAC address filters. Unicast filters are installed via
rte_eth_dev_mac_addr_add API while multicast filters are installed
via rte_eth_dev_set_mc_addr_list API.

So in total, supported MAC filters are equal to DMAC filters plus
mcast filters.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support RETA and RSS hash

This patch will implement RETA and RSS hash apis. Also added
device argument to lock rx context.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

net/cnxk: support clock read

Patch implements read raw clock operation for cn9k and
cn10k.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support time read/write/adjust

Patch implements read/write/adjust time operations for
cn9k and cn10k platforms.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support Rx/Tx timestamp read

Patch implements Rx/Tx timestamp read operations for cn9k
and cn10k platforms.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support timesync operations

Patch implements timesync enable/disable operations for
cn9k and cn10k platforms.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support base PTP timesync

Base PTP timesync support is added for cn9k and cn10k platforms.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: get PTP status

Once PTP status is changed at H/W i.e. enable/disable then
it is propagated to user via registered callback.

So corresponding callback is registered to get PTP status.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support flow RSS

Added support for RSS action in rte flow code based on ROC.

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>

net/cnxk: support registers dump

With this patch implemented api to dump platform registers for
debug purposes.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

net/cnxk: support firmware version query

Add callback to get ethdev firmware version.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

net/cnxk: get flow operations

This patch adds flow ops get operation to enable rte_flow_ops.A

This patch also add support for flow dev dump API.
Every flow rule added will be dumped in the below format.

MCAM Index:1881
Interface :NIX-RX (0)
Priority  :1
NPC RX Action:0X00000000404001
ActionOp:NIX_RX_ACTIONOP_UCAST (1)
PF_FUNC: 0X400
RQ Index:0X004
Match Id:0000
Flow Key Alg:0
NPC RX VTAG Action:0X00000000008100
VTAG0:relptr:0
lid:0X1
type:0
Patterns:
NPC_PARSE_NIBBLE_CHAN:000
NPC_PARSE_NIBBLE_LA_LTYPE:LA_ETHER
NPC_PARSE_NIBBLE_LB_LTYPE:NONE
NPC_PARSE_NIBBLE_LC_LTYPE:LC_IP
NPC_PARSE_NIBBLE_LD_LTYPE:LD_TCP
NPC_PARSE_NIBBLE_LE_LTYPE:NONE
LA_ETHER, hdr offset:0, len:0X6, key offset:0X8,\
Data:0X4AE124FC7FFF, Mask:0XFFFFFFFFFFFF
LA_ETHER, hdr offset:0XC, len:0X2, key offset:0X4, Data:0XCA5A,\
Mask:0XFFFF
LC_IP, hdr offset:0XC, len:0X8, key offset:0X10,\
Data:0X0A01010300000000, Mask:0XFFFFFFFF00000000
LD_TCP, hdr offset:0, len:0X4, key offset:0X18, Data:0X03450000,\
Mask:0XFFFF0000
MCAM Raw Data :
DW0     :0000CA5A01202000
DW0_Mask:0000FFFF0FF0F000
DW1     :00004AE124FC7FFF
DW1_Mask:0000FFFFFFFFFFFF
DW2     :0A01010300000000
DW2_Mask:FFFFFFFF00000000
DW3     :0000000003450000
DW3_Mask:00000000FFFF0000
DW4     :0000000000000000
DW4_Mask:0000000000000000
DW5     :0000000000000000
DW5_Mask:0000000000000000
DW6     :0000000000000000
DW6_Mask:0000000000000000

Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
Reviewed-by: Kiran Kumar K <kirankumark@marvell.com>

net/cnxk: support flow API

Adding initial version of rte_flow support for cnxk family device.
Supported rte_flow ops are flow_validate, flow_create, flow_destroy,
flow_flush, flow_query, flow_isolate.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>

net/cnxk: add NPC configuration

Adding support to configure NPC on device initialization. This involves
reading the MKEX and initializing the necessary data.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>

net/cnxk: support pending Tx mbuf cleanup

Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are
accumalated as a pending mbuf.
Hence operation is NOP for cnxk platform.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support close and reset

Patch implements device close and reset operations for cn9k
and cn10k platforms.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>

net/cnxk: support queue infos query

Initial apis to get default queue information.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>

net/cnxk: support extended statistics

Initial implementation of xstats operations.

Signed-off-by: Satha Rao <skoteshwar@marvell.com>