crypto/cnxk: add symmetric capabilities

Add symmetric crypto capabilities for cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add digest support

Add support for digest support for various algorithms.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add KASUMI decryption

Add KASUMI decrypt support.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add KASUMI encryption

Add KASUMI encrypt support.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add ZUC and SNOW3G decryption

Add PDCP opcode which handles ZUC and SNOW3G.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add ZUC and SNOW3G encryption

Add PDCP opcode which handles ZUC and SNOW3G.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add flexi cipher decryption

Add flexi crypto cipher decrypt support in enqueue API. Flexi crypto
opcode covers a broad set of ciphers including variants of AES.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add flexi cipher encryption

Add flexi crypto cipher encrypt in enqueue API. Flexi crypto
opcode covers a broad set of ciphers including variants of AES.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add chained operation in session

Add support for chained operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add AEAD operation in session

Add support for AEAD operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add auth operation in session

Add support for auth operations in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add cipher operation in session

Add support for cipher operation in session.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add dequeue burst

Add dequeue_burst op in cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add enqueue burst

Add enqueue_burst op in cn9k & cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add session operations framework

Add session ops
- sym_session_get_size
- sym_session_configure
- sym_session_clear

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add queue pair operations

Add ops for
- queue_pair_setup()
- queue_pair_release()

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add device control operations

Add ops for
- dev_configure()
- dev_start()
- dev_stop()
- dev_close()
- dev_infos_get()

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add probe and remove

Add probe & remove for cn9k & cn10k crypto PMDs.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

crypto/cnxk: add driver skeleton

Add driver skeleton for crypto_cn9k & crypto_cn10k
PMDs leveraging cnxk common framework.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add SE set key for crypto

The set key functions are added in roc to be used
in crypto operations.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add IPsec common code

Add code that can be leveraged across
lookaside and inline IPsec on cn10k.

Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add EC vectors for asymmetric crypto

EC group static vectors are required for CPT asymmetric
crypto operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add FPM tables for asymmetric crypto

Add scalar FPM tables to be used for asymmetric crypto
operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add lmtline initialization

Add routine to initialize LMTLINE which facilitates instruction
submission to CPT. Add common macros required in the enqueue
operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add AE definitions for asymmetric crypto

Microcode AE opcodes support asymmetric operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add IE definitions for IPsec

Microcode IE opcodes support IPsec operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add SE definitions for symmetric crypto

Microcode SE opcodes support symmetric operations. Add defines
and structs defined by microcode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add inline IPsec configuration mailbox

Add mbox to configure inbound & outbound inline IPsec.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT LF flush

Add routine to flush context from CPT context processor cache.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT diagnostics

Add routines to fetch and dump CPT statistics and states.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Srujana Challa <schalla@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT LF configuration

Add routines to init & fini CPT LFs. CPT LFs are
queues to the hardware enabling instruction submissions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add mailbox to configure RXC

Add mailbox to configure timeouts and thresholds in
CPT RXC unit.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT get/set

Add APIs to set & get CPT device.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT configuration

Add routines to init, fini, configure & clear CPT device.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: update Rx inline IPsec mailbox format

Updates Rx inline IPSEC mailbox message format to sync
with latest CPT PF driver.

Signed-off-by: Srujana Challa <schalla@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

common/cnxk: add CPT hardware definitions

Add CPT hardware definitions. CPT is the hardware block on
cnxk family of processors, that can be used to offload
cryptographic operations.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Vidya Sagar Velumuri <vvelumuri@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

net/tap: define offload capabilities constants

Since the offload values are always the same, these can
just be data instead of code.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Keith Wiles <keith.wiles@intel.com>

net/tap: remove useless offload capability functions

Since these always return 0, they were doing nothing useful.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Keith Wiles <keith.wiles@intel.com>

app/testpmd: change port link speed without stopping all

When we use the following cmd to modify the link speed of specified
port: "port config <port_id> speed xxx duplex xxx", we have to stop
all ports. It's not necessary.

Fixes: 82113036e4e5 ("ethdev: redesign link speed config")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Xiaoyun Li <xiaoyun.li@intel.com>

net/cxgbe: remove basic queue statistics

Remove queue stats from basic stats because they're now available
via xstats API. Also remove RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support extended statistics

Add support to fetch port and queue stats via xstats API.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

doc: add link status event requirements in hns3 guide

Add link status event pre-conditions in the driver documentation.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/memif: improve memcpy performance

Replacing memcpy() with rte_memcpy() improved 64 byte packet
performance by 33% on a POWER9 system and by 10% on an x86_64
system.

Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

ethdev: fix doc of flow action

The struct rte_flow_action was missing from DPDK API documentation.

Fixes: 3850cf0c8c37 ("ethdev: add tunnel encap/decap actions")
Cc: stable@dpdk.org
Signed-off-by: Jan Viktorin <viktorin@cesnet.cz>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Aman Deep Singh <aman.deep.singh@intel.com>

net/hns3: fix traffic management

In a multi-TC scenario, if the length of packets destined for different
TCs is different, for example, 64B and 1500B packets destined for TC0 and
TC1 respectively. There is a problem that the bandwidth of the TC to which
large packets are sent is preempted by the TC to which small packets are
sent on the Kunpeng 920 network engine. As a result, the TC bandwidth
accuracy is inaccurate.

To solve this problem, this patch made the following adjustments:
1/ During initialization, firmware reports the capability bit indicating
whether the TM function is supported.
2/ The command word for configuring TC and port rate limiting is added,
instead of reusing the existing command word. And firmware configured
to the correct module.
3/ When the PF driver is loaded, firmware completes the default
initialization of the TC and port.

Fixes: c09c7847d892 ("net/hns3: support traffic management")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/nfp: fix multi-process PF probing

This patch creates a new function for handling PF probing of a secondary
process. A CPP handle is obtained for the CPP bridge service and the
service itself is also registered during secondary process
initialization. DPDK services aren't shared between processes so it is
not enough to only have the primary register the service if it is also
needed in a secondary process. This implies that both the primary and
secondary will have their own copy of the bridge service.

Fixes: 5e15e799d697 ("net/nfp: create separate entity for PF device")
Cc: stable@dpdk.org
Signed-off-by: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Signed-off-by: Simon Horman <simon.horman@corigine.com>

net/nfp: improve PF probing logic

When using rte_eth_dev_pci_generic_probe() during probing a
rte_eth_dev will be created with the name field corresponding to the PCI
address of the device. NFP4000/6000 devices only have a single PF (but
potentially multiple physical ports). This means that in a simple two
port example the rte_eth_devices[] array will be populated with two
devices: 0000:02:00.0 and 0000:02:00.0_port1. This is inconsistent and
not ideal. It will also cause issues when a secondary process tries to
attach to these ports.

This patch removes the use of rte_eth_dev_pci_generic_probe() and
allocates eth_dev's for each physical port during PF initialization,
giving them more consistent names.

Fixes: 5e15e799d697 ("net/nfp: create separate entity for PF device")
Cc: stable@dpdk.org
Signed-off-by: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Signed-off-by: Simon Horman <simon.horman@corigine.com>

app/testpmd: build on Windows

- Disable unsupported apps on Windows
- Enable building of testpmd on Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: disable flow director functions if unneeded

Function print_fdir_mask and print_fdir_flex_payload are only called
when either i40e or ixgbe presents. Extend existing #if defined to
include these two functions, to remove "unused function" compilation
warning.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: replace POSIX-specific code

- Make printf format OS independent
- Replace htons with RTE_BE16
- Replace POSIX specific inet_aton with OS independent inet_pton
- Replace sleep with rte_delay_us_sleep
- Replace random with rte_rand
- #ifndef mman related code for now
- Fix header inclusion
- Include rte_os_shim.h in testpmd.h
- Remove redundant headers

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

app/testpmd: fix type of FEC mode parsing output

Passing an uint32_t pointer to an enum pointer parameter causes
pointer-sign warning on Windows (converts between pointers to
integer types with different sign), since enum is implicitly
converted to int on Windows.

And the current enum pointer parameter of that function is actually
misleading and should be fixed as an uint32_t pointer parameter.

Fixes: b19da32e3151 ("app/testpmd: add FEC command")
Cc: stable@dpdk.org
Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

app/testpmd: resolve name collisions with Windows types

Resolve name collisions with Windows types.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add clock function

Add clock_gettime() on Windows in rte_os_shim.h.

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add device event stubs

Add device event stubs in eal_dev.c for Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

eal/windows: add macros required by testpmd

Add required macros by testpmd on Windows in rte_os_shim.h

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

lib: build testpmd dependencies on Windows

Enable building libraries that testpmd depends on for Windows

Signed-off-by: Jie Zhou <jizh@linux.microsoft.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

test/cksum: add L3/L4 checksum API test

Add a simple unit test for checksum API.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net: introduce functions to verify L4 checksums

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() and
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

Since these functions should be used to calculate the checksum to set in
a packet, introduce 2 new helpers for checksum verification. They return
0 if the checksum is valid in the packet.

Use this new helper in net/tap driver.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on TCP packets

Since commit d5df2ae0428a ("net: fix unneeded replacement of TCP
checksum 0"), the functions rte_ipv4_udptcp_cksum() or
rte_ipv6_udptcp_cksum() can return either 0x0000 or 0xffff when used to
verify a packet containing a valid checksum.

This new behavior broke the checksum verification in tap driver for TCP
packets: these packets are marked with PKT_RX_L4_CKSUM_BAD.

Fix this by checking the 2 possible values. A next commit will introduce
a checksum verification helper to simplify this a bit.

Fixes: d5df2ae0428a ("net: fix unneeded replacement of TCP checksum 0")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/tap: fix Rx checksum flags on IP options packets

When packet type is IPV4_EXT, the checksum is always marked as good in
the mbuf offload flags.

Since we know the header lengths, we can easily call
rte_ipv4_udptcp_cksum() in this case too.

Fixes: 8ae3023387e9 ("net/tap: add Rx/Tx checksum offload support")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/cxgbe: add MAC match-all to track promiscuous traffic

Chelsio T6 ASIC doesn't track Rx promisc traffic dropped due to lack
of Rx buffers and hence the imissed counter doesn't increment. Add
support for RAW MAC filter to insert a wildcard matchall rule at
the end of MPS TCAM to make MPS track the promisc traffic. This
rule will only be added/removed when promisc mode is turned on/off
on the interface.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: use C11-style compiler builtins for atomics

Replace rte_atomic ops with C11 atomics.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/sfc: fix outer L4 checksum Rx

Do not return "GOOD" flag if the status was bad.

Fixes: 9e6e7f479ae9b ("net/sfc: support Rx checksum offload for EF100")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

net/pfe: remove unnecessary null check

ethdev library does the check of dev before calling the link update.

Fixes: acd4818ea2a45 ("net/pfe: add link status update")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Acked-by: Gagandeep Singh <g.singh@nxp.com>

app/testpmd: move conntrack update after null check

Use out only after it was checked not to be null.

Fixes: 4d07cbefe3ba0 ("app/testpmd: add commands for conntrack")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Aman Deep Singh <aman.deep.singh@intel.com>

net/hns3: support Tx push quick doorbell for performance

Kunpeng 930 support Tx push mode which could improve performance.
It works like below:
 1. Add PCIe bar45 which support driver direct write the Tx descriptor
    or tail reg to it.
 2. Support three operations: a) direct write one Tx descriptor, b)
    direct write two Tx descriptors, c) direct write tail reg.
 3. The original tail reg located at bar23, the above bar45 tail reg
    could provide better bandwidth from the hardware perspective.

The hns3 driver only support direct write tail reg (also have the name
of quick doorbell), the detail:
Considering compatibility, firmware will report Tx push capa if the
hardware support it.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: remove unnecessary blank lines

Delete redundant blank lines to make:
1.Return value judgment follow the function call.
2.No blank lines at the end of a code block defined by braces.

Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix maximum queues on configuration failure

The "cfg_max_queues" maintains configured max queue numbers from user,
and is equal to the maximum of "nb_rx_queues" and "nb_tx_queues" in
"dev->data".

From the ethdev layer framework, "nb_rx/tx_queues" in "dev->data" were set
to zero in rte_eth_dev_configure() if ops.dev_configure in PMD fails to be
executed, In addition, if ops.dev_configure in HNS3 PMD failed, the fake
queues are also cleared on a device that does not support independent Rx/Tx
queues.

Therefore, the "cfg_max_queues" should be also set to zero when
dev_configure fails.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix VLAN strip log

When the current VLAN stripping is set, the log print always prompts
that the enabling fails, bug if may actually be the disabling failure.

Fixes: 411d23b9eafb ("net/hns3: support VLAN")
Cc: stable@dpdk.org
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix fake queue rollback

When the device supports independent Rx/Tx queues, fake queues do not need
to be created in unequal Rx/Tx queues case. However, dev_configure fails
to be executed on the device supported independent Rx/Tx queues, the
current rollback code logic contains the fake queue. As a result, the fake
queue is created. When dev_configure is successfully called again, these
fake queues still exists and are configured to the hardware.

Fixes: fa29fe45a7b4 ("net/hns3: support queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: fix delay for waiting to stop Rx/Tx

When the primary process executes dev_stop or is being reset, the packet
sending and receiving functions is changed. In this moment, the primary
process requests secondary processes to change their Rx/Tx functions, and
delays a period of time in case of crashes when queues are still in use.
The delay time depends on the number of queues actually used, instead of
the maximum number of queues supported by the device.

Fixes: 23d4b61fee5d ("net/hns3: support multiple process")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

net/hns3: increase VF reset retry maximum

When the device is very busy, VF reset may have to be
retried many times to succeed, leading to the current
max reset fail retry count not enough.

Modify max reset fail retry count to 30 to enhance
the reliability of reset function.

Fixes: 2790c6464725 ("net/hns3: support device reset")
Cc: stable@dpdk.org
Signed-off-by: Hongbo Zheng <zhenghongbo3@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>

eal: handle compressed firmware

Introduce an internal firmware loading helper to remove code duplication
in our drivers and handle xz compressed firmware by calling libarchive.

This helper tries to look for .xz suffixes so that drivers are not aware
the firmware has been compressed.

libarchive is set as an optional dependency: without libarchive, a
runtime warning is emitted so that users know there is a compressed
firmware.

Windows implementation is left as an empty stub.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Igor Russkikh <irusskikh@marvell.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Tested-by: Haiyue Wang <haiyue.wang@intel.com>

net/ice: factorize firmware loading

Both "normal" and "dcf" inits have their copy of some firmware loading
code.

The DSN query is moved in specific parts for the "normal" and "dcf" init.

A common helper ice_load_pkg is then introduced and takes an adapter
pointer as its main input.

This helper takes care of finding the right firmware file and loading
it.
The adapter active_pkg_type field is set by this helper.

The ice_access macro is removed from the osdep.h header: osdep.h should
only hosts wrappers for base driver code.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>

telemetry: add extra log message on socket bind failure

If the library fails to create the needed socket, add an additional
check to report if the error is due to a missing DPDK runtime dir.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Ciara Power <ciara.power@intel.com>

eal: create runtime dir even when shared data is not used

When multi-process is not wanted and DPDK is run with the "no-shconf"
flag, the telemetry library still needs a runtime directory to place the
unix socket for telemetry connections. Therefore, rather than not
creating the directory when this flag is set, we can change the code to
attempt the creation anyway, but not error out if it fails. If it
succeeds, then telemetry will be available, but if it fails, the rest of
DPDK will run without telemetry. This ensures that the "in-memory" flag
will allow DPDK to run even if the whole filesystem is read-only, for
example.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

drivers/net: fix memzone allocations for DMA memory

Caught by code review.

Using a random name for memzone allocations can result in init failures
in the unlikely case that a name collision occurs.
Use a simple sequential generator on 64 bits.

Fixes: 3f50f072ff06 ("i40e: fix memzone freeing")
Fixes: 22b123a36d07 ("net/avf: initialize PMD")
Fixes: 5f0978e96220 ("net/ice/base: add OS specific implementation")
Fixes: 737f30e1c3ab ("net/hns3: support command interface with firmware")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>

net/ice: fix build on RHEL 7

This patch fixed the unmatched integer type issue in the comparison,
which causing compilation failure on RHEL.

drivers/net/ice/ice_dcf_sched.c:353:15: error:
comparison between signed and unsigned integer expressions
   if (node_id > tc_nb * hw->num_vfs) {
               ^

CI reported the failure:
http://mails.dpdk.org/archives/test-report/2021-July/201905.html

Fixes: 3a6bfc37eaf4 ("net/ice: support QoS config VF bandwidth in DCF")

Signed-off-by: Ting Xu <ting.xu@intel.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>

net/sfc: fix outer match in MAE backend

The current code doesn't use match on invalid outer rule ID
in action rules of non-encap. flows. This is wrong and must
be fixed. Do that and explain correct behaviour in the code.

Fixes: dadff137931c ("net/sfc: support encap flow items in transfer rules")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: check ID overflow in action port ID

The container in the action configuration is U32,
but the ID is U16, and overflow check is missing.

Fixes: 1fb65e4dae8a ("net/sfc: support flow action port ID in transfer rules")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

net/sfc: support VLAN presence match in transfer rules

Take into account VLAN presence fields in items ETH and VLAN.

Provided that the item ETH does not match on the EtherType,
the pattern behaviour will be as follows:

- ETH (mask->has_vlan = 0) | IPv4        = match both tagged and untagged;
- ETH (mask->has_vlan = 1) | IPv4        = match as per spec->has_vlan;
- ETH (mask->has_vlan = 0) | VLAN | IPv4 = match only tagged.

Similar logic applies to double tagging.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

common/sfc_efx/base: add MAE VLAN presence match bits

Introduce necessary infrastructure for these fields to
be set, validated and compared during class comparison.
Enumeration and mappings envisaged are MCDI-compatible.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>

common/sfc_efx/base: update MCDI headers

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>

doc: add transfer rules VXLAN actions in sfc guide

Actions VXLAN_DECAP and VXLAN_ENCAP need to be listed
among actions supported for transfer flows.

Fixes: 6ab6c40d1e83 ("net/sfc: support action VXLAN decap in transfer rules")
Fixes: 1bbd1ec2348a ("net/sfc: support action VXLAN encap in MAE backend")
Cc: stable@dpdk.org
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>

examples/l3fwd: remove useless reloads in EM main loop

This patch aligns the l3fwd EM code with the changes made to LPM in
commit 74fb854a3de6 ("examples/l3fwd: remove useless reloads in LPM
main loop").
This change ensures the compiler knows that the lcore config variables
are constant values and the compiler will then optimize the code
accordingly.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

examples/l3fwd: remove useless reloads in FIB main loop

This patch aligns the l3fwd FIB code with the changes made to LPM in
commit 74fb854a3de6 ("examples/l3fwd: remove useless reloads in LPM
main loop").
This change ensures the compiler knows that the lcore config variables
are constant values and the compiler will then optimize the code
accordingly.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

app/eventdev: add option to enable per port pool

Add option to configure unique mempool for each ethernet device
port. The new option available with `pipeline_atq` and
`pipeline_queue` tests.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

event/octeontx2: configure aura backpressure

In poll mode driver of octeontx2 the RQ is connected to a CQ and it is
responsible for asserting backpressure to the CGX channel.
When event eth Rx adapter is configured, the RQ is connected to a event
queue, to enable backpressure we need to configure AURA assigned to a
given RQ to backpressure CGX channel.
Event device expects unique AURA to be configured per ethernet device.
If multiple RQ from different ethernet devices use the same AURA,
the backpressure will be disabled, application can override this
using devargs:

-a 0002:0e:00.0,force_rx_bp=1

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

mempool/octeontx2: fix shift calculation

Shift is used to generate an 8-bit saturate value from the current
aura used count. The shift value should be derived from the log2 of
block count if it is greater than 256 else the shift should be 0.

Fixes: 7bcc47cbe2fa ("mempool/octeontx2: add mempool alloc op")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

event/dsw: flag adapters capabilities

Set the appropriate capability flags for the RX, crypto and timer
eventdev adapters to use.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Tested-by: Heng Wang <heng.wang@ericsson.com>

event/cnxk: fix clang build on Arm

clang-10 build issue log:
drivers/event/cnxk/cnxk_tim_worker.h:372:23:
warning: value size does not match register size
specified by the constraint and modifier [-Wasm-operand-widths]
                             : [rem] "=&r"(rem)
                                           ^
cnxk/cnxk_tim_worker.h:365:17: note: use constraint modifier "w"
                             "ldxr %[rem], [%[crem]]  \n"
                                             ^~~~~~
                                             %w[rem]

Changed variable type to match register size, which placates clang.

Fixes: 300b796262a1 ("event/cnxk: add timer arm routine")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

app/eventdev: remove unnecessary barrier from order test

For "order_launch_lcores" function, wmb after that the main lcore
updates the variable "t->err", which represents the end of the test
signal, is unnecessary. Because after the main lcore updates this
signal variable, it will jump out of the launch function loop, and wait
other lcores stop or return error in the main function(evt_main.c).
During this time, there is no storing operation and thus no need for
wmb.

Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

vdpa/mlx5: fix TSO offload without checksum

Packet was corrupted when TSO requested without CSUM update.

Enables CSUM automatically if only TSO requested.

Fixes: 2aa8444b0084 ("vdpa/mlx5: support stateless offloads")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: use DPDK allocations for in-flight data

Inflight metadata are allocated using glibc's calloc.
This patch converts them to rte_zmalloc_socket to take
care of the NUMA affinity.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: allocate all data on same node as virtqueue

This patch saves the NUMA node the virtqueue is allocated
on at init time, in order to allocate all other data on the
same node.

While most of the data are allocated before numa_realloc()
is called and so the data will be reallocated properly, some
data like the log cache are most likely allocated after.

For the virtio device metadata, we decide to allocate them
on the same node as the VQ 0.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: improve NUMA reallocation

This patch improves the numa_realloc() function by making use
of rte_realloc_socket(), which takes care of the memory copy
and freeing of the old data.

Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix NUMA reallocation with multi-queue

Since the Vhost-user device initialization has been reworked,
enabling the application to start using the device as soon as
the first queue pair is ready, NUMA reallocation no more
happened on queue pairs other than the first one since
numa_realloc() was returning early if the device was running.

This patch fixes this issue by reallocating the device metadata
only if the device is running. For the virtqueues, a vring state
change notification is sent to notify the application of its
disablement. Since the callback is supposed to be blocking, it
is safe to reallocate it afterwards.

Fixes: d0fcc38f5fa4 ("vhost: improve device readiness notifications")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing cache logging NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the log cache on the new NUMA node was
not done. This patch fixes this by reallocating it if it has
been allocated already, which means a live-migration is
on-going.

Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing guest pages table NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the guest pages table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: e246896178e6 ("vhost: get guest/host physical address mappings")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

vhost: fix missing memory table NUMA realloc

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the Vhost memory table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
Cc: stable@dpdk.org
Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>