vhost: prefetch descriptor after the read barrier

This patch moves the prefetch after the available index
is read to avoid prefetching a descriptor not available yet.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: enforce desc flags and content read ordering

A read barrier is required to ensure that the ordering between
descriptor's flags and content reads is enforced.

1. read flags = desc->flags
if (flags & AVAIL_BIT)
2.   read desc->id

There is a control dependency between steps 1 and step 2.
2 could be speculatively executed before 1, which could result
in 'id' to not be updated yet.

Fixes: 2f3225a7d69b ("vhost: add vector filling support for packed ring")
Cc: stable@dpdk.org
Reported-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: enforce avail index and desc read ordering

A read barrier is required to ensure the ordering between
available index and the descriptor reads is enforced.

1. read avail_head = avail->idx
2. read cur_idx = last_avail_idx
if (cur_idx != avail_head) {
    3. read idx = avail->ring[cur_idx]
    4. read desc[idx]
}

There is a control dependency between step 1 and steps 3 & 4,
3 could be speculatively executed before 1, which could result
in 'idx' to not being updated yet.

Fixes: 4796ad63ba1f ("examples/vhost: import userspace vhost application")
Cc: stable@dpdk.org
Reported-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

net/vhost: fix double free of MAC address

The common data freeing has been moved to rte_eth_dev_release_port(),
so freeing mac_addrs like this in eth_dev_close() is unnecessary and
will cause double free.

Fixes: e16adf08e54d ("ethdev: free all common data when releasing port")
Cc: stable@dpdk.org
Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

net/cxgbe: implement reset hit counters for offloaded flows

Implement logic to reset hit counters for offloaded flows.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/failsafe: remove unnecessary debug messages

The failsafe driver spews messages every time it is polling
making debugging other things more difficult. Just remove it.

Example:
net_failsafe: Hot-plug mutex was locked by thread 140182066431744 by the hot-plug alarm
net_failsafe: Hot-plug mutex was locked by thread 140182066431744 by the hot-plug alarm
...

Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/tap: add buffer overflow checks before checksum

The checksum calculation APIs take only the packet headers pointers as
parameters, so they assume that the lengths reported in those headers
are correct. However, a malicious packet could claim to be far larger
than it is, so we need to check the header lengths in the driver before
calling the checksum API.

A better fix would be to allow the lengths to be passed into the API
function, but that would be an API break, so fixing in TAP driver for
now.

Fixes: 8ae3023387e9 ("net/tap: add Rx/Tx checksum offload support")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>

net: fix underflow for checksum of invalid IPv4 packets

If we receive a packet with an invalid IP header, where the total packet
length is reported as less than the IP header length, we would end up
getting an underflow in the length subtraction.

This could cause us to checksum e.g. 4GB of data in the case where the
result of the subtraction was -1.

We fix this by having the function return 0 - an invalid sum - when
the length is less than the header length.

Fixes: af75078fece3 ("first public release")
Fixes: 6006818cfb26 ("net: new checksum functions")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

net/mlx5: fix Multi-Packet RQ mempool free

When MPRQ mempool is freed, the pointer stored in priv structure must be
reset to null. Otherwise, the mempool can be freed again if the port is
restarted.

Fixes: 7d6bf6b866b8 ("net/mlx5: add Multi-Packet Rx support")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

doc: update queue number per VF for i40e

Updated the doc and release notes on the support for requesting more
queues.

Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: support PF respond VF request more queues

This patch respond the VIRTCHNL_OP_REQUEST_QUEUES msg from VF, and
process to allocated more queues for the requested VF. If successful,
PF will notify VF to reset. If unsuccessful, PF will send message to
inform VF.

Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: support VF request more queues

Before this patch, VF gets a default number of queues from the PF.
This patch enables VF to request a different number. When VF configures
more queues, it will send VIRTCHNL_OP_REQUEST_QUEUES to PF to request
more queues, if success, PF will reset the VF.

User can run "port stop all", "port config port_id rxq/txq queue_num"
and "port start all" to reconfigure queue number.

Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: fix config name in comment

This patch fixes I40E RxTx module to use the proper config setting,
CONFIG_RTE_LIBRTE_I40E_INC_VECTOR.

Fixes: 9ed94e5bb04e ("i40e: add vector Rx")
Cc: stable@dpdk.org
Signed-off-by: Rami Rosen <ramirose@gmail.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/cxgbe: fix other misc build issues for Windows

Fix following build errors reported by Intel C++ compiler in Windows
build.

C:\> t4_hw.c(5105): warning #147: declaration is incompatible with
"int t4_bar2_sge_qregs(struct adapter *, unsigned int, unsigned int,
u64={uint64_t={unsigned __int64}} *, unsigned int *)"
(declared at line 524 of "..\..\..\..\drivers\net\cxgbe\base\common.h")
    int t4_bar2_sge_qregs(struct adapter *adapter, unsigned int qid,
        ^

C:\> cxgbe_filter.c(42): error : expected an expression
        n_user_filters = mult_frac(adap->tids.nftids,
                         ^

C:\> sge.c(400): error : expression must be a pointer to a complete
object type
                  (uint16_t)(RTE_PTR_ALIGN((char *)mbuf->buf_addr +
                             ^

Build Environment:
1. Target OS: Microsoft Windows Server 2016
2. Compiler: Intel C++ Compiler from Intel Parallel Studio XE 2019 [1]
3. Development Tools:
   3.1 Microsoft Visual Studio 2017 Professional
   3.2 Windows Software Development Kit (SDK) v10.0.17763
   3.3 Windows Driver Kit (WDK) v10.0.17763

[1] https://software.intel.com/en-us/parallel-studio-xe

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: define symbols only when not available

Define symbols only when they are not available.

This fixes following types of issues reported by Intel C++ compiler
in Windows build.

C:\> cxgbe_compat.h(154): warning #47: incompatible redefinition of
macro "min"
        #define min(a, b) RTE_MIN(a, b)
                ^

C:\> t4_hw.c(338): warning #266: function "bzero" declared implicitly
            bzero(p, 0, size);
            ^

C:\> t4_hw.c(5337): warning #266: function "htonl" declared implicitly
            rvc.op_to_viid = htonl(V_FW_CMD_OP(FW_RSS_VI_CONFIG_CMD) |
                             ^

C:\> sge.c(361): error : expected an expression
        struct sge_eth_rxq *rxq = container_of(q, struct sge_eth_rxq, fl);
                                  ^

C:\> sge.c(1350): error : identifier "caddr_t" is undefined
  static void inline_tx_mbuf(const struct sge_txq *q, caddr_t from,
                                                      ^
[...]

Build Environment:
1. Target OS: Microsoft Windows Server 2016
2. Compiler: Intel C++ Compiler from Intel Parallel Studio XE 2019 [1]
3. Development Tools:
   3.1 Microsoft Visual Studio 2017 Professional
   3.2 Windows Software Development Kit (SDK) v10.0.17763
   3.3 Windows Driver Kit (WDK) v10.0.17763

[1] https://software.intel.com/en-us/parallel-studio-xe

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: fix macros related to logs for Windows

Replace "args..." with "fmt, ..." and directly use __VA_ARGS__.

This fixes following errors reported by Intel C++ compiler in Windows
build.

C:\> cxgbe_compat.h(28): error : expected a ")"
        #define dev_printf(level, fmt, args...) \
                                       ^

C:\> cxgbe_compat.h(31): error : expected a ")"
        #define dev_err(x, args...) dev_printf(ERR, args)
                           ^
[...]

Build Environment:
1. Target OS: Microsoft Windows Server 2016
2. Compiler: Intel C++ Compiler from Intel Parallel Studio XE 2019 [1]
3. Development Tools:
   3.1 Microsoft Visual Studio 2017 Professional
   3.2 Windows Software Development Kit (SDK) v10.0.17763
   3.3 Windows Driver Kit (WDK) v10.0.17763

[1] https://software.intel.com/en-us/parallel-studio-xe

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: use relative paths for includes

The Intel C++ compiler is not able to locate the header files without
relative paths in Windows build. Following errors are seen for these
header files.

C:\> base\common.h(9): error : cannot open source file "cxgbe_compat.h"
        #include "cxgbe_compat.h"
                                 ^
[...]

Fix by explicitly stating header file location using relative paths.
Also, remove automatically including header files for Linux, to keep
it consistent across both OS.

Build Environment:
1. Target OS: Microsoft Windows Server 2016
2. Compiler: Intel C++ Compiler from Intel Parallel Studio XE 2019 [1]
3. Development Tools:
   3.1 Microsoft Visual Studio 2017 Professional
   3.2 Windows Software Development Kit (SDK) v10.0.17763
   3.3 Windows Driver Kit (WDK) v10.0.17763

[1] https://software.intel.com/en-us/parallel-studio-xe

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/ice: support descriptor ops

Add below ops,
rx_descriptor_status
tx_descriptor_status

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support statistics

Add below ops,
stats_get
stats_reset
xstats_get
xstats_get_names
xstats_reset

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Jeff Guo <jia.guo@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support advance Rx/Tx

Add Rx functions, scatter and bulk.
Add Tx function, simple.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support EEPROM information getting

Add below ops,
get_eeprom_length
get_eeprom

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support FW version getting

Add ops fw_version_get.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support Rx queue interruption

Add below ops,
rx_queue_intr_enable
rx_queue_intr_disable

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support RSS

Add below ops,
reta_update
reta_query
rss_hash_update
rss_hash_conf_get

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support VLAN ops

Add below ops,
ice_vlan_filter_set
ice_vlan_offload_set
ice_vlan_tpid_set
ice_vlan_pvid_set

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support MAC ops

Add below ops,
mac_addr_set
mac_addr_add
mac_addr_remove

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support MTU setting

Add ops mtu_set.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice: support basic Rx/Tx

Add basic RX & TX support.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support packet type getting

Add ops dev_supported_ptypes_get.

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support queue information getting

Add below ops,
rxq_info_get
txq_info_get
rx_queue_count

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support link update

Add ops link_update.
LSC interrupt is also enabled in this patch.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support getting device information

Add ops dev_infos_get.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support device and queue ops

Normally when starting/stopping the device the queue
should be started and stopped. Support them both in
this patch.

Below ops are added,
dev_configure
dev_start
dev_stop
dev_close
dev_reset
rx_queue_start
rx_queue_stop
tx_queue_start
tx_queue_stop
rx_queue_setup
rx_queue_release
tx_queue_setup
tx_queue_release

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: support device initialization

Update the documents too.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice/base: add OS specific implementation

Add some MACRO defination and small functions which
are specific for DPDK.
Add readme too.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add structures for Rx/Tx queues

Add the structures that define how the Rx/Tx queues
are used.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add protocol structures and defines

Add the structures and defines that define what
protocols the NIC can handle.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add various headers

Add various headers that define status codes and
basic defines for use in the code.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add common functions

Add code that multiple other features use.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add code to work with the NVM

Add code to read/write/query the NVM image.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add virtual switch code

Add code to handle the virtual switch within the NIC.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add basic Tx scheduler

Add code for the basic TX scheduler.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add control queue information

Add the structures for the control queues.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add device IDs for Intel E800 Series NICs

Add all the device IDs that represent the NIC.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add sideband queue info

Add the commands, error codes, and structures
for the sideband queue.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add admin queue structures and commands

Add the commands, error codes, and structures for
the admin queue.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add basic structures

Add the structures required by the NIC.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ice/base: add registers for Intel E800 Series NIC

Add the registers that comprise the Intel(R) 800
Series NIC. There is no functionality in this patch.

Signed-off-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/cxgbe: skip parsing match items with no spec

Skip parsing pattern match items that have no spec. This fixes NULL
dereference when accessing their non-existent spec.

Fixes: ee61f5113b17 ("net/cxgbe: parse and validate flows")
Cc: stable@dpdk.org
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: fix overlapping regions in TID table

Location of filter TID table should be after active TID table memory,
and not from the beginning of TID table memory. This fixes memory
corruption due to overlapping regions.

Fixes: 3a381a4116ed ("net/cxgbe: query firmware for HASH filter resources")
Cc: stable@dpdk.org
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/ena: update version to 2.0.0

The ENAv2 is introducing many new features, mainly the LLQ feature
(Low Latency Queue) which allows the device to process packets faster
and as a result, the latency is noticeably lower.

The second major feature is configurable depth of hw queues where Rx
and Tx can be reconfigured independently and maximum depth of Rx queue
is 8k.

The release also includes many bug fixes and minor new features, like
improved statistics counters and extended statistics.

Driver is still compatible with ENAv1 device.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>

docs: update ENA PMD documentation and features

Previously, the features list was indicating unsupported ENA PMD
features and were missing few ones, that were actually supported.

The features file was updated, so it is now reflecting current driver
state.

The documentation was updated with the more actual example and features,
especially ones which are ENA and not listed in the features file.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>

net/ena: update completion queue after cleanup

After Rx or Tx cleanup update completion queue head by calling
ena_com_update_dev_comp_head().

Fixes: 1daff5260ff8 ("net/ena: use unmasked head and tail")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: add new way of getting Rx drops

The Rx drops cannot be acquired using the older API. Now, it must be
read in keep alive message.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: expose extended stats

ENA PMD is having it's own custom statistics counters. They are exposed
to the application by using the xstats DPDK API.

The deprecated and unused statistics are removed, together with old API.

Signed-off-by: Solganik Alexander <sashas@lightbitslabs.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>

net/ena: add per-queue software counters stats

Those counters provide information regards sent/received bytes and
packets per queue.

Signed-off-by: Solganik Alexander <sashas@lightbitslabs.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>

net/ena: fix cleanup for out of order packets

When wrong req_id is detected some previous mbufs could be used for
receiving different segments of received packets. In such cases chained
mbufs will be twice returned to pool.

To prevent it chained mbuf is now freed just after error detection.

To simplify cleaning, pointers taken for Rx ring are set to NULL.

As after ena_rx_queue_release_bufs and ena_tx_queue_release_bufs queues
are not used updating of next_to_clean pointer is not necessary.

Fixes: c2034976673d ("net/ena: add Rx out of order completion")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: fix invalid reference to variable in union

Use empty_rx_reqs instead of empty_tx_reqs.
As those two variables are part of union this not cause
any failure, but for consistency should be changed.

Fixes: c2034976673d ("net/ena: add Rx out of order completion")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: add supported RSS offloads types

The PMD was not passing RSS offloads values although it was supporting
the RSS. To allow application to probe the PMD for RSS support, the
missing information was added.

Fixes: 1173fca25af9 ("ena: add polling-mode driver")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: adjust new line in log messages

Only PMD_*_LOG is adding new line character to the log message.
All printouts were adjusted for consistency.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: do not reconfigure queues on reset

Reset function should return the port to initial state, in which no Tx
and Rx queues are setup. Then application should reconfigure the queues.

According to DPDK documentation the rte_eth_dev_reset() itself is a
generic function which only does some hardware reset operations through
calling dev_unint() and dev_init().

ena_com_dev_reset which perform NIC registers reset should be called
during stop.

Fixes: 2081d5e2e92d ("net/ena: add reset routine")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: destroy queues if start failed

If start function fails, previously created queues have to be removed.

ena_queue_restart_all() and ena_queue_restart() are renamed to
ena_queue_start_all() and ena_queue_start().

ena_free_io_queues_all() is renamed to ena_queue_stop_all().

Fixes: df238f84c0a2 ("net/ena: recreate HW IO rings on start and stop")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: call additional doorbells if needed

Before sending next packet, check if calling doorbell is needed.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: increase maximum Rx ring size

Some of ENA devices supports 8k Rx rings. Maximum supported size is
received upon device initialization.
As ENA_DEFAULT_RING_SIZE_RX macro is upper limit, it needs to be
adjusted.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: support LLQv2

LLQ (Low Latency Queue) is the feature that allows pushing header
directly to the device through PCI before even DMA is triggered.
It reduces latency, because device can start preparing packet before
payload is sent through DMA.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>

net/ena: skip packet with wrong request id

When invalid req_id is received, the reset should be handled by the
application, as it is indicating invalid rings state, so further Rx
is not making any sense.

Fixes: c2034976673d ("net/ena: add Rx out of order completion")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: add HW queues depth setup

The device now allows driver to reconfigure Tx and Rx queues depth
independently. Moreover, maximum size for Tx and Rx can be different.
Those maximum values are received from the device.

After reset, previous ring configuration is restored.

If number of descriptor is set to RTE_ETH_DEV_FALLBACK_RX_RINGSIZE
or RTE_ETH_DEV_FALLBACK_TX_RINGSIZE, the maximum value is restored.

Remove checks, if provided number is not too big, as this is done in
generic functions (rte_eth_rx_queue_setup and rte_eth_tx_queue_setup).

Maximum number of segments is being set for Rx packets and provided to
ena_com_rx_pkt() for validation.

Unused definitions were removed.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: add reset reason in Rx error

Whenever the driver will receive too many descriptors from the device,
it should trigger the device reset with reset reason set to
ENA_REGS_RESET_TOO_MANY_RX_DESCS.

Fixes: 241da076b1f7 ("net/ena: adjust error checking and cleaning")
Cc: stable@dpdk.org
Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena: pass number of CPUs to the host info structure

The new ena_com allows the number of CPUs to be passed to the device in
the host info structure.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

net/ena/base: update communication layer for the ENAv2

ena_com is the communication layer provided by the vendor and common to
all ENA drivers.
This patch updates it to version from 2018.09.26.

It adds support for ENAv2 device together with LLQ feature, adds
doorbell optimization and reconfiguration of HW queues depth
independently.

The driver was adjusted to the new changes in the HAL.

Signed-off-by: Rafal Kozik <rk@semihalf.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>

app/testpmd: batch MAC swap for performance on x86

Do four packets macswap in same loop iterate to squeeze more
CPU cycles.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

app/testpmd: improve MAC swap performance for x86

The patch optimizes the mac swap operation by taking advantage
of SSE instructions, it only impacts x86 platform.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

app/testpmd: move MAC swap functions

Move macswap workload to dedicate function, so we can further enable
platform specific optimized version.

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>

doc: update ifc NIC document

Add the SW assisted VDPA live migration feature into NIC doc.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: support SW assisted VDPA live migration

In SW assisted live migration mode, driver will stop the device and
setup a mediated virtio ring to relay the communication between the
virtio driver and the VDPA device.

This data path intervention will allow SW to help on guest dirty page
logging for live migration.

This SW fallback is event driven relay thread, so when the network
throughput is low, this SW fallback will take little CPU resource, but
when the throughput goes up, the relay thread's CPU usage will goes up
accordingly.

User needs to take all the factors including CPU usage, guest perf
degradation, etc. into consideration when selecting the live migration
support mode.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: use vhost lib function for used ring logging

Vhost lib has already provided a helper for used ring logging, driver
could use it to reduce code.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: add LM mode parameter

This patch series enables a new method for live migration, i.e. software
assisted live migration. This patch provides a device argument for user
to choose the methold.

When "sw-live-migration=1", driver/device will do live migration with a
relay thread dealing with dirty page logging. Without this parameter,
device will do dirty page logging and there's no relay thread consuming
CPU resource.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: detect if VDPA mode is specified

If user wants the VF to be used in VDPA (vhost data path acceleration)
mode, then the user can add a "vdpa=1" parameter for the device.

So if driver does not find this option, it should quit and let the bus
continue the probe.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: store only registered device instance

If driver fails to register ifc VF device into vhost lib, then this
device should not be stored.

Fixes: a3f8150eac6d ("net/ifcvf: add ifcvf vDPA driver")
Cc: stable@dpdk.org
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/ifc: add probing error logs

Driver probe may fail for different causes, debug message is helpful for
debugging issue.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: provide helpers for virtio ring relay

This patch provides two helpers for vdpa device driver to perform a
relay between the guest virtio ring and a mediated virtio ring.

The available ring relay will synchronize the available entries, and
help to do desc validity checking.

The used ring relay will synchronize the used entries from mediated ring
to guest ring, and help to do dirty page logging for live migration.

The later patch will leverage these two helpers.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: provide helper for host notifier ctrl

VDPA driver can decide if it needs to enable/disable the host notifier
mapping, so exposing a API can allow flexibility. A later patch will
base on this.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: remove unused function

vhost_detach_vdpa_device() is internally defined but not used, remove
it in this patch.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: enable packed virtqueues by default

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio-user: fail if cq used with packed vq

Until we have support for control virtqueues let's disable it and
fail device initalization if specified as a parameter.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio-user: add option to use packed queues

Add option to enable packed queue support for virtio-user
devices.

Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: support packed queue in send command

Use packed virtqueue format when reading and writing descriptors
to/from the ring.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: implement Rx path for packed queues

Implement the receive part.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: implement Tx path for packed queues

This implements the transmit path for devices with
support for packed virtqueues.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: dump packed virtqueue data

Add support to dump packed virtqueue data to the
VIRTQUEUE_DUMP() macro.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: vring init for packed queues

Add and initialize descriptor data structures.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: add packed virtqueue helpers

Add helper functions to set/clear and check descriptor flags.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: add packed virtqueue defines

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: fix race condition when adding fd in the fdset

fdset_add can call fdset_shrink_nolock which call fdset_move
concurrently to poll that is call in fdset_event_dispatch.

This patch add a mutex to protect poll from been call at the same time
fdset_add call fdset_shrink_nolock.

Fixes: 1b815b89599c ("vhost: try to shrink pfdset when fdset_add fails")
Cc: stable@dpdk.org
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

doc: add af_packet PMD guide

As of commit 364e08f2bbc0, DPDK allows an application to send and
receive raw packets using an AF_PACKET and PACKET_MMAP, when using
Linux Kernel. This complements it by adding a simple guide with the
following information:
- An introduction, where a brief explanation of this driver is given,
  pointing out the dependency on PACKET_MMAP;
- Which options are supported at configuration time, while setting up an
  interface, and it's inherent limitations;
- What the prerequisites are;
- A command line example of how to set up a DPDK port using the
  af_packet driver.

Since there's a dependency in PACKET_MMAP, the guide also points to the
original Kernel documentation, so the reader can get more details.

Signed-off-by: Tiago Lam <tiago.lam@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/netvsc: fix probe when VF not found

It is possible that the VF device exists but DPDK doesn't know
about it. This could happen if device was blacklisted or more
likely the necessary device (Mellanox) was not part of the DPDK
configuration.

In either case, the right thing to do is just keep working
but only with the slower para-virtual device.

Fixes: dc7680e8597c ("net/netvsc: support integrated VF")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>

net/netvsc: fix transmit descriptor pool cleanup

On device close or startup errors, the transmit descriptor pool
was being left behind.

Fixes: 4e9c73e96e83 ("net/netvsc: add Hyper-V network device")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>

net/netvsc: support receive without VLAN strip

In some cases, VLAN stripping is not desireable. If necessary
re-insert stripped VLAN tag.

Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>

net/i40e: fix statistics inconsistency

While calculating the input packet count per port, discarded packets
should be reduced, right now only PF VSI discarded packets are reduced.
But while calculating the input byte count per port, Rx byte count is
used, which should take all discarded packets into account, including
VF VSI ones.
This will cause inconsistency in stat counters in some cases.

This patch would take all VSI stats as packet and byte count to address
the issue.

Fixes: 763de290cbd1 ("net/i40e: fix packet count for PF")
Cc: stable@dpdk.org
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: clear VF reset flags after reset

The reset flags vf->vf_reset and vf->pend_msg are set when VF received
VIRTCHNL_EVENT_RESET_IMPENDING. So after resetting done, these flags
should be cleared.

Fixes: 8cacf78469a7 ("net/i40e: fix VF initialization error")
Cc: stable@dpdk.org
Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>