mbuf: support dynamic fields and flags

Many features require to store data inside the mbuf. As the room in mbuf
structure is limited, it is not possible to have a field for each
feature. Also, changing fields in the mbuf structure can break the API
or ABI.

This commit addresses these issues, by enabling the dynamic registration
of fields or flags:

- a dynamic field is a named area in the rte_mbuf structure, with a
  given size (>= 1 byte) and alignment constraint.
- a dynamic flag is a named bit in the rte_mbuf structure.

The typical use case is a PMD that registers space for an offload
feature, when the application requests to enable this feature.  As
the space in mbuf is limited, the space should only be reserved if it
is going to be used (i.e when the application explicitly asks for it).

The registration can be done at any moment, but it is not possible
to unregister fields or flags.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

eal: fix memory config allocation for multi-process

Currently, mem config will be mapped without using the virtual
area reservation infrastructure, which means it will be mapped
at an arbitrary location. This may cause failures to map the
shared config in secondary process due to things like PCI
whitelist arguments allocating memory in a space where the
primary has allocated the shared mem config.

Fix this by using virtual area reservation to reserve space for
the mem config, thereby avoiding the problem and reserving the
shared config (hopefully) far away from any normal memory
allocations.

Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

mem: make base address hint OS specific

Not all OS's follow Linux's memory layout, which may lead to
problems following the suggested common address hint absent
of a base-virtaddr flag. Make this address hint OS-specific.

Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

vfio: fix truncated BAR offset for 32-bit

When 32-bit application is built on 64-bit system it is possible that
the offset of the resource is outside of the 32-bit value.

The problem with the unsigned long is, that it is 32-bit and not 64-bit
when using armhf compiler. Although the system is returning u64 value,
we are losing it's value if it's higher than 32-bit in the conversion
process. It can further cause mmap to fail due to offset being 0 or to
map not intended memory region.

To make it more portable, the uint64_t value is now being used for
storing offset instead of unsigned long. The size of being 32-bit seems
to be fine as the 32-bit application won't be able to access bigger
memory and it is further converted to size_t anyway. But for better
readability and to be consistent, it's type was changed to size_t as
well.

Fixes: 0205f873557c ("vfio: fix overflow of BAR region offset and size")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

eal: move CPU operations to OS specific headers

Moving RTE_CPU* definitions from the common code to the Linux and
FreeBSD rte_os.h file to avoid #ifdef clutter.

Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Signed-off-by: Antara Ganesh Kolar <antara.ganesh.kolar@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>

reciprocal: fix off-by-one with 32-bit divisor

Fix off-by-one error in 64bit reciprocal division when divisor is 32bit.

Caught with the unit test:

RTE>>reciprocal_division
Validating unsigned 32bit division.
Validating unsigned 64bit division.
Validating unsigned 64bit division with 32bit divisor.
Division failed, 16983222950483802557/819 = expected 20736535959076681
result 20736535959076682
Validating division by power of 2.
Test Failed

Fixes: 6d45659eacb8 ("eal: add u64-bit variant for reciprocal divide")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

test: fix global variable multiple definitions

Multiple global variable are defined in multiple unit test files with
same name, but all unit test files are linked into single executable,
which means those variables share same storage which is not the
intention, fixed by making global variables 'static'.

Issue has been detected by '-fno-common' gcc flag.

Fixes: fdeb30fa7102 ("test/bitrate: add unit tests for bitrate library")
Fixes: c3eabff124e6 ("distributor: add unit tests")
Fixes: 0e925aef2779 ("app/test: add EFD functional and perf tests")
Fixes: 359e17bf081f ("app/test: improve hash unit tests")
Fixes: c7eb0972e74b ("test/hash: add lock-free r/w concurrency")
Fixes: 1e3676a06e4c ("test/latency: add unit tests for latencystats library")
Fixes: 0cc67a96e486 ("test/member: add functional and perf tests")
Fixes: e6a14121f4ae ("test/rcu: remove arbitrary limit on max core count")
Fixes: 104dbec2081a ("test/rcu: increase size of core numbers")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

app/testpmd: fix global variable multiple definitions

Some flow config related global variables are defined in a header file
which was causing multiple definitions of the variables, fixed it by
moving them to the .c file.

Issue has been detected by '-fno-common' gcc flag.

Also while being there,
removed duplicated 'ACTION_RAW_ENCAP_MAX_DAT definition,
moved 'vxlan_encap_conf' & 'nvgre_encap_conf' initialization to
  'cmdline_flow.c' which is better location than 'testpmd.c'
relocated 'action_raw_encap_data' & 'action_raw_decap_data' struct
   definitions slightly within the file

Fixes: 1960be7d32f8 ("app/testpmd: add VXLAN encap/decap")
Fixes: dcd962fc6b4e ("app/testpmd: add NVGRE encap/decap")
Fixes: a1191d39cb57 ("app/testpmd: add MPLSoUDP encapsulation")
Fixes: 3e77031be855 ("app/testpmd: add MPLSoGRE encapsulation")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

compress/octeontx: fix global variable multiple definitions

'octtx_zip_logtype_driver' global variable is defined in a header file
which was causing multiple definitions of the variable, fixed it by
moving it to the .c file.

Issue has been detected by '-fno-common' gcc flag.

Fixes: 43e610bb8565 ("compress/octeontx: introduce octeontx zip PMD")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ashish Gupta <ashishg@marvell.com>

crypto/virtio: fix global variable multiple definitions

'virtio_hw_internal' global variable is defined in both 'crypto/virtio'
and 'net/virtio' PMDs. This means they share same storage when
application linked with static DPDK library, which is not the intention.

Fixing by adding 'crypto_' prefix to the 'crypto/virtio' driver.

Issue has been detected by '-fno-common' gcc flag.

Fixes: 25500d4b8076 ("crypto/virtio: support device init")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

crypto/null: fix global variable multiple definitions

'null_logtype_driver' global variable is defined in a header file which
was causing multiple definitions of the variable, fixed it by moving it
to the .c file.

Issue has been detected by '-fno-common' gcc flag.

Fixes: 735b783d8c2b ("crypto/null: add dynamic logging")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/igb: fix global variable multiple definitions

Filtering related global variables are defined in a header file which
was causing multiple definitions of the variables, fixed it by moving
them to the .c file.

Issue has been detected by '-fno-common' gcc flag.

Fixes: 22bb13410cb2 ("net/igb: create consistent filter")
Fixes: 424ae915baf0 ("net/e1000: move RSS to flow API")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

bus/fslmc: fix global variable multiple definitions

'qman_version' global variable is defined in a header file which was
causing multiple definitions of the variable, fixed it by moving it to
the .c file.

Issue has been detected by '-fno-common' gcc flag.

Fixes: 293c0ca94c36 ("bus/fslmc: support memory backed portals with QBMAN 5.0")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Sachin Saxena <sachin.saxena@nxp.com>

examples/bpf: remove duplicate mbuf definition

Get rid of duplicate definitions for rte_mbuf and related macros.
Include rte_mbuf_core.h instead.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Michel Machado <michel@digirati.com.br>

mbuf: move definitions into a separate file

Right now inclusion of rte_mbuf.h header can cause inclusion of
some arch/os specific headers.
That prevents it to be included directly by some
non-DPDK (but related) entities: KNI, BPF programs, etc.
To overcome that problem usually a separate definitions of rte_mbuf
structure is created within these entities.
That aproach has a lot of drawbacks: code duplication, error prone, etc.
This patch moves rte_mbuf structure definition (and some related macros)
into a separate file that can be included by both rte_mbuf.h and
other non-DPDK entities.

Note that it doesn't introduce any change for current DPDK code.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Michel Machado <michel@digirati.com.br>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

eal: move cache line and IOVA related definitions

Right now RTE_CACHE_ and IOVA definitions are located inside rte_memory.h
That might cause an unwanted inclusions of arch/os specific header files.
See [1] for particular problem example.
Probably the simplest way to deal with such problems -
move these definitions into rte_commmon.h

Note that this move doesn't introduce any change in functionality.

[1] https://bugs.dpdk.org/show_bug.cgi?id=321

Suggested-by: Vipin Varghese <vipin.varghese@intel.com>
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Michel Machado <michel@digirati.com.br>

port: add eventdev port type

Adding a new port type called eventdev to the
rte_port library.

Signed-off-by: Rahul Shah <rahul.r.shah@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: support 64-bit values

Modify internal structure and functions to support 64-bit
values for rates and stats parameters.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: add 64-bit values

To support high bandwidth network interfaces, all rates (port,
subport level token bucket and traffic class rates, pipe level
token bucket and traffic class rates) and stats counters defined
in public data structures (rte_sched.h) are modified to support
64 bit counters.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

sched: remove redundant code

Remove redundant data structure fields from port level data
structures and update the release notes.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

examples/qos_sched: add subport config flexibility

Modify qos sample app to allow different subports of the same port
to have different configuration in terms of number of pipes, pipe
queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

examples/ip_pipeline: add subport config flexibility to TM

Modify ip pipeline traffic management function to allow different
subports of the same port to have different configuration in terms
of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

net/softnic: add subport config flexibility to TM

Modify softnic traffic management function to allow different
subports of the same port to have different configuration in
terms of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

test/sched: add subport config flexibility

Modify tests function to allow different subports of the same port
to have different configuration in terms of number of pipes, pipe
queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: update queue stats read for config flexibility

Modify pipe queue stats read function to allow different subports
of the same port to have different configuration in terms of number
of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: update pkt dequeue for flexible config

Modify scheduler packet dequeue operation to allow different
subports of the same port to have different configuration in terms
of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: update grinder functions for config flexibility

Modify packet grinder functions of the schedule to allow different
subports of the same port to have different configuration in terms
of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: update memory compute to support flexiblity

Update memory footprint compute function for allowing subports of
the same port to have different configuration in terms of number of
pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: modify pkt enqueue for config flexibility

Modify scheduler packet enqueue operation of the scheduler to allow
different subports of the same port to have different configuration
in terms of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: modify pipe functions for config flexibility

Modify pipe level functions to allow different subports of the same
port to have different configuration in terms of number of pipes,
pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: add pipe config to subport level

Add pipes configuration from the port level to allow different
subports of the same port to have different configuration in terms
of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: remove pipe params config from port level

Remove pipes configuration from the port level to allow different
subports of the same port to have different configuration in terms
of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: modify internal structs for config flexibility

Update internal structures related to port and subport to allow
different subports of the same port to have different configuration
in terms of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

sched: add pipe config params to subport struct

Add pipe configuration parameters to subport level structure to
allow different subports of the same port to have different
configuration in terms of number of pipes, pipe queue sizes, etc.

Signed-off-by: Jasvinder Singh <jasvinder.singh@intel.com>
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>

net/ice: add GTP tunnel type

Add GTP tunnel type in ice pmd tunneling parameters parse
to enable Tx checksum offload.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

mbuf: add GTP tunnel type

Add GTP tunnel type flag in mbuf for future use in GTP
Tx checksum offload.

Signed-off-by: Ting Xu <ting.xu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

app/testpmd: increase RSS max queue size

Max queue indices ACTION_RSS_QUEUE_NUM , which limits testpmd actions
rss queue size, has worked for a long time.

As more powered PMD is merged, more RSS queues are supported(e.g. Intel
ice driver FDIR supports up to 128 RSS queue numbers).

The original max queue indices number doesn't satisfy PMD's requirement
and it needs to become bigger.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/enetc: enable dpaax library

dpaaX is used to maintain a local copy of PA->VA translations.

Using the rte_mem_virt2iova or rte_mem_virt2phy is expensive. This
library is an attempt to reduce the overall cost associated with
this translation.

This patch enables this dpaaX library by populating a dpaaX's
table for PA to VA translation. This change will also help the
caam JR driver in PA to VA translation when used with enetc driver
over enetc supported SoCs.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: print MAC address logs at notice level

To display random MAC address as notice,
a log level NOTICE is added.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: set random MAC in case no MAC for SI

for SGMII interfaces, there can be 0 value
written on MAC registers.
This patch set the random MAC address for those
interfaces.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/enetc: fix BD ring alignment

enetc BD rings should be aligned to 128
instead of RTE_CACHE_LINE_SIZE.

Fixes: 469c6111a799 ("net/enetc: enable Rx and Tx")
Cc: stable@dpdk.org
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

net/mlx5: fix DevX event registration timing

The DevX counter management triggers an asynchronous event to get back
the new counters values from the HW.

The counter management doesn't trigger 2 parallel events for the same
pool, hence, the pool cannot be updated again in the event waiting time.

When the port is stopped, the DevX event mechanism wrongly was
destroyed what remained all the waiting pools in waiting state forever.

As a result, the counters of the stuck pools were never updated again.

Separate the DevX interrupt installation from the dev installation and
remove the DevX interrupt unregistration\registration from the
stop\start operations.

Now, the DevX interrupt should be installed in probe and uninstalled in
close.

Cc: stable@dpdk.org
Fixes: f15db67df09c ("net/mlx5: accelerate DV flow counter query")

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/ice: fix queue index for switch filter

When calculate queue index for queue action in
switch filter, vsi base queue should be included.

Fixes: 57c4f2693588 ("net/ice: enable switch filter")
Cc: stable@dpdk.org
Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/af_xdp: support unaligned umem chunks

This patch enables the unaligned chunks feature for AF_XDP which allows
chunks to be placed at arbitrary places in the umem, as opposed to them
being required to be aligned to 2k. This allows for DPDK application
mempools to be mapped directly into the umem and in turn enable zero copy
transfer between umem and the PMD.

This patch replaces the zero copy via external mbuf mechanism introduced
in commit e9ff8bb71943 ("net/af_xdp: enable zero copy by external mbuf").
The pmd_zero copy vdev argument is also removed as now the PMD will
auto-detect presence of the unaligned chunks feature and enable it if so
and otherwise fall back to copy mode if not detected.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

ethdev: add HIGIG2 key field to flow API

Add new rte_flow_item_higig2_hdr in order to match higig2 header.
It is a layer 2.5 protocol and used in Broadcom switches.
Header format is based on the following document.
http://read.pudn.com/downloads558/doc/comm/2301468/HiGig_protocol.pdf

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

drivers/net: enable promiscuous and multicast by default

The promiscuous and multicast fields are now initialised as enabled for
some virtual PMDs. This allows the devices to be used when running
applications that attempt to enable promiscuous or multicast mode.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

ethdev: check device promiscuous state

The promiscuous enable and disable functions now check the
promiscuous state of the device before checking if the dev_ops
function exists for the device.

This change is necessary to allow sample applications run on
virtual PMDs, as previously -ENOTSUP returned when the promiscuous
enable function was called. This caused the sample application to
fail unnecessarily.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/af_packet: improve Tx statistics accuracy

When sendto call fails and ENOBUFS/EAGAIN error is being set
some of the packets are actually successfully transmitted.
There is no available count of those packets, so in order to
make the statistics more accurate, all the previously enqueued
packets will be considered successful, even though this is not
entirely correct.

Statistics numbers before this update:

Pktgen:
   Total Rx Pkts:               1360084
         Tx Pkts:               2000000
testpmd:
   RX-packets: 1408346  RX-missed: 0       RX-bytes:  84503418
   TX-packets: 526486   TX-errors: 881851  TX-bytes:  31589724

Statistics numbers after this update:

Pktgen:
   Total Rx Pkts:               1329872
         Tx Pkts:               2000000
testpmd:
   RX-packets: 1389156  RX-missed: 0       RX-bytes:  83349360
   TX-packets: 1389156  TX-errors: 0       TX-bytes:  83349360

Fixes: 74b7fc0a0ff1 ("net/af_packet: fix packet bytes counting")
Cc: stable@dpdk.org
Signed-off-by: Flavia Musatescu <flavia.musatescu@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/mlx5: fix crash on hash Rx queue handling for drop

When to create hrxq for the drop, it could fail on creating qp and goto
the error handle which will release created ind_table by calling drop
release function, which takes rte_ethdev as the only parameter and uses
the priv->drop_queue.hrxq as input to release.

Unfortunately, at this point, the hrxq is not allocated and
priv->drop_queue.hrxq is still NULL, which leads to a segfault.

This patch fixes the above by allocating the hrxq at first place and
when the error happens, hrxq is released as the last one.

This patch also release other allocated resources by the correct order,
which is missing previously.

Fixes: 78be885295b8 ("net/mlx5: handle drop queues as regular queues")
Cc: stable@dpdk.org
Reported-by: Zengmo Gao <gaozengmo@jd.com>
Signed-off-by: Xiaoyu Min <jackmin@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/ice: enable advanced RSS

This patch supports:

- symmetric hash by rte_flow RSS action.
- input set change by rte_flow RSS action.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/ice: add RSS configuration for GTPU/PPPoE

This patch adds rss support for gtpu with input set teid and
pppoe/pppod with input set source mac and session id.

Signed-off-by: Simei Su <simei.su@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/i40e: support flow director on AVX Rx

This commit adds FDIR ID support to the AVX2 based receive
path routine. Support for both 16B and 32B descriptors is
implemented.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/i40e: support flow director on SSE Rx

This commit adds an implementation to the SSE vector implementation of
RX routine and moves some common defines from a c file to the header
file.

I40e can have 16 and 32 byte descriptors, and the Flow
Director ID data and indication-bit are in different locations
for each size descriptor. The support is implemented in two
separate functions as they require vastly different operations.

The 16B descriptor re-purposes the "filter-status" u32 field
to indicate FDIR ID when the FLM bit is set. No extra loads
are required, however we do have to store to mbuf->fdir.hi,
which is not stored to in the RX path before this patch.

The 32B descriptor requires loading the 2nd 16 bytes of each
descriptor, to get the FLEXBH_STAT and FD Filter ID from
qword3. The resulting data must also be stored to mbuf->fdir.hi,
same as the 16B code path.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/i40e: cache flow director enable value in Rx queue

This commit adds a fdir_enable flag in a uint8_t sized hole
the rx queue structure The flag enables the rx code path to
easily identify if fdir is active. This can be used to skip
fdir id processing when it is not required.

The flag is zero by default (as rxq is zmalloc-ed at startup),
and the flag is set to 1 on configuration of a flow director rule.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>

net/axgbe: fix double unlock

One issue caught by Coverity 340835
*unlock: axgbe_phy_set_mode unlocks pdata->phy_mutex
*double_unlock: axgbe_phy_sfp_detect unlocks pdata->phy_mutex
while it is unlocked.

In axgbe_phy_sfp_detect()/axgbe_phy_set_redrv_mode(),
axgbe_phy_get_comm_ownership() and axgbe_phy_put_comm_ownership()
are invoked subsequently.

Currently in axgbe_phy_get_comm_ownership(), during one of the case
'phy_data->comm_owned' is not protected and before returning 0, lock is
not called and unlock is called in axgbe_phy_put_comm_ownership()
directly which is incorrect.

Ideally, the variable 'phy_data->comm_owned' needs to be protected.
During success scenario, lock is called in
axgbe_phy_get_comm_ownership() followed by unlock in
axgbe_phy_put_comm_ownership().  In failure case, unlock is invoked in
axgbe_phy_get_comm_ownership() itself appropriately.

The fix is to protect 'phy_data->comm_owned' in the identified case
ensuring locks/unlocks properly exist.

Coverity issue: 340835
Fixes: a5c7273771e8 ("net/axgbe: add phy programming APIs")
Cc: stable@dpdk.org
Signed-off-by: Pallantla Poornima <pallantlax.poornima@intel.com>
Acked-by: Ravi Kumar <ravi1.kumar@amd.com>

net/iavf: enable AVX2 for iavf

This patch enables AVX data path for iavf PMD.

Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

net/mlx5: add flow match on GENEVE item

This commit adds support for matching flows on Geneve headers.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: query HCA for enabled FLEX parser protocols

This commit add querying the HCA which FLEX protocols are already
enabled.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: retry on link update failure

mlx5_link_update immediately returns when called with no-wait parameter
and its call for retrieving the link status returns with EAGAIN error.
This is too harsh on busy systems where a first call fails with EAGAIN
from time to time.
This patch adds a (very limited) retry on such cases in order to allow
retrieving the link status.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: fix build with strict alignment enabled

This patch converts some of the casts to unaligned integer types.
The memcpy call is replaced with explicit copying because it
may require type qualifiers in some environments.

Fixes: 18a1c20044c0 ("net/mlx5: implement Tx burst template")
Cc: stable@dpdk.org
Reported-by: Jeremy Plsek <jplsek@iol.unh.edu>
Signed-off-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx5: fix Tx packets statistics

The transmitter packets counter was not updated
correctly in the loop inside the tx_burst routines.

Fixes: f32a3f5216a3 ("net/mlx5: fix completion queue overflow for large burst")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

net/mlx4: remove dependency on libmnl in meson

There is no dependency on this library for mlx4.

Fixes: 1dd7c7e38c19 ("net/mlx4: support meson build")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>

app/testpmd: fix help for loop topology option

The testpmd --help option did not show all possible choices for port
topology previously. The loop topology option is now added.

Fixes: 3e2006d6186c ("app/testpmd: add loopback topology")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net: add missing endianness annotations

OVS currently maintains a copy of those headers with the right endianness
annotations so that sparse checks can pass.

We introduced rte_beXX_t for better readibility in v17.08.
Let's make use of them, OVS then only needs to override those rte_beXX_t
types by exposing a tweaked rte_byteorder.h header.

Other existing dpdk users won't be affected since rte_beXX_t types are
mapped to uintXX_t types.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

doc: fix a common typo in NIC guides

'CRC striping' should be spelled 'CRC stripping'.

Fixes: 3eee1f067e7c ("fm10k: add guide")
Fixes: 7a4d9f6676d7 ("doc: add liquidio")
Fixes: f820b5896631 ("doc: add octeontx ethdev driver documentation")
Fixes: 920717e4d8ba ("net/octeontx2: add device start operation")
Fixes: f994cecafdcf ("doc: add ThunderX nicvf")
Cc: stable@dpdk.org
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

maintainers: add next-net-brcm sub-tree

Adding vendor specific sub-tree for Broadcom drivers.
The next-net-brcm sub-tree will be sub-tree of the next-net.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

net/octeontx2: support multicast filter

Patch adds mc filter support for otx2 eth devices.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

common/octeontx2: upgrade mbox definition to version 2

Sync mail box data structures to version 0x0002.
This patch checks for mismatch in mail box revision and
avoids initializing octeontx2 pci device if
there is a mismatch.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>

net/ice: cleanup RSS/FDIR profile on device init

When app restarts, RSS/FDIR profile ID will increase which may
run out of all the profile resources.

The patch cleanup RSS/FDIR profile resources when device init
to fix this issue.

Fixes: d7d150b93070 ("net/ice: enable RSS when device init")
Cc: stable@dpdk.org
Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director GTPU tunnel

Enable FDIR GTPU matching for RTE_FLOW

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director VXLAN tunnel

Enable FDIR vxlan tunnel matching for RTE_FLOW

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: reject duplicated flow for flow director

Enable duplication lookup for existing flow director rule entry.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: support flow director counter

This patch add FDIR statistical counter support
and it includes RTE_FLOW count actions support
and query support.

RTE_FLOW count actions support id and shared.
RTE_FLOW query record packet hits by default.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add flow director counter resource init/release

The patch integrates the counter resource init/release into fdir's
init/release scenario

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: enable flow director queue group

FDIR can send packet to a group of queues and distruibte it by RSS.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add flow director create and destroy

Add ice_create_fdir_filter to create a rule. If a flow is matched by
flow director filter, filter rule will be set to HW. For now common
pattern and queue/passthru/drop/mark actions are supported.

Signed-off-by: Yahui Cao <yahui.cao@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: configure HW flow director rule

This patch adds a HW FDIR rule to the FDIR HW table
without adding a FDIR filter.

Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: enable flow director engine

Enable flow director engine, including initialization and teardown.
 - Control VSI create and release.
 - Queue pair allocated, set up and release.
 - Programming packet create and release.
 - FDIR profile create and release.

Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

app/testpmd: support extended RSS offload types

This patch adds cmdline support for extended rss types configuration.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

ethdev: extend RSS offload types

This patch reserves several bits as input set selection from the
high end of the 64 bits. It is combined with exisiting ETH_RSS_*
to represent RSS types. This patch also checks the simultaneous
use of SRC_ONLY and DST_ONLY of the same level.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Ori Kam <orika@mellanox.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

ethdev: decouple flow types and RSS offload types

This patch decouples RTE_ETH_FLOW_* and ETH_RSS_*. The former defines
flow types and the latter defines RSS offload types.

Signed-off-by: Simei Su <simei.su@intel.com>
Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

net/ixgbe: fix address of first segment

This patch fixes (dereference after null check) coverity issue.
The address of first segmented packets was not set correctly during
reassembling packets which led to this issue.

Coverity issue: 13245
Fixes: 8a44c15aa57d ("net/ixgbe: extract non-x86 specific code from vector driver")
Cc: stable@dpdk.org
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/i40e: fix address of first segment

This patch fixes (dereference after null check) coverity issue.
The address of first segmented packets was not set correctly during
reassembling packets which led to this issue.

Coverity issue: 343422, 343403
Fixes: ca74903b75cf ("net/i40e: extract non-x86 specific code from vector driver")
Cc: stable@dpdk.org
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: invoke callback when link status change

Needs to call _rte_eth_dev_callback_process to run registered
callbacks when link status change.

Fixes: cf911d90e366 ("net/ice: support link update")
Cc: stable@dpdk.org
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: rework switch filter

The patch reworks packet process engine's binary classifier
(switch) for the new framework. It also adds support for new
packet type like PPPoE for switch filter.

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add pattern manifest

The patch adds a manifest for all patterns which can be selected
by low level filter engine to compose their supported list.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: rework for generic flow enabling

The patch reworks the generic flow API (rte_flow) implementation.
It introduces an abstract layer which provides a unified interface
for low-level filter engine (switch, fdir, hash) to register supported
patterns and actions and implement flow validate/create/destroy/flush/
query activities.

The patch also removes the existing switch filter implementation to
avoid compile error. Switch filter implementation for the new framework
will be added in the following patch.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: add devargs to control pipeline mode

Added a devarg to control the mode in generic flow API.
We use none-pipeline mode by default.

Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

net/ice: clean up redundant assignment and indentations

The patch removes redundant code and cleans up some wrong indentations.

Signed-off-by: Ying A Wang <ying.a.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

vhost: improve performance by supporting large buffer

The rte_vhost_dequeue_burst supports two ways of dequeuing data.
If the data fits into a buffer, then all data is copied and a
single linear buffer is returned. Otherwise it allocates
additional mbufs and chains them together to return a multiple
segments mbuf.

While that covers most use cases, it forces applications that
need to work with larger data sizes to support multiple segments
mbufs. The non-linear characteristic brings complexity and
performance implications to the application.

To resolve the issue, add support to attach external buffer
to a pktmbuf and let the host provide during registration if
attaching an external buffer to pktmbuf is supported and if
only linear buffer are supported.

Signed-off-by: Flavio Leitner <fbl@sysclose.org>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

net/virtio: fix descriptor addressed in Tx

Previous fix removes usage of rte_pktmbuf_prepend() to get pointer
to virtio net header which changes mbuf data_off and data_len.
Size of virtio net header is added to segment length when Tx descriptor
is composed, but segment address (calculated using data_off) is not
adjusted to take size of virtio net header into account.

Fixes: 1ae55ad38e5e ("net/virtio: fix mbuf data and packet length mismatch")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add packed ring support to vring related APIs

This patch add packed ring support in two APIs
so user can get the packed ring`.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add APIs to get inflight ring

This patch introduces two APIs. one is for getting inflgiht
ring and the other is for getting base.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add APIs to operate inflight ring

This patch introduces three APIs to operate the inflight
ring. Three APIs are set, set last and clear. It includes
split and packed ring.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: checkout resubmit inflight information

This patch shows how to checkout the inflight ring and construct
the resubmit information also include destroying resubmit info.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: support inflight info sharing

This patch introduces two new messages VHOST_USER_GET_INFLIGHT_FD
and VHOST_USER_SET_INFLIGHT_FD to support transferring a shared
buffer between qemu and backend.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add inflight structures

This patch adds the inflight queue region structure include
the split and packed.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add packed ring into vring struct

This patch add the packed ring in the rte_vhost_vring.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: add inflight description

This patch add the inflight message description and
the inflight share fd protocol feature flag.

Signed-off-by: Lin Li <lilin24@baidu.com>
Signed-off-by: Xun Ni <nixun@baidu.com>
Signed-off-by: Yu Zhang <zhangyu31@baidu.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: prevent zero copy mode if IOMMU is on

The simultaneous use of dequeue_zero_copy and IOMMU is problematic.
Not only because IOVA_VA mode is not supported but also because the
potential invalidation of guest pages while the buffers are in use,
is not handled.

Prevent these two features to be enabled simultaneously.

Fixes: 69c90e98f483 ("vhost: enable IOMMU support")
Cc: stable@dpdk.org
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>