doc: add asymmetric crypto in programmer guide

Update cryptodev programmer guide with description of
asymmetric crypto framework in lib cryptodev.

Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
Signed-off-by: Umesh Kartha <umesh.kartha@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: add asymmetric capability

Extend cryptodev with asymmetric capability APIs and
definitions.

Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
Signed-off-by: Umesh Kartha <umesh.kartha@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: support asymmetric operations

Extend DPDK librte_cryptodev to:
- define asym op type in rte_crypto_op_type and associated
  op pool create/alloc APIs
- define asym session and associated session APIs

If PMD shows in its feature flag that it supports both sym and
asym then it must support those on all its qps.

Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
Signed-off-by: Umesh Kartha <umesh.kartha@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: add asymmetric xform and op definitions

Add rte_crypto_asym.h with supported xforms
and associated op structures and APIs

API currently supports:
- RSA Encrypt, Decrypt, Sign and Verify
- Modular Exponentiation and Inversion
- DSA Sign and Verify
- Diffie-Hellman private key exchange
- Diffie-Hellman public key exchange
- Diffie-Hellman shared secret compute
- Diffie-Hellman public/private key pair generation
using xform chain

Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
Signed-off-by: Umesh Kartha <umesh.kartha@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/scheduler: add minimum head/tailroom requirement

Minimum head/tailroom requirement for each PMD has to be considered
while populating the dev_info.

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

test/crypto: skip validation of head/tailroom used by PMD

Crypto PMDs would specify the head/tailroom it would use while
processing the crypto requests. This need to be considered while
verifying buffers processed by crypto PMDs.

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

app/crypto-perf: honour min headroom/tailroom

Crypto dev would specify its headroom and tailroom requirement and the
application is expected to honour this while creating buffers.

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: add min headroom and tailroom requirement

Enabling crypto devs to specify the minimum headroom and tailroom it
expects in the mbuf. For net PMDs, standard headroom has to be honoured
by applications, which is not strictly followed for crypto devs. This
prevents crypto devs from using free space in mbuf (available as
head/tailroom) for internal requirements in crypto operations. Addition
of head/tailroom requirement will help PMDs to communicate such
requirements to the application.

The availability and use of head/tailroom is an optimization if the
hardware supports use of head/tailroom for crypto-op info. For devices
that do not support using the head/tailroom, they can continue to operate
without any performance-drop.

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: rename experimental private data APIs

The name private_data is confusing in these APIs:
rte_cryptodev_sym_session_set_private_data()
rte_cryptodev_sym_session_get_private_data()
It refers to data added at the end of the session hdr for
use by the application.
The session already contains sess_private_data[index]
which is used to store private pmd data and most references to private
data refer to that.
e.g. external apis
rte_cryptodev_sym_get_private_session_size() and internal
set/get_session_private_data() refer to sess_private_data[].

So rename to user_data, i.e.
rte_cryptodev_sym_session_set_user_data()
rte_cryptodev_sym_session_get_user_data()

Refers to changes introduced here:
https://patches.dpdk.org/patch/38172/

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>

cryptodev: check if symmetric sessions are supported

Since asymmetric functionality will be implemented soon,
not all PMDs must support symmetric sessions.
Therefore, a check is added if a device does not implement
the symmetric functions, meaning that the device does not
support symmetric operations.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: rename PMD symmetric session API

The PMD specific API to configure, clear and
obtain session private size is renamed, including
the word _sym_ to clarify that it is API
for symmetric sessions, so there will not be any
conflicts for asymmetric and other type of sessions
in the future.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove attach/detach session API

As announced in the previous release,
The API to attach/dettach a session to a queue pair
is removed, as it was only used in DPAA, and it is not
actually needed.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: replace mbuf scatter gather flag

The current mbuf scatter gatter feature flag is
too ambiguous, as it is not clear if input and/or output
buffers can be scatter gather mbufs or not, plus
if in-place and/or out-of-place is supported.

Therefore, five new flags will replace this flag:
- RTE_CRYPTODEV_FF_IN_PLACE_SGL
- RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT
- RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT
- RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT
- RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove old get session size functions

Removed rte_cryptodev_get_header_session_size
and rte_cryptodev_get_private_session_size functions,
as they have been substituted with functions
specific for symmetric operations, with _sym_ word
after "rte_cryptodev_".

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove queue start/stop functions

Removed cryptodev queue start/stop functions,
as they were marked deprecated in 18.05, since they
were not implemented by any driver.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: remove unneeded deprecation notice

In release 18.05, a deprecation notice to remove the `sym`
structure in the cryptodev info structure was sent.
However, only one of the fields inside the structure will
be removed, so the notice is not actually correct.
In any case, it needs to be removed.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove max number of sessions parameter

Most crypto PMDs do not have a limitation
of the number of the sessions that can be handled
internally. The value that was set before was not
actually used at all, since the sessions are created
at the application level.
Therefore, this value is not parsed from the initial
crypto parameters anymore and it is set to 0,
meaning that there is no actual limit.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: define value for unlimited sessions

Currently, the info structure contains the maximum number
of sessions that a device can manage.
This field was useful when the session mempool was created inside
each device, but now it is created at the application level.

Most PMDs do not have a limitation on the sessions managed,
but a few do, therefore this field must remain in the structure.
However, a new value, 0, can be used to indicate that
a device does not have an actual maximum of sessions.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/mvsam: parse max number of sessions

The maximum number of sessions device argument will be removed,
as most PMDs do not have a limitation on this number.
Therefore, the MVSAM PMD needs to parse this value internally.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Tomasz Duszynski <tdu@semihalf.com>

examples/ipsec-secgw: check for max supported sessions

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

examples/l2fwd-crypto: limit number of sessions

Calculate the number of sessions required for the application,
knowing that there is only one session required per device.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

test/crypto: limit number of sessions

Instead of using the maximum number of sessions
allowed by the PMDs (which will change to unlimited most
of the PMDs), limit the number to a small sufficient amount.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

app/crypto-perf: limit number of sessions

Instead of creating a fixed number of sessions,
calculate the necessary number based on number of devices
and queue pairs used.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: remove max number of sessions per queue

The cryptodev info structure currently contains
the maximum number of sessions that can be used in a queue pair.
This is only set in DPAA_SEC PMD, and since it is calculated
based on the maximum number of sessions (which is not used
anymore), this field can be removed.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

cryptodev: replace bus specific struct with generic dev

Structure rte_cryptodev_info has currently PCI device
information ("struct rte_pci_device") in it.

This information is not generic to all devices,
so this gets replaced with the generic "rte_device" structure,
compatible with all crypto devices.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

compressdev: add huffman encoding flags

Added Huffman fixed and dynamic encoding feature flags,
so an application can query if a device supports
these two types, when performing DEFLATE compression.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Shally Verma <shally.verma@caviumnetworks.com>

compressdev: replace mbuf scatter gather flag

The current mbuf scatter gather feature flag is
too ambiguous, as it is not clear if input and/or output
buffers can be scatter gather mbufs or not.

Therefore, three new flags will replace this flag:
- RTE_COMP_FF_OOP_SGL_IN_SGL_OUT
- RTE_COMP_FF_OOP_SGL_IN_FB_OUT
- RTE_COMP_FF_OOP_LB_IN_SGL_OUT

Note that out-of-place flat buffers is supported by default
and in-place is not supported by the library.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Shally Verma <shally.verma@caviumnetworks.com>

doc: rename compress feature flag

Renamed feature "Bypass" to "Pass-through",
as it is a more explicit name, meaning that the PMD
is capable of passing the mbufs through it,
without making any modifications (i.e.. NULL algorithm).

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Shally Verma <shally.verma@caviumnetworks.com>

doc: cleanup ISA-L PMD feature matrix

In PMD feature matrices (.ini files), it is not required to
have the list of features that are not supported,
just the ones that are.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Lee Daly <lee.daly@intel.com>

test/crypto: fix device id when stopping port

dev_id is valid, while ts_params->valid_devs[devid] may
result a different value in case multiple devices are present
and any of the device is being used.

Fixes: 202d375c60bc ("app/test: add cryptodev unit and performance tests")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/snow3g: add dynamic logging

Registered new dynamic logtype for driver and
replaced SNOW3G_LOG_ERR and CDEV_LOG_ERR
with the new SNOW3G_LOG macro, which uses the new logtype.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Reshma Pattan <reshma.pattan@intel.com>

security: fix crash on destroy null session

rte_security_session_destroy should return -EINVAL if session is NULL,
but segfaults because of rte_mempool_from_obj(NULL) call.

Fixes: c261d1431bd8 ("security: introduce security API and framework")
Cc: stable@dpdk.org
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/virtio: fix IV physical address

The physical address of IV required by Virtio was computed using
crypto operations' physical address plus the offset. However not
all crypto ops will have physical address field initialized and
compute it runtimely is costly. This patch fixes this problem by
adding iv field in virtio_crypto_op_cookie and does a memcpy of
iv instead.

Fixes: 82adb12a1fce ("crypto/virtio: support burst enqueue/dequeue")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Reviewed-by: Jay Zhou <jianjay.zhou@huawei.com>

common/qat: replace snprintf

Replaced snprintf with strlcpy for safer string copy

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>

security: change to SPDX license tags

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

doc: fix limitations for dpaa2 crypto

Fixes: 37f96eb01bce ("crypto/dpaa2_sec: support scatter gather")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

doc: fix limitations for dpaa crypto

Fixes: a74af788c632 ("crypto/dpaa_sec: support scatter gather")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

crypto/aesni_gcm: add dynamic logging

Registered new dynamic logtype for driver and
replaced GCM_LOG_ERR and CDEV_LOG_ERR
with the new AESNI_GCM_LOG macro, which uses the new logtype.

Signed-off-by: Chaitanya Babu Talluri <tallurix.chaitanya.babu@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/zuc: add dynamic logging

Registered new dynamic logtype for driver and
replaced ZUC_LOG_ERR and CDEV_LOG_ERR
with the new ZUC_LOG macro, which uses the new logtype.

Signed-off-by: Agalya Babu RadhaKrishnan <agalyax.babu.radhakrishnan@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/kasumi: add dynamic logging

Registered new dynamic logtype for driver and
replaced KASUMI_LOG_ERR and CDEV_LOG_ERR
with the new KASUMI_LOG macro, which uses the new logtype.

Signed-off-by: Naga Suresh Somarowthu <naga.sureshx.somarowthu@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/aesni_mb: add dynamic logging

Registered new dynamic logtype for driver and
replaced MB_LOG_ERR and CDEV_LOG_ERR
with the new AESNI_MB_LOG macro, which uses the new logtype.

Signed-off-by: Naga Suresh Somarowthu <naga.sureshx.somarowthu@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/null: add dynamic logging

Registered new dynamic logtype for driver and
replaced NULL_CRYPTO_LOG_ERR and CDEV_LOG_ERR
with the new NULL_LOG macro, which uses the new logtype.

Signed-off-by: Pallantla Poornima <pallantlax.poornima@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/openssl: add dynamic logging

Registered new dynamic logtype for driver and
replaced OPENSSL_LOG_ERR and CDEV_LOG_ERR macros with
new OPENSSL_LOG macro, which uses the new logtype.

Signed-off-by: Naga Suresh Somarowthu <naga.sureshx.somarowthu@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/scheduler: add dynamic logging

Registered new dynamic logtype for driver and
replaced CS_LOG_ERR and RTE_LOG macros with
new CR_SCHED macro, which uses the new logtype.

Signed-off-by: Hari Kumar <hari.kumarx.vemula@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

cryptodev: remove debug compilation option

For cryptodev dynamic logging, conditional compilation of
debug logs is not actually required.

Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
Reviewed-by: Reshma Pattan <reshma.pattan@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

crypto/qat: move common qat files to common dir

-  moved common qat files to common/qat dir.
  -  changed common/qat/Makefile, common/qat/meson.build,
     drivers/Makefile, crypto/Makefile
     to add possibility of using new files locations
  -  added README file into crypto/qat to clarify where
     the build is made from
  -  updated MAINTAINERS file

Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: re-organise build file content

This patch groups sources and related dependencies into
common and sym sections in build files.

Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

crypto/qat: add weak functions

This patch adds following weak functions to facilitate conditional
compilation of code for those services:
  -  qat_sym_dev_create
  -  qat_asym_dev_create
  -  qat_comp_dev_create
  -  qat_sym_dev_destroy
  -  qat_asym_dev_destroy
  -  qat_comp_dev_destroy
and removes unused files with empty definitions of above functions.

Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>

vhost: add notification for packed ring

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add Tx support for packed ring

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add Rx support for packed ring

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add vector filling support for packed ring

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: create descriptor mapping function

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add shadow used ring support for packed rings

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: append shadow used ring function names with split

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: extract split ring handling from Rx and Tx functions

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: clear batch copy index at copy time

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: make indirect desc table copy desc type agnostic

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: clear shadow used table index at flush time

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add vring address setup for packed queues

Add code to set up packed queues when enabled.

Signed-off-by: Yuanhan Liu <yliu@fridaylinux.org>
Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add helpers for packed virtqueues

Add some helper functions to check descriptor flags
and check if a vring is of type packed.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: add virtio packed virtqueue defines

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: improve prefetching in enqueue path

This is an optimization to prefetch next buffer while the
current one is being processed.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: prefetch first descriptor in dequeue path

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: improve prefetching in dequeue path

This is an optimization to prefetch next buffer while the
current one is being processed.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: use buffer vectors in dequeue path

To ease packed ring layout integration, this patch makes
the dequeue path to re-use buffer vectors implemented for
enqueue path.

Doing this, copy_desc_to_mbuf() is now ring layout type
agnostic.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

vhost: use shadow used ring in dequeue path

Relax used ring contention by reusing the shadow used
ring feature used by enqueue path.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>

doc: update CRC feature with new offload flag

Fixes: 3d12dceed2df ("ethdev: add new offload flag to keep CRC")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

examples: fix RSS hash function configuration

ethdev layer introduced checks for application requested RSS hash
functions and returns error for ones unsupported by hardware

This check breaks some sample applications which blindly configures
RSS hash functions without checking underlying hardware support.

Updated examples to mask out unsupported RSS has functions during device
configuration.
Prints a log if configuration values updated by this check.

Fixes: aa1a6d87f15d ("ethdev: force RSS offload rules again")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Tested-by: Meijuan Zhao <meijuanx.zhao@intel.com>
Tested-by: Yingya Han <yingyax.han@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>

net/bonding: do not clear active slave count

When the bond PMD is stopped, the active slave count is reset.
For 802.3ad mode this potentially leaks memory and clears state since
a second sequential activate_slave() will occur when the bond PMD is
restarted and the LSC callback is triggered while the active slave
count is 0. To fix this, don't clear the active slave count when
stopping. Only deactivate_slave() should be used to clear the slaves.

Fixes: 2efb58cbab6e ("bond: new link bonding library")
Cc: stable@dpdk.org
Signed-off-by: Chas Williams <chas3@att.com>

net/cxgbevf: add missing Tx byte counters

Fixes: a0a344a8f728 ("net/cxgbe: add VF port statistics")
Cc: stable@dpdk.org
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: fix Rx channel map and queue type

Set the Rx channel map and ingress queue type properly to allow firmware
to manage the internal mapping correctly.

Fixes: 6c2809628cd5 ("net/cxgbe: improve latency for slow traffic")
Cc: stable@dpdk.org
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: query firmware for max queues available

Query firmware for max Tx and Rx queues that can be allocated.
Move the code to determine max queues to common place for both
PF and VF.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: enable more RSS hash functions

Enable RSS on IPv4 fragmented packets and IPv6 packets with extension
headers based on 2-tuple hash.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: add link up and down ops

Add ops to set link up and down for both PF and VF. If wait_to_complete
is set, poll for link update for up to 10 seconds max.

Original work by Surendra Mobiya <surendra@chelsio.com>

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support to redirect packets to egress port

Add action to redirect matched packets to specified egress physical
port without sending them to host.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support to match on ingress physical port

Add support to match packets based on ingress physical port.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support to flush flows in HASH region

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: query hit counters for flows in HASH region

Add interface to enable hit counters for flows offloaded in HASH
region.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support to delete flows in HASH region

Add interface to delete offloaded flows in HASH region. Use the
hash index saved during insertion to delete the corresponding flow.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: support to offload flows to HASH region

Add interface to offload flows to HASH region. Translate internal
filter specification to requests to offload flows to HASH region.
Save the returned hash index of the offloaded flow for deletion later.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: add Compressed Local IP region

CLIP region holds destination IPv6 addresses to be matched for
corresponding flows. Query firmware for CLIP resources and allocate
table to manage them. Also update LE-TCAM to use CLIP to reduce
number of slots needed to offload IPv6 flows.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: validate flows offloaded to HASH region

Fetch supported match items in HASH region. Ensure the mask
is all set for all the supported match items to be offloaded
to HASH region. Otherwise, offload them to LE-TCAM region.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

net/cxgbe: query firmware for HASH filter resources

Fetch available HASH filter resources and allocate table for managing
them. Currently only supported on Chelsio T6 family of NICs.

Signed-off-by: Shagun Agrawal <shaguna@chelsio.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>

ethdev: remove old offload API

In DPDK 17.11, the ethdev offloads API has changed:
commit cba7f53b717d ("ethdev: introduce Tx queue offloads API")
commit ce17eddefc20 ("ethdev: introduce Rx queue offloads API")
The new API is documented in the programmer's guide:
http://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html#hardware-offload

For reminder, the main concepts in the new API were:
- All offloads are disabled by default
- Distinction between per port and per queue offloads.

The transition bits are now removed:
- Translation of the old API in ethdev
- rte_eth_conf.rxmode.ignore_offload_bitfield
- ETH_TXQ_FLAGS_IGNORE

The old API bits are now removed:
- Rx per-port rte_eth_conf.rxmode.[bit-fields]
- Tx per-queue rte_eth_txconf.txq_flags
- ETH_TXQ_FLAGS_NO*

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Shahaf Shuler <shahafs@mellanox.com>

net/fm10k: remove unused constant

The macro FM10K_SIMPLE_TX_FLAG was used with old Tx queue flags.
It is no longer used and was forgotten when cleaning old Tx flags.

Fixes: 1778ef67e2e7 ("net/fm10k: remove dependence on Tx queue flags")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

ethdev: convert remaining apps to new offload API

Some test applications and examples were not converted
to the new offload API introduced in 17.11.

For reference, see "Hardware Offload" in
doc/guides/prog_guide/poll_mode_drv.rst

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

test: remove unused configuration for bonding

The global variables rx_mode and fdir_conf
are not used in this test file.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

doc: remove code from KNI example guide

The example code is showing how to use KNI, and can be found in
examples/kni/
The documentation guide for this example is explaining the code
to ease the understanding of the example.
And inside this documentation, there are a lot of examples code
which are copy/pasted. It is really too much and hard to maintain.

The code inside this documentation is replaced by the name
of the functions.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

test/virtual_pmd: report CRC strip capability

Add DEV_RX_OFFLOAD_CRC_STRIP to virtual drivers since they don't
use CRC at all, when an application requires this offload virtual
PMDs should not return error.

Fixes: 3d12dceed2df ("ethdev: add new offload flag to keep CRC")

Signed-off-by: Phil Yang <phil.yang@arm.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/ark: use SPDX tags

Replace the BSD license header with the SPDX tag.

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/sfc: support Rx descriptor status for EF10 ESSB datapath

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

net/sfc: support ops to check ready descriptors in ESSB Rx

Implement EF10 ESSB Rx datapath callback to get number of pending
descriptors.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

net/sfc: fix double-free in EF10 ESSB Rx queue purge

Number of buffers left in completed descriptor may be 0. If so,
all buffers of the descriptor are freed once again.

Fixes: 390f9b8d82c9 ("net/sfc: support equal stride super-buffer Rx mode")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

net/sfc: discard packets with bad CRC on EF10 ESSB Rx

Fixes: 390f9b8d82c9 ("net/sfc: support equal stride super-buffer Rx mode")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>

net/pcap: capture only ingress packets from Rx iface

Support rx of in direction packets only
Useful for apps that also tx to eth_pcap ports in order to not see them
echoed back in as rx when out direction is also captured

Example:
In case using rx_iface and sending *single* packet to eth1
it will loop forever as the when it is sent to tx_iface=eth1
it will be captured again on the rx_iface=eth1 and so on
  $RTE_TARGET/app/testpmd l 0-3 -n 4 \
--vdev 'net_pcap0,rx_iface=eth1,tx_iface=eth1'
  …
  ---------------------- Forward statistics for port 0  ------------
  RX-packets: 758            RX-dropped: 0             RX-total: 758
  TX-packets: 758            TX-dropped: 0             TX-total: 758
  ------------------------------------------------------------------
While if using rx_iface_in it will not be captured on the way out and
be forwarded only once
  $RTE_TARGET/app/testpmd l 0-3 -n 4 \
--vdev 'net_pcap0,rx_iface_in=eth1,tx_iface=eth1'
  …
  ---------------------- Forward statistics for port 0  ------------
  RX-packets: 1              RX-dropped: 0             RX-total: 1
  TX-packets: 1              TX-dropped: 0             TX-total: 1
  ------------------------------------------------------------------

Signed-off-by: Ido Goshen <ido@cgstowernetworks.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

net/sfc: cut non VLAN ID bits from TCI

TCI may contain PCP or DEI bits. Matching of these bits is not
supported, but the bits still may be set in specification value and
not covered by mask. So, these bits should be ignored.

Fixes: 894080975e1e ("net/sfc: support VLAN in flow API filters")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Roman Zhukov <roman.zhukov@oktetlabs.ru>

ethdev: document xstats API assumption

Documents the assumption that 'xstats[i].id == i' and
key=xstats_names[i].name, value=xstats[i].value

xstats[i].id is still used for xstats _by_id() APIs.

This patch reverts some part of the commit 6d52d1d4afb4 ("ethdev:
clarify extended statistics documentation")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: David Marchand <david.marchand@6wind.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

net/virtio: improve offload check performance

Instead of checking the multiple Virtio features bits for
every packet, let's do the check once at configure time and
store it in virtio_hw struct.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>

net/virtio: do not use simple Rx if TCP LRO or VLAN strip

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>