From: John Daley Date: Tue, 11 May 2021 19:25:26 +0000 (-0700) Subject: net/enic: enable GENEVE offload via VNIC configuration X-Git-Url: http://git.droids-corp.org/?a=commitdiff_plain;h=61c7b522d9062242d648d09b61d5137740bc2ebb;p=dpdk.git net/enic: enable GENEVE offload via VNIC configuration The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include Geneve offload. Use that setting to decide whether to enable or disable Geneve offload and remove the devarg 'geneve-opt'. Also, the firmware now allows the driver to change the Geneve port number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in addition to VXLAN. Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE") Cc: stable@dpdk.org Signed-off-by: John Daley Reviewed-by: Hyong Youb Kim --- diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 4e7629c5cd..91bdcd065a 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -294,35 +294,31 @@ inner and outer packets can be IPv4 or IPv6. RSS hash calculation, therefore queue selection, is done on inner packets. -In order to enable overlay offload, the 'Enable VXLAN' box should be checked +In order to enable overlay offload, enable VXLAN and/or Geneve on vNIC via CIMC or UCSM followed by a reboot of the server. When PMD successfully -enables overlay offload, it prints the following message on the console. +enables overlay offload, it prints one of the following messages on the console. .. code-block:: console - Overlay offload is enabled + Overlay offload is enabled (VxLAN) + Overlay offload is enabled (Geneve) + Overlay offload is enabled (VxLAN, Geneve) By default, PMD enables overlay offload if hardware supports it. To disable it, set ``devargs`` parameter ``disable-overlay=1``. For example:: -a 12:00.0,disable-overlay=1 -By default, the NIC uses 4789 as the VXLAN port. The user may change -it through ``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as -the current NIC has a single VXLAN port number, the user cannot -configure multiple port numbers. - -Geneve headers with non-zero options are not supported by default. To -use Geneve with options, update the VIC firmware to the latest version -and then set ``devargs`` parameter ``geneve-opt=1``. When Geneve with -options is enabled, flow API cannot be used as the features are -currently mutually exclusive. When this feature is successfully -enabled, PMD prints the following message. - -.. code-block:: console - - Geneve with options is enabled +By default, the NIC uses 4789 and 6081 as the VXLAN and Geneve ports, +respectively. The user may change them through +``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as the current +NIC has a single VXLAN port number and a single Geneve port number, +the user cannot configure multiple port numbers for each tunnel type. +Geneve offload support has evolved over VIC models. On older models, +Geneve offload and advanced filters are mutually exclusive. This is +enforced by UCSM and CIMC, which only allow one of the two features +to be selected at one time. Newer VIC models do not have this restriction. Ingress VLAN Rewrite -------------------- diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 985c845de4..b59576a575 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -121,6 +121,13 @@ New Features * Added flow counters to extended stats. * Added PCI function stats to extended stats. +* **Updated Cisco enic driver.** + + Updated Cisco enic driver GENEVE tunneling support: + + * Added support to control GENEVE tunneling via UCSM/CIMC and removed devarg. + * Added GENEVE port number configuration. + * **Updated Hisilicon hns3 driver.** Updated Hisilicon hns3 driver with new features and improvements, including: diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index 526273cef5..55c08eb3dc 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -1318,7 +1318,7 @@ int vnic_dev_capable_geneve(struct vnic_dev *vdev) int ret; ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait); - return ret == 0 && (a1 & FEATURE_GENEVE_OPTIONS); + return ret == 0 && !!(a1 & FEATURE_GENEVE_OPTIONS); } uint64_t vnic_dev_capable_cq_entry_size(struct vnic_dev *vdev) diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h index 7687951c90..2a97a33044 100644 --- a/drivers/net/enic/base/vnic_enet.h +++ b/drivers/net/enic/base/vnic_enet.h @@ -55,6 +55,7 @@ struct vnic_enet_config { #define VENETF_NICSWITCH 0x80000 /* NICSWITCH enabled */ #define VENETF_RSSHASH_UDPIPV4 0x100000 /* Hash on UDP + IPv4 fields */ #define VENETF_RSSHASH_UDPIPV6 0x200000 /* Hash on UDP + IPv6 fields */ +#define VENETF_GENEVE 0x400000 /* GENEVE offload */ #define VENET_INTR_TYPE_MIN 0 /* Timer specs min interrupt spacing */ #define VENET_INTR_TYPE_IDLE 1 /* Timer specs idle time before irq */ diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h index cd66348f2f..47bfdac2cf 100644 --- a/drivers/net/enic/enic.h +++ b/drivers/net/enic/enic.h @@ -118,17 +118,17 @@ struct enic { uint32_t flow_filter_mode; uint8_t filter_actions; /* HW supported actions */ uint64_t cq_entry_sizes; /* supported CQ entry sizes */ + bool geneve; bool vxlan; bool cq64; /* actually using 64B CQ entry */ bool cq64_request; /* devargs cq64=1 */ bool disable_overlay; /* devargs disable_overlay=1 */ uint8_t enable_avx2_rx; /* devargs enable-avx2-rx=1 */ - uint8_t geneve_opt_avail; /* Geneve with options offload available */ - uint8_t geneve_opt_enabled; /* Geneve with options offload enabled */ uint8_t geneve_opt_request; /* devargs geneve-opt=1 */ bool nic_cfg_chk; /* NIC_CFG_CHK available */ bool udp_rss_weak; /* Bodega style UDP RSS */ uint8_t ig_vlan_rewrite_mode; /* devargs ig-vlan-rewrite */ + uint16_t geneve_port; /* current geneve port pushed to NIC */ uint16_t vxlan_port; /* current vxlan port pushed to NIC */ int use_simple_tx_handler; int use_noscatter_vec_rx_handler; diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index ab64480f39..8d5797523b 100644 --- a/drivers/net/enic/enic_ethdev.c +++ b/drivers/net/enic/enic_ethdev.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -67,7 +68,6 @@ static const struct vic_speed_capa { #define ENIC_DEVARG_CQ64 "cq64" #define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay" #define ENIC_DEVARG_ENABLE_AVX2_RX "enable-avx2-rx" -#define ENIC_DEVARG_GENEVE_OPT "geneve-opt" #define ENIC_DEVARG_IG_VLAN_REWRITE "ig-vlan-rewrite" #define ENIC_DEVARG_REPRESENTOR "representor" @@ -81,13 +81,6 @@ enicpmd_dev_flow_ops_get(struct rte_eth_dev *dev, ENICPMD_FUNC_TRACE(); - /* - * Currently, when Geneve with options offload is enabled, host - * cannot insert match-action rules. - */ - if (enic->geneve_opt_enabled) - return -ENOTSUP; - if (enic->flow_filter_mode == FILTER_FLOWMAN) *ops = &enic_fm_flow_ops; else @@ -972,26 +965,32 @@ static int enicpmd_dev_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, static int udp_tunnel_common_check(struct enic *enic, struct rte_eth_udp_tunnel *tnl) { - if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN) + if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN && + tnl->prot_type != RTE_TUNNEL_TYPE_GENEVE) return -ENOTSUP; if (!enic->overlay_offload) { - ENICPMD_LOG(DEBUG, " vxlan (overlay offload) is not " - "supported\n"); + ENICPMD_LOG(DEBUG, " overlay offload is not supported\n"); return -ENOTSUP; } return 0; } -static int update_vxlan_port(struct enic *enic, uint16_t port) +static int update_tunnel_port(struct enic *enic, uint16_t port, bool vxlan) { - if (vnic_dev_overlay_offload_cfg(enic->vdev, - OVERLAY_CFG_VXLAN_PORT_UPDATE, - port)) { - ENICPMD_LOG(DEBUG, " failed to update vxlan port\n"); + uint8_t cfg; + + cfg = vxlan ? OVERLAY_CFG_VXLAN_PORT_UPDATE : + OVERLAY_CFG_GENEVE_PORT_UPDATE; + if (vnic_dev_overlay_offload_cfg(enic->vdev, cfg, port)) { + ENICPMD_LOG(DEBUG, " failed to update tunnel port\n"); return -EINVAL; } - ENICPMD_LOG(DEBUG, " updated vxlan port to %u\n", port); - enic->vxlan_port = port; + ENICPMD_LOG(DEBUG, " updated %s port to %u\n", + vxlan ? "vxlan" : "geneve", port); + if (vxlan) + enic->vxlan_port = port; + else + enic->geneve_port = port; return 0; } @@ -999,34 +998,48 @@ static int enicpmd_dev_udp_tunnel_port_add(struct rte_eth_dev *eth_dev, struct rte_eth_udp_tunnel *tnl) { struct enic *enic = pmd_priv(eth_dev); + uint16_t port; + bool vxlan; int ret; ENICPMD_FUNC_TRACE(); ret = udp_tunnel_common_check(enic, tnl); if (ret) return ret; + vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN); + if (vxlan) + port = enic->vxlan_port; + else + port = enic->geneve_port; /* - * The NIC has 1 configurable VXLAN port number. "Adding" a new port - * number replaces it. + * The NIC has 1 configurable port number per tunnel type. + * "Adding" a new port number replaces it. */ - if (tnl->udp_port == enic->vxlan_port || tnl->udp_port == 0) { + if (tnl->udp_port == port || tnl->udp_port == 0) { ENICPMD_LOG(DEBUG, " %u is already configured or invalid\n", tnl->udp_port); return -EINVAL; } - return update_vxlan_port(enic, tnl->udp_port); + return update_tunnel_port(enic, tnl->udp_port, vxlan); } static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev, struct rte_eth_udp_tunnel *tnl) { struct enic *enic = pmd_priv(eth_dev); + uint16_t port; + bool vxlan; int ret; ENICPMD_FUNC_TRACE(); ret = udp_tunnel_common_check(enic, tnl); if (ret) return ret; + vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN); + if (vxlan) + port = enic->vxlan_port; + else + port = enic->geneve_port; /* * Clear the previously set port number and restore the * hardware default port number. Some drivers disable VXLAN @@ -1034,12 +1047,13 @@ static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev, * enic does not do that as VXLAN is part of overlay offload, * which is tied to inner RSS and TSO. */ - if (tnl->udp_port != enic->vxlan_port) { - ENICPMD_LOG(DEBUG, " %u is not a configured vxlan port\n", + if (tnl->udp_port != port) { + ENICPMD_LOG(DEBUG, " %u is not a configured tunnel port\n", tnl->udp_port); return -EINVAL; } - return update_vxlan_port(enic, RTE_VXLAN_DEFAULT_PORT); + port = vxlan ? RTE_VXLAN_DEFAULT_PORT : RTE_GENEVE_DEFAULT_PORT; + return update_tunnel_port(enic, port, vxlan); } static int enicpmd_dev_fw_version_get(struct rte_eth_dev *eth_dev, @@ -1145,8 +1159,6 @@ static int enic_parse_zero_one(const char *key, enic->disable_overlay = b; if (strcmp(key, ENIC_DEVARG_ENABLE_AVX2_RX) == 0) enic->enable_avx2_rx = b; - if (strcmp(key, ENIC_DEVARG_GENEVE_OPT) == 0) - enic->geneve_opt_request = b; return 0; } @@ -1189,7 +1201,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev) ENIC_DEVARG_CQ64, ENIC_DEVARG_DISABLE_OVERLAY, ENIC_DEVARG_ENABLE_AVX2_RX, - ENIC_DEVARG_GENEVE_OPT, ENIC_DEVARG_IG_VLAN_REWRITE, ENIC_DEVARG_REPRESENTOR, NULL}; @@ -1201,7 +1212,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev) enic->cq64_request = true; /* Use 64B entry if available */ enic->disable_overlay = false; enic->enable_avx2_rx = false; - enic->geneve_opt_request = false; enic->ig_vlan_rewrite_mode = IG_VLAN_REWRITE_MODE_PASS_THRU; if (!dev->device->devargs) return 0; @@ -1214,8 +1224,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev) enic_parse_zero_one, enic) < 0 || rte_kvargs_process(kvlist, ENIC_DEVARG_ENABLE_AVX2_RX, enic_parse_zero_one, enic) < 0 || - rte_kvargs_process(kvlist, ENIC_DEVARG_GENEVE_OPT, - enic_parse_zero_one, enic) < 0 || rte_kvargs_process(kvlist, ENIC_DEVARG_IG_VLAN_REWRITE, enic_parse_ig_vlan_rewrite, enic) < 0) { rte_kvargs_free(kvlist); @@ -1391,5 +1399,4 @@ RTE_PMD_REGISTER_PARAM_STRING(net_enic, ENIC_DEVARG_CQ64 "=0|1" ENIC_DEVARG_DISABLE_OVERLAY "=0|1 " ENIC_DEVARG_ENABLE_AVX2_RX "=0|1 " - ENIC_DEVARG_GENEVE_OPT "=0|1 " ENIC_DEVARG_IG_VLAN_REWRITE "=trunk|untag|priority|pass"); diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index 2a06d46872..2affd380c6 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "enic_compat.h" #include "enic.h" @@ -1719,6 +1720,85 @@ set_mtu_done: return rc; } +static void +enic_disable_overlay_offload(struct enic *enic) +{ + /* + * Disabling fails if the feature is provisioned but + * not enabled. So ignore result and do not log error. + */ + if (enic->vxlan) { + vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_DISABLE); + } + if (enic->geneve) { + vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_DISABLE); + } +} + +static int +enic_enable_overlay_offload(struct enic *enic) +{ + if (enic->vxlan && vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_ENABLE) != 0) { + dev_err(NULL, "failed to enable VXLAN offload\n"); + return -EINVAL; + } + if (enic->geneve && vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_ENABLE) != 0) { + dev_err(NULL, "failed to enable Geneve offload\n"); + return -EINVAL; + } + enic->tx_offload_capa |= + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | + (enic->geneve ? DEV_TX_OFFLOAD_GENEVE_TNL_TSO : 0) | + (enic->vxlan ? DEV_TX_OFFLOAD_VXLAN_TNL_TSO : 0); + enic->tx_offload_mask |= + PKT_TX_OUTER_IPV6 | + PKT_TX_OUTER_IPV4 | + PKT_TX_OUTER_IP_CKSUM | + PKT_TX_TUNNEL_MASK; + enic->overlay_offload = true; + + if (enic->vxlan && enic->geneve) + dev_info(NULL, "Overlay offload is enabled (VxLAN, Geneve)\n"); + else if (enic->vxlan) + dev_info(NULL, "Overlay offload is enabled (VxLAN)\n"); + else + dev_info(NULL, "Overlay offload is enabled (Geneve)\n"); + + return 0; +} + +static int +enic_reset_overlay_port(struct enic *enic) +{ + if (enic->vxlan) { + enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT; + /* + * Reset the vxlan port to the default, as the NIC firmware + * does not reset it automatically and keeps the old setting. + */ + if (vnic_dev_overlay_offload_cfg(enic->vdev, + OVERLAY_CFG_VXLAN_PORT_UPDATE, + RTE_VXLAN_DEFAULT_PORT)) { + dev_err(enic, "failed to update vxlan port\n"); + return -EINVAL; + } + } + if (enic->geneve) { + enic->geneve_port = RTE_GENEVE_DEFAULT_PORT; + if (vnic_dev_overlay_offload_cfg(enic->vdev, + OVERLAY_CFG_GENEVE_PORT_UPDATE, + RTE_GENEVE_DEFAULT_PORT)) { + dev_err(enic, "failed to update vxlan port\n"); + return -EINVAL; + } + } + return 0; +} + static int enic_dev_init(struct enic *enic) { int err; @@ -1785,85 +1865,32 @@ static int enic_dev_init(struct enic *enic) /* set up link status checking */ vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */ + enic->overlay_offload = false; /* - * When Geneve with options offload is available, always disable it - * first as it can interfere with user flow rules. + * First, explicitly disable overlay offload as the setting is + * sticky, and resetting vNIC may not disable it. */ - if (enic->geneve_opt_avail) { - /* - * Disabling fails if the feature is provisioned but - * not enabled. So ignore result and do not log error. - */ - vnic_dev_overlay_offload_ctrl(enic->vdev, - OVERLAY_FEATURE_GENEVE, - OVERLAY_OFFLOAD_DISABLE); - } - enic->overlay_offload = false; - if (enic->disable_overlay && enic->vxlan) { - /* - * Explicitly disable overlay offload as the setting is - * sticky, and resetting vNIC does not disable it. - */ - if (vnic_dev_overlay_offload_ctrl(enic->vdev, - OVERLAY_FEATURE_VXLAN, - OVERLAY_OFFLOAD_DISABLE)) { - dev_err(enic, "failed to disable overlay offload\n"); - } else { - dev_info(enic, "Overlay offload is disabled\n"); - } - } - if (!enic->disable_overlay && enic->vxlan && - /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */ - vnic_dev_overlay_offload_ctrl(enic->vdev, - OVERLAY_FEATURE_VXLAN, - OVERLAY_OFFLOAD_ENABLE) == 0) { - enic->tx_offload_capa |= - DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | - DEV_TX_OFFLOAD_GENEVE_TNL_TSO | - DEV_TX_OFFLOAD_VXLAN_TNL_TSO; - enic->tx_offload_mask |= - PKT_TX_OUTER_IPV6 | - PKT_TX_OUTER_IPV4 | - PKT_TX_OUTER_IP_CKSUM | - PKT_TX_TUNNEL_MASK; - enic->overlay_offload = true; - dev_info(enic, "Overlay offload is enabled\n"); - } - /* Geneve with options offload requires overlay offload */ - if (enic->overlay_offload && enic->geneve_opt_avail && - enic->geneve_opt_request) { - if (vnic_dev_overlay_offload_ctrl(enic->vdev, - OVERLAY_FEATURE_GENEVE, - OVERLAY_OFFLOAD_ENABLE)) { - dev_err(enic, "failed to enable geneve+option\n"); - } else { - enic->geneve_opt_enabled = 1; - dev_info(enic, "Geneve with options is enabled\n"); + enic_disable_overlay_offload(enic); + /* Then, enable overlay offload according to vNIC flags */ + if (!enic->disable_overlay && (enic->vxlan || enic->geneve)) { + err = enic_enable_overlay_offload(enic); + if (err) { + dev_info(NULL, "failed to enable overlay offload\n"); + return err; } } /* - * Reset the vxlan port if HW vxlan parsing is available. It + * Reset the vxlan/geneve port if HW parsing is available. It * is always enabled regardless of overlay offload * enable/disable. */ - if (enic->vxlan) { - enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT; - /* - * Reset the vxlan port to the default, as the NIC firmware - * does not reset it automatically and keeps the old setting. - */ - if (vnic_dev_overlay_offload_cfg(enic->vdev, - OVERLAY_CFG_VXLAN_PORT_UPDATE, - RTE_VXLAN_DEFAULT_PORT)) { - dev_err(enic, "failed to update vxlan port\n"); - return -EINVAL; - } - } + err = enic_reset_overlay_port(enic); + if (err) + return err; if (enic_fm_init(enic)) dev_warning(enic, "Init of flowman failed.\n"); return 0; - } static void lock_devcmd(void *priv) diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index 689bf748ae..a8f5332a40 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -179,10 +179,9 @@ int enic_get_vnic_config(struct enic *enic) enic->vxlan = ENIC_SETTING(enic, VXLAN) && vnic_dev_capable_vxlan(enic->vdev); - if (vnic_dev_capable_geneve(enic->vdev)) { - dev_info(NULL, "Geneve with options offload available\n"); - enic->geneve_opt_avail = 1; - } + enic->geneve = ENIC_SETTING(enic, GENEVE) && + vnic_dev_capable_geneve(enic->vdev); + /* Supported CQ entry sizes */ enic->cq_entry_sizes = vnic_dev_capable_cq_entry_size(enic->vdev); sizes = enic->cq_entry_sizes;