-.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Intel Corporation nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2010-2015 Intel Corporation.
Link Bonding Poll Mode Driver Library
=====================================
In addition to Poll Mode Drivers (PMDs) for physical and virtual hardware,
-Intel® DPDK also includes a pure-software library that
-allows physical PMD's to be bonded together to create a single logical PMD.
+DPDK also includes a pure-software library that
+allows physical PMDs to be bonded together to create a single logical PMD.
+
+.. figure:: img/bond-overview.*
+
+ Bonded PMDs
-|bond-overview|
The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups of
-``rte_eth_dev`` ports of the same speed and duplex to provide
-similar the capabilities to that found in Linux bonding driver to allow the
-aggregation of multiple (slave) NICs into a single logical interface between a
-server and a switch. The new bonded PMD will then process these interfaces
-based on the mode of operation specified to provide support for features such
-as redundant links, fault tolerance and/or load balancing.
+``rte_eth_dev`` ports of the same speed and duplex to provide similar
+capabilities to that found in Linux bonding driver to allow the aggregation
+of multiple (slave) NICs into a single logical interface between a server
+and a switch. The new bonded PMD will then process these interfaces based on
+the mode of operation specified to provide support for features such as
+redundant links, fault tolerance and/or load balancing.
The librte_pmd_bond library exports a C API which provides an API for the
creation of bonded devices as well as the configuration and management of the
The Link Bonding PMD Library is enabled by default in the build
configuration files, the library can be disabled by setting
- ``CONFIG_RTE_LIBRTE_PMD_BOND=n`` and recompiling the Intel® DPDK.
+ ``CONFIG_RTE_LIBRTE_PMD_BOND=n`` and recompiling the DPDK.
Link Bonding Modes Overview
---------------------------
-Currently the Link Bonding PMD library supports 4 modes of operation:
+Currently the Link Bonding PMD library supports following modes of operation:
* **Round-Robin (Mode 0):**
-|bond-mode-0|
+.. figure:: img/bond-mode-0.*
+
+ Round-Robin (Mode 0)
+
This mode provides load balancing and fault tolerance by transmission of
packets in sequential order from the first available slave device through
* **Active Backup (Mode 1):**
-|bond-mode-1|
+.. figure:: img/bond-mode-1.*
+
+ Active Backup (Mode 1)
+
In this mode only one slave in the bond is active at any time, a different
slave becomes active if, and only if, the primary active slave fails,
* **Balance XOR (Mode 2):**
-|bond-mode-2|
+.. figure:: img/bond-mode-2.*
+
+ Balance XOR (Mode 2)
+
This mode provides transmit load balancing (based on the selected
transmission policy) and fault tolerance. The default policy (layer2) uses
a simple calculation based on the packet flow source and destination MAC
- addresses aswell as the number of active slaves available to the bonded
+ addresses as well as the number of active slaves available to the bonded
device to classify the packet to a specific slave to transmit on. Alternate
transmission policies supported are layer 2+3, this takes the IP source and
destination addresses into the calculation of the transmit slave port and
destination addresses as well as the TCP/UDP source and destination port.
.. note::
- The colouring differences of the packets are used to identify different flow
+ The coloring differences of the packets are used to identify different flow
classification calculated by the selected transmit policy
* **Broadcast (Mode 3):**
-|bond-mode-3|
+.. figure:: img/bond-mode-3.*
+
+ Broadcast (Mode 3)
+
This mode provides fault tolerance by transmission of packets on all slave
ports.
* **Link Aggregation 802.3AD (Mode 4):**
-|bond-mode-4|
+.. figure:: img/bond-mode-4.*
+
+ Link Aggregation 802.3AD (Mode 4)
+
This mode provides dynamic link aggregation according to the 802.3ad
specification. It negotiates and monitors aggregation groups that share the
* **Transmit Load Balancing (Mode 5):**
-|bond-mode-5|
+.. figure:: img/bond-mode-5.*
+
+ Transmit Load Balancing (Mode 5)
+
This mode provides an adaptive transmit load balancing. It dynamically
changes the transmitting slave, according to the computed load. Statistics
----------------------
The librte_pmd_bond bonded device are compatible with the Ethernet device API
-exported by the Ethernet PMDs described in the *Intel® DPDK API Reference*.
+exported by the Ethernet PMDs described in the *DPDK API Reference*.
The Link Bonding Library supports the creation of bonded devices at application
startup time during EAL initialization using the ``--vdev`` option as well as
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+Any flow which was configured to the bond device also is configured to the added
+slave.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more error-proof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+As RSS configurations, there is flow consistency in the bonded slaves for the
+next rte flow operations:
+
+Validate:
+ - Validate flow for each slave, failure at least for one slave causes to
+ bond validation failure.
+
+Create:
+ - Create the flow in all slaves.
+ - Save all the slaves created flows objects in bonding internal flow
+ structure.
+ - Failure in flow creation for existed slave rejects the flow.
+ - Failure in flow creation for new slaves in slave adding time rejects
+ the slave.
+
+Destroy:
+ - Destroy the flow in all slaves and release the bond internal flow
+ memory.
+
+Flush:
+ - Destroy all the bonding PMD flows in all the slaves.
+
+.. note::
+
+ Don't call slaves flush directly, It destroys all the slave flows which
+ may include external flows or the bond internal LACP flow.
+
+Query:
+ - Summarize flow counters from all the slaves, relevant only for
+ ``RTE_FLOW_ACTION_TYPE_COUNT``.
+
+Isolate:
+ - Call to flow isolate for all slaves.
+ - Failure in flow isolation for existed slave rejects the isolate mode.
+ - Failure in flow isolation for new slaves in slave adding time rejects
+ the slave.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).
Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
must register callbacks with that slave directly.
The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
A bonding device must have a minimum of one slave before the bonding device
itself can be started.
+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS and rte flow configurations should be managed through
+the bonding device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
Using Link Bonding Devices
--------------------------
-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
Link bonding devices can be created at application startup time using the
``--vdev`` EAL command line option. The device name must start with the
-eth_bond prefix followed by numbers or letters. The name must be unique for
+net_bonding prefix followed by numbers or letters. The name must be unique for
each device. Each device can have multiple options arranged in a comma
separated list. Multiple devices definitions can be arranged by calling the
``--vdev`` option multiple times.
.. code-block:: console
- $RTE_TARGET/app/testpmd -c f -n 4 --vdev 'eth_bond0,bond_opt0=..,bond opt1=..'--vdev 'eth_bond1,bond _opt0=..,bond_opt1=..'
+ $RTE_TARGET/app/testpmd -l 0-3 -n 4 --vdev 'net_bonding0,bond_opt0=..,bond opt1=..'--vdev 'net_bonding1,bond _opt0=..,bond_opt1=..'
Link Bonding EAL Options
^^^^^^^^^^^^^^^^^^^^^^^^
There are multiple ways of definitions that can be assessed and combined as
long as the following two rules are respected:
-* A unique device name, in the format of eth_bondX is provided,
+* A unique device name, in the format of net_bondingX is provided,
where X can be any combination of numbers and/or letters,
and the name is no greater than 32 characters long.
mode=2
* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
.. code-block:: console
- $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=0, slave=0000:00a:00.01,slave=0000:004:00.00' -- --port-topology=chained
+ $RTE_TARGET/app/testpmd -l 0-3 -n 4 --vdev 'net_bonding0,mode=0, slave=0000:00a:00.01,slave=0000:004:00.00' -- --port-topology=chained
Create a bonded device in round robin mode with two slaves specified by their PCI address and an overriding MAC address:
.. code-block:: console
- $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=0, slave=0000:00a:00.01,slave=0000:004:00.00,mac=00:1e:67:1d:fd:1d' -- --port-topology=chained
+ $RTE_TARGET/app/testpmd -l 0-3 -n 4 --vdev 'net_bonding0,mode=0, slave=0000:00a:00.01,slave=0000:004:00.00,mac=00:1e:67:1d:fd:1d' -- --port-topology=chained
Create a bonded device in active backup mode with two slaves specified, and a primary slave specified by their PCI addresses:
.. code-block:: console
- $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=1, slave=0000:00a:00.01,slave=0000:004:00.00,primary=0000:00a:00.01' -- --port-topology=chained
+ $RTE_TARGET/app/testpmd -l 0-3 -n 4 --vdev 'net_bonding0,mode=1, slave=0000:00a:00.01,slave=0000:004:00.00,primary=0000:00a:00.01' -- --port-topology=chained
Create a bonded device in balance mode with two slaves specified by their PCI addresses, and a transmission policy of layer 3 + 4 forwarding:
.. code-block:: console
- $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=2, slave=0000:00a:00.01,slave=0000:004:00.00,xmit_policy=l34' -- --port-topology=chained
-
-.. |bond-overview| image:: img/bond-overview.svg
-.. |bond-mode-0| image:: img/bond-mode-0.svg
-.. |bond-mode-1| image:: img/bond-mode-1.svg
-.. |bond-mode-2| image:: img/bond-mode-2.svg
-.. |bond-mode-3| image:: img/bond-mode-3.svg
-.. |bond-mode-4| image:: img/bond-mode-4.svg
-.. |bond-mode-5| image:: img/bond-mode-5.svg
+ $RTE_TARGET/app/testpmd -l 0-3 -n 4 --vdev 'net_bonding0,mode=2, slave=0000:00a:00.01,slave=0000:004:00.00,xmit_policy=l34' -- --port-topology=chained
git.droids-corp.org / dpdk.git / blobdiff