-.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Intel Corporation nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2010-2014 Intel Corporation.
Quality of Service (QoS) Framework
==================================
| | | |
+---+------------------------+--------------------------------------------------------------------------------+
| 7 | Dropper | Congestion management using the Random Early Detection (RED) algorithm |
- | | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED). |
+ | | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED) |
+ | | | or Proportional Integral Controller Enhanced (PIE). |
| | | Drop packets based on the current scheduler queue load level and packet |
| | | priority. When congestion is experienced, lower priority packets are dropped |
| | | first. |
| | | | token bucket per pipe. |
| | | | |
+---+--------------------+----------------------------+---------------------------------------------------------------+
- | 4 | Traffic Class (TC) | 4 | #. TCs of the same pipe handled in strict priority order. |
+ | 4 | Traffic Class (TC) | 13 | #. TCs of the same pipe handled in strict priority order. |
| | | | |
| | | | #. Upper limit enforced per TC at the pipe level. |
| | | | |
| | | | adjusted value that is shared by all the subport pipes. |
| | | | |
+---+--------------------+----------------------------+---------------------------------------------------------------+
- | 5 | Queue | 4 | #. Queues of the same TC are serviced using Weighted Round |
- | | | | Robin (WRR) according to predefined weights. |
+ | 5 | Queue | High priority TCs: 1, | #. All the high priority TCs (TC0, TC1, ...,TC11) have |
+ | | | Lowest priority TC: 4 | exactly 1 queue, while the lowest priority TC (TC12), |
+ | | | | called Best Effort (BE), has 4 queues. |
+ | | | | |
+ | | | | #. Queues of the lowest priority TC (BE) are serviced using |
+ | | | | Weighted Round Robin (WRR) according to predefined weights|
+ | | | | weights. |
| | | | |
+---+--------------------+----------------------------+---------------------------------------------------------------+
The congestion management scheme implemented by the enqueue pipeline described above is very basic:
packets are enqueued until a specific queue becomes full,
then all the packets destined to the same queue are dropped until packets are consumed (by the dequeue operation).
-This can be improved by enabling RED/WRED as part of the enqueue pipeline which looks at the queue occupancy and
+This can be improved by enabling RED/WRED or PIE as part of the enqueue pipeline which looks at the queue occupancy and
packet priority in order to yield the enqueue/drop decision for a specific packet
(as opposed to enqueuing all packets / dropping all packets indiscriminately).
Strict priority scheduling of traffic classes within the same pipe is implemented by the pipe dequeue state machine,
which selects the queues in ascending order.
-Therefore, queues 0..3 (associated with TC 0, highest priority TC) are handled before
-queues 4..7 (TC 1, lower priority than TC 0),
-which are handled before queues 8..11 (TC 2),
-which are handled before queues 12..15 (TC 3, lowest priority TC).
+Therefore, queue 0 (associated with TC 0, highest priority TC) is handled before
+queue 1 (TC 1, lower priority than TC 0),
+which is handled before queue 2 (TC 2, lower priority than TC 1) and it continues until queues of all TCs except the
+lowest priority TC are handled. At last, queues 12..15 (best effort TC, lowest priority TC) are handled.
Upper Limit Enforcement
'''''''''''''''''''''''
| # | Subport or pipe field | Unit | Description |
| | | | |
+===+=======================+=======+=======================================================================+
- | 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the 4 TCs of the |
+ | 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the TCs of the |
| | | | current subport / pipe. |
| | | | |
| | | | See Section `Internal Time Reference`_ for the |
| | | | explanation of why the time is maintained in byte units. |
| | | | |
+---+-----------------------+-------+-----------------------------------------------------------------------+
- | 2 | tc_period | Bytes | Time between two consecutive updates for the 4 TCs of the current |
+ | 2 | tc_period | Bytes | Time between two consecutive updates for the all TCs of the current |
| | | | subport / pipe. This is expected to be many times bigger than the |
| | | | typical value of the token bucket tb_period. |
| | | | |
Weighted Round Robin (WRR)
""""""""""""""""""""""""""
-The evolution of the WRR design solution from simple to complex is shown in :numref:`table_qos_12`.
+The evolution of the WRR design solution for the lowest priority traffic class (best effort TC) from simple to complex is shown in :numref:`table_qos_12`.
.. _table_qos_12:
| | | |
+-----+---------------------------+-------------------------------------------------------------------------+
-Typically, the subport TC oversubscription feature is enabled only for the lowest priority traffic class (TC 3),
+Typically, the subport TC oversubscription feature is enabled only for the lowest priority traffic class,
which is typically used for best effort traffic,
with the management plane preventing this condition from occurring for the other (higher priority) traffic classes.
-To ease implementation, it is also assumed that the upper limit for subport TC 3 is set to 100% of the subport rate,
-and that the upper limit for pipe TC 3 is set to 100% of pipe rate for all subport member pipes.
+To ease implementation, it is also assumed that the upper limit for subport best effort TC is set to 100% of the subport rate,
+and that the upper limit for pipe best effort TC is set to 100% of pipe rate for all subport member pipes.
Implementation Overview
'''''''''''''''''''''''
The algorithm computes a watermark, which is periodically updated based on the current demand experienced by the subport member pipes,
-whose purpose is to limit the amount of traffic that each pipe is allowed to send for TC 3.
+whose purpose is to limit the amount of traffic that each pipe is allowed to send for best effort TC.
The watermark is computed at the subport level at the beginning of each traffic class upper limit enforcement period and
the same value is used by all the subport member pipes throughout the current enforcement period.
illustrates how the watermark computed as subport level at the beginning of each period is propagated to all subport member pipes.
At the beginning of the current enforcement period (which coincides with the end of the previous enforcement period),
-the value of the watermark is adjusted based on the amount of bandwidth allocated to TC 3 at the beginning of the previous period that
+the value of the watermark is adjusted based on the amount of bandwidth allocated to best effort TC at the beginning of the previous period that
was not left unused by the subport member pipes at the end of the previous period.
-If there was subport TC 3 bandwidth left unused,
+If there was subport best effort TC bandwidth left unused,
the value of the watermark for the current period is increased to encourage the subport member pipes to consume more bandwidth.
-Otherwise, the value of the watermark is decreased to enforce equality of bandwidth consumption among subport member pipes for TC 3.
+Otherwise, the value of the watermark is decreased to enforce equality of bandwidth consumption among subport member pipes for best effort TC.
The increase or decrease in the watermark value is done in small increments,
so several enforcement periods might be required to reach the equilibrium state.
-This state can change at any moment due to variations in the demand experienced by the subport member pipes for TC 3, for example,
+This state can change at any moment due to variations in the demand experienced by the subport member pipes for best effort TC, for example,
as a result of demand increase (when the watermark needs to be lowered) or demand decrease (when the watermark needs to be increased).
When demand is low, the watermark is set high to prevent it from impeding the subport member pipes from consuming more bandwidth.
| | | |
| | | tc3_cons = subport_tc3_credits_per_period - subport_tc3_credits; |
| | | |
- | | | tc3_cons_max = subport_tc3_credits_per_period - (tc0_cons + tc1_cons + |
- | | | tc2_cons); |
+ | | | tc4_cons = subport_tc4_credits_per_period - subport_tc4_credits; |
+ | | | |
+ | | | tc5_cons = subport_tc5_credits_per_period - subport_tc5_credits; |
+ | | | |
+ | | | tc6_cons = subport_tc6_credits_per_period - subport_tc6_credits; |
+ | | | |
+ | | | tc7_cons = subport_tc7_credits_per_period - subport_tc7_credits; |
+ | | | |
+ | | | tc8_cons = subport_tc8_credits_per_period - subport_tc8_credits; |
+ | | | |
+ | | | tc9_cons = subport_tc9_credits_per_period - subport_tc9_credits; |
+ | | | |
+ | | | tc10_cons = subport_tc10_credits_per_period - subport_tc10_credits; |
+ | | | |
+ | | | tc11_cons = subport_tc11_credits_per_period - subport_tc11_credits; |
| | | |
- | | | if(tc3_consumption > (tc3_consumption_max - MTU)){ |
+ | | | tc_be_cons_max = subport_tc_be_credits_per_period - (tc0_cons + tc1_cons + |
+ | | | tc2_cons + tc3_cons + tc4_cons + tc5_cons + tc6_cons + tc7_cons + tc8_cons + |
+ | | | tc9_cons + tc10_cons + tc11_cons); |
+ | | | |
+ | | | if(tc_be_consumption > (tc_be_consumption_max - MTU)){ |
| | | |
| | | wm -= wm >> 7; |
| | | |
then the performance of the port scheduler for the same level of active traffic is expected to be worse than
the performance of a small set of message passing queues.
-.. _Dropper:
+.. _Droppers:
-Dropper
--------
+Droppers
+--------
The purpose of the DPDK dropper is to drop packets arriving at a packet scheduler to avoid congestion.
-The dropper supports the Random Early Detection (RED),
+The dropper supports the Proportional Integral Controller Enhanced (PIE), Random Early Detection (RED),
Weighted Random Early Detection (WRED) and tail drop algorithms.
:numref:`figure_blk_diag_dropper` illustrates how the dropper integrates with the scheduler.
The DPDK currently does not support congestion management
High-level Block Diagram of the DPDK Dropper
-The dropper uses the Random Early Detection (RED) congestion avoidance algorithm as documented in the reference publication.
-The purpose of the RED algorithm is to monitor a packet queue,
+The dropper uses one of two congestion avoidance algorithms:
+ - the Random Early Detection (RED) as documented in the reference publication.
+ - the Proportional Integral Controller Enhanced (PIE) as documented in RFC8033 publication.
+
+The purpose of the RED/PIE algorithm is to monitor a packet queue,
determine the current congestion level in the queue and decide whether an arriving packet should be enqueued or dropped.
+
The RED algorithm uses an Exponential Weighted Moving Average (EWMA) filter to compute average queue size which
gives an indication of the current congestion level in the queue.
In this situation, all arriving packets are dropped.
The flow through the dropper is illustrated in :numref:`figure_flow_tru_droppper`.
-The RED/WRED algorithm is exercised first and tail drop second.
+The RED/WRED/PIE algorithm is exercised first and tail drop second.
.. _figure_flow_tru_droppper:
Flow Through the Dropper
+The PIE algorithm periodically updates the drop probability based on the latency samples.
+The current latency sample but also analyze whether the latency is trending up or down.
+This is the classical Proportional Integral (PI) controller method, which is known for
+eliminating steady state errors.
+
+When a congestion period ends, we might be left with a high drop probability with light
+packet arrivals. Hence, the PIE algorithm includes a mechanism by which the drop probability
+decays exponentially (rather than linearly) when the system is not congested.
+This would help the drop probability converge to 0 more quickly, while the PI controller ensures
+that it would eventually reach zero.
The use cases supported by the dropper are:
The EWMA filter weight parameter is specified as an inverse log value,
for example, a filter weight parameter value of 9 corresponds to a filter weight of 1/29.
+A PIE configuration contains the parameters given in :numref:`table_qos_16a`.
+
+.. _table_qos_16a:
+
+.. table:: PIE Configuration Parameters
+
+ +--------------------------+---------+---------+------------------+
+ | Parameter | Minimum | Maximum | Default |
+ | | | | |
+ +==========================+=========+=========+==================+
+ | Queue delay reference | 1 | uint16 | 15 |
+ | Latency Target Value | | | |
+ | Unit: ms | | | |
+ +--------------------------+---------+---------+------------------+
+ | Max Burst Allowance | 1 | uint16 | 150 |
+ | Unit: ms | | | |
+ +--------------------------+---------+---------+------------------+
+ | Tail Drop Threshold | 1 | uint16 | 64 |
+ | Unit: bytes | | | |
+ +--------------------------+---------+---------+------------------+
+ | Period to calculate | 1 | uint16 | 15 |
+ | drop probability | | | |
+ | Unit: ms | | | |
+ +--------------------------+---------+---------+------------------+
+
+The meaning of these parameters is explained in more detail in the next sections.
+The format of these parameters as specified to the dropper module API.
+They could made self calculated for fine tuning, within the apps.
+
.. _Enqueue_Operation:
Enqueue Operation
| Method | Relative Performance |
| | |
+====================================================================================+======================+
- | Current dropper method (see :ref:`Section 23.3.2.1.3 <Dropper>`) | 100% |
+ | Current dropper method (see :ref:`Section 23.3.2.1.3 <Droppers>`) | 100% |
| | |
+------------------------------------------------------------------------------------+----------------------+
| Fixed-point method with small (512B) look-up table | 148% |
The source files for the DPDK dropper are located at:
-* DPDK/lib/librte_sched/rte_red.h
+* DPDK/lib/sched/rte_red.h
-* DPDK/lib/librte_sched/rte_red.c
+* DPDK/lib/sched/rte_red.c
Integration with the DPDK QoS Scheduler
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RED functionality in the DPDK QoS scheduler is disabled by default.
-To enable it, use the DPDK configuration parameter:
-
-::
-
- CONFIG_RTE_SCHED_RED=y
-
-This parameter must be set to y.
-The parameter is found in the build configuration files in the DPDK/config directory,
-for example, DPDK/config/common_linuxapp.
+The parameter is found in the build configuration files in the DPDK/config directory.
RED configuration parameters are specified in the rte_red_params structure within the rte_sched_port_params structure
that is passed to the scheduler on initialization.
RED parameters are specified separately for four traffic classes and three packet colors (green, yellow and red)
tc 3 wred inv prob = 10 10 10
tc 3 wred weight = 9 9 9
+ tc 4 wred min = 28 22 16
+ tc 4 wred max = 32 32 32
+ tc 4 wred inv prob = 10 10 10
+ tc 4 wred weight = 9 9 9
+
+ tc 5 wred min = 28 22 16
+ tc 5 wred max = 32 32 32
+ tc 5 wred inv prob = 10 10 10
+ tc 5 wred weight = 9 9 9
+
+ tc 6 wred min = 28 22 16
+ tc 6 wred max = 32 32 32
+ tc 6 wred inv prob = 10 10 10
+ tc 6 wred weight = 9 9 9
+
+ tc 7 wred min = 28 22 16
+ tc 7 wred max = 32 32 32
+ tc 7 wred inv prob = 10 10 10
+ tc 7 wred weight = 9 9 9
+
+ tc 8 wred min = 28 22 16
+ tc 8 wred max = 32 32 32
+ tc 8 wred inv prob = 10 10 10
+ tc 8 wred weight = 9 9 9
+
+ tc 9 wred min = 28 22 16
+ tc 9 wred max = 32 32 32
+ tc 9 wred inv prob = 10 10 10
+ tc 9 wred weight = 9 9 9
+
+
+ tc 10 wred min = 28 22 16
+ tc 10 wred max = 32 32 32
+ tc 10 wred inv prob = 10 10 10
+ tc 10 wred weight = 9 9 9
+
+ tc 11 wred min = 28 22 16
+ tc 11 wred max = 32 32 32
+ tc 11 wred inv prob = 10 10 10
+ tc 11 wred weight = 9 9 9
+
+ tc 12 wred min = 28 22 16
+ tc 12 wred max = 32 32 32
+ tc 12 wred inv prob = 10 10 10
+ tc 12 wred weight = 9 9 9
+
With this configuration file, the RED configuration that applies to green,
yellow and red packets in traffic class 0 is shown in :numref:`table_qos_18`.
git.droids-corp.org / dpdk.git / blobdiff