-.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Intel Corporation nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2010-2017 Intel Corporation.
Quota and Watermark Sample Application
======================================
-The Quota and Watermark sample application is a simple example of packet processing using Intel® Data Plane Development Kit (Intel® DPDK) that
-showcases the use of a quota as the maximum number of packets enqueue/dequeue at a time and low and high watermarks
-to signal low and high ring usage respectively.
+The Quota and Watermark sample application is a simple example of packet
+processing using Data Plane Development Kit (DPDK) that showcases the use
+of a quota as the maximum number of packets enqueue/dequeue at a time and
+low and high thresholds, or watermarks, to signal low and high ring usage
+respectively.
-Additionally, it shows how ring watermarks can be used to feedback congestion notifications to data producers by
+Additionally, it shows how the thresholds can be used to feedback congestion notifications to data producers by
temporarily stopping processing overloaded rings and sending Ethernet flow control frames.
This sample application is split in two parts:
The MAC addresses of the forwarded Ethernet frames are not affected.
Internally, packets are pulled from the ports by the master logical core and put on a variable length processing pipeline,
-each stage of which being connected by rings, as shown in Figure 12.
+each stage of which being connected by rings, as shown in :numref:`figure_pipeline_overview`.
-.. _figure_12:
+.. _figure_pipeline_overview:
-**Figure 12. Pipeline Overview**
+.. figure:: img/pipeline_overview.*
-.. image15_png has been renamed
+ Pipeline Overview
-|pipeline_overview|
An adjustable quota value controls how many packets are being moved through the pipeline per enqueue and dequeue.
-Adjustable watermark values associated with the rings control a back-off mechanism that
+Adjustable threshold values associated with the rings control a back-off mechanism that
tries to prevent the pipeline from being overloaded by:
* Stopping enqueuing on rings for which the usage has crossed the high watermark threshold
On top of serving as an example of quota and watermark usage,
this application can be used to benchmark ring based processing pipelines performance using a traffic- generator,
-as shown in Figure 13.
+as shown in :numref:`figure_ring_pipeline_perf_setup`.
-.. _figure_13:
+.. _figure_ring_pipeline_perf_setup:
-**Figure 13. Ring-based Processing Pipeline Performance Setup**
+.. figure:: img/ring_pipeline_perf_setup.*
-.. image16_png has been renamed
-
-|ring_pipeline_perf_setup|
+ Ring-based Processing Pipeline Performance Setup
Compiling the Application
-------------------------
-#. Go to the example directory:
-
- .. code-block:: console
-
- export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/quota_watermark
-
-#. Set the target (a default target is used if not specified). For example:
-
- .. code-block:: console
-
- export RTE_TARGET=x86_64-native-linuxapp-gcc
-
- See the *Intel® DPDK Getting Started Guide* for possible RTE_TARGET values.
-
-#. Build the application:
-
- .. code-block:: console
+To compile the sample application see :doc:`compiling`.
- make
+The application is located in the ``quota_watermark`` sub-directory.
Running the Application
-----------------------
-p PORTMASK: A hexadecimal bitmask of the ports to configure
-To run the application in a linuxapp environment with four logical cores and ports 0 and 2,
+To run the application in a linux environment with four logical cores and ports 0 and 2,
issue the following command:
.. code-block:: console
- ./qw/build/qw -c f -n 4 -- -p 5
+ ./qw/build/qw -l 0-3 -n 4 -- -p 5
-Refer to the *Intel® DPDK Getting Started Guide* for general information on running applications and
+Refer to the *DPDK Getting Started Guide* for general information on running applications and
the Environment Abstraction Layer (EAL) options.
Running the Control Application
The --proc-type=secondary option is necessary for the EAL to properly initialize the control application to
use the same huge pages as the core application and thus be able to access its rings.
-To run the application in a linuxapp environment on logical core 0, issue the following command:
+To run the application in a linux environment on logical core 0, issue the following command:
.. code-block:: console
- ./qwctl/build/qwctl -c 1 -n 4 --proc-type=secondary
+ ./qwctl/build/qwctl -l 0 -n 4 --proc-type=secondary
-Refer to the *Intel® DPDK Getting Started* Guide for general information on running applications and
+Refer to the *DPDK Getting Started* Guide for general information on running applications and
the Environment Abstraction Layer (EAL) options.
qwctl is an interactive command line that let the user change variables in a running instance of qw.
/* Bind the drivers to usable devices */
- ret = rte_eal_pci_probe();
+ ret = rte_pci_probe();
if (ret < 0)
- rte_exit(EXIT_FAILURE, "rte_eal_pci_probe(): error %d\n", ret);
+ rte_exit(EXIT_FAILURE, "rte_pci_probe(): error %d\n", ret);
- if (rte_eth_dev_count() < 2)
- rte_exit(EXIT_FAILURE, "Not enough ethernet port available\n");
+ if (rte_eth_dev_count_avail() < 2)
+ rte_exit(EXIT_FAILURE, "Not enough Ethernet port available\n");
}
To fully understand this code, it is recommended to study the chapters that relate to the *Poll Mode Driver*
-in the *Intel® DPDK Getting Started Guide* and the *Intel® DPDK API Reference*.
+in the *DPDK Getting Started Guide* and the *DPDK API Reference*.
Shared Variables Setup
^^^^^^^^^^^^^^^^^^^^^^
-The quota and low_watermark shared variables are put into an rte_memzone using a call to setup_shared_variables():
+The quota and high and low watermark shared variables are put into an rte_memzone using a call to setup_shared_variables():
.. code-block:: c
void
setup_shared_variables(void)
{
- const struct rte_memzone *qw_memzone;
-
- qw_memzone = rte_memzone_reserve(QUOTA_WATERMARK_MEMZONE_NAME, 2 * sizeof(int), rte_socket_id(), RTE_MEMZONE_2MB);
+ const struct rte_memzone *qw_memzone;
- if (qw_memzone == NULL)
- rte_exit(EXIT_FAILURE, "%s\n", rte_strerror(rte_errno));
+ qw_memzone = rte_memzone_reserve(QUOTA_WATERMARK_MEMZONE_NAME,
+ 3 * sizeof(int), rte_socket_id(), 0);
+ if (qw_memzone == NULL)
+ rte_exit(EXIT_FAILURE, "%s\n", rte_strerror(rte_errno));
- quota = qw_memzone->addr;
- low_watermark = (unsigned int *) qw_memzone->addr + sizeof(int);
- }
+ quota = qw_memzone->addr;
+ low_watermark = (unsigned int *) qw_memzone->addr + 1;
+ high_watermark = (unsigned int *) qw_memzone->addr + 2;
+ }
-These two variables are initialized to a default value in main() and
+These three variables are initialized to a default value in main() and
can be changed while qw is running using the qwctl control program.
Application Arguments
.. code-block:: c
/* Create a pool of mbuf to store packets */
-
- mbuf_pool = rte_mempool_create("mbuf_pool", MBUF_PER_POOL, MBUF_SIZE, 32, sizeof(struct rte_pktmbuf_pool_private),
- rte_pktmbuf_pool_init, NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+ mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", MBUF_PER_POOL, 32, 0,
+ MBUF_DATA_SIZE, rte_socket_id());
if (mbuf_pool == NULL)
rte_panic("%s\n", rte_strerror(rte_errno));
The rte_mempool is a generic structure used to handle pools of objects.
-In this case, it is necessary to create a pool that will be used by the driver,
-which expects to have some reserved space in the mempool structure, sizeof(struct rte_pktmbuf_pool_private) bytes.
+In this case, it is necessary to create a pool that will be used by the driver.
-The number of allocated pkt mbufs is MBUF_PER_POOL, with a size of MBUF_SIZE each.
+The number of allocated pkt mbufs is MBUF_PER_POOL, with a data room size
+of MBUF_DATA_SIZE each.
A per-lcore cache of 32 mbufs is kept.
The memory is allocated in on the master lcore's socket, but it is possible to extend this code to allocate one mbuf pool per socket.
-Two callback pointers are also given to the rte_mempool_create() function:
-
-* The first callback pointer is to rte_pktmbuf_pool_init() and is used to initialize the private data of the mempool,
- which is needed by the driver.
- This function is provided by the mbuf API, but can be copied and extended by the developer.
-
-* The second callback pointer given to rte_mempool_create() is the mbuf initializer.
-
-The default is used, that is, rte_pktmbuf_init(), which is provided in the rte_mbuf library.
-If a more complex application wants to extend the rte_pktmbuf structure for its own needs,
-a new function derived from rte_pktmbuf_init() can be created.
+The rte_pktmbuf_pool_create() function uses the default mbuf pool and mbuf
+initializers, respectively rte_pktmbuf_pool_init() and rte_pktmbuf_init().
+An advanced application may want to use the mempool API to create the
+mbuf pool with more control.
Ports Configuration and Pairing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pair_ports();
The configure_eth_port() and init_ring() functions are used to configure a port and a ring respectively and are defined in init.c.
-They make use of the Intel® DPDK APIs defined in rte_eth.h and rte_ring.h.
+They make use of the DPDK APIs defined in rte_eth.h and rte_ring.h.
pair_ports() builds the port_pairs[] array so that its key-value pairs are a mapping between reception and transmission ports.
It is defined in init.c.
The application uses the master logical core to poll all the ports for new packets and enqueue them on a ring associated with the port.
Each logical core except the last runs pipeline_stage() after a ring for each used port is initialized on that core.
-pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See Figure 14.
+pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See :numref:`figure_threads_pipelines`.
.. code-block:: c
Receive, Process and Transmit Packets
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-.. _figure_14:
+.. _figure_threads_pipelines:
-Figure 14 shows where each thread in the pipeline is.
-It should be used as a reference while reading the rest of this section.
+.. figure:: img/threads_pipelines.*
-**Figure 14. Threads and Pipelines**
+ Threads and Pipelines
-.. image17_png has been renamed
-
-|threads_pipelines|
In the receive_stage() function running on the master logical core,
the main task is to read ingress packets from the RX ports and enqueue them
/* Process each port round robin style */
for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) {
- if (!is_bit_set(port_id, portmask))
- continue;
-
- ring = rings[lcore_id][port_id];
-
- if (ring_state[port_id] != RING_READY) {
- if (rte_ring_count(ring) > *low_watermark)
- continue;
- else
- ring_state[port_id] = RING_READY;
- }
-
- /* Enqueue received packets on the RX ring */
-
- nb_rx_pkts = rte_eth_rx_burst(port_id, 0, pkts, *quota);
-
- ret = rte_ring_enqueue_bulk(ring, (void *) pkts, nb_rx_pkts);
- if (ret == -EDQUOT) {
- ring_state[port_id] = RING_OVERLOADED;
- send_pause_frame(port_id, 1337);
- }
+ if (!is_bit_set(port_id, portmask))
+ continue;
+
+ ring = rings[lcore_id][port_id];
+
+ if (ring_state[port_id] != RING_READY) {
+ if (rte_ring_count(ring) > *low_watermark)
+ continue;
+ else
+ ring_state[port_id] = RING_READY;
+ }
+
+ /* Enqueue received packets on the RX ring */
+ nb_rx_pkts = rte_eth_rx_burst(port_id, 0, pkts,
+ (uint16_t) *quota);
+ ret = rte_ring_enqueue_bulk(ring, (void *) pkts,
+ nb_rx_pkts, &free);
+ if (RING_SIZE - free > *high_watermark) {
+ ring_state[port_id] = RING_OVERLOADED;
+ send_pause_frame(port_id, 1337);
+ }
+
+ if (ret == 0) {
+
+ /*
+ * Return mbufs to the pool,
+ * effectively dropping packets
+ */
+ for (i = 0; i < nb_rx_pkts; i++)
+ rte_pktmbuf_free(pkts[i]);
+ }
}
For each port in the port mask, the corresponding ring's pointer is fetched into ring and that ring's state is checked:
previous_lcore_id = get_previous_lcore_id(lcore_id);
for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) {
- if (!is_bit_set(port_id, portmask))
- continue;
-
- tx = rings[lcore_id][port_id];
- rx = rings[previous_lcore_id][port_id];
- if (ring_state[port_id] != RING_READY) {
- if (rte_ring_count(tx) > *low_watermark)
- continue;
- else
- ring_state[port_id] = RING_READY;
- }
-
- /* Dequeue up to quota mbuf from rx */
-
- nb_dq_pkts = rte_ring_dequeue_burst(rx, pkts, *quota);
-
- if (unlikely(nb_dq_pkts < 0))
- continue;
-
- /* Enqueue them on tx */
-
- ret = rte_ring_enqueue_bulk(tx, pkts, nb_dq_pkts);
- if (ret == -EDQUOT)
- ring_state[port_id] = RING_OVERLOADED;
+ if (!is_bit_set(port_id, portmask))
+ continue;
+
+ tx = rings[lcore_id][port_id];
+ rx = rings[previous_lcore_id][port_id];
+
+ if (ring_state[port_id] != RING_READY) {
+ if (rte_ring_count(tx) > *low_watermark)
+ continue;
+ else
+ ring_state[port_id] = RING_READY;
+ }
+
+ /* Dequeue up to quota mbuf from rx */
+ nb_dq_pkts = rte_ring_dequeue_burst(rx, pkts,
+ *quota, NULL);
+ if (unlikely(nb_dq_pkts < 0))
+ continue;
+
+ /* Enqueue them on tx */
+ ret = rte_ring_enqueue_bulk(tx, pkts,
+ nb_dq_pkts, &free);
+ if (RING_SIZE - free > *high_watermark)
+ ring_state[port_id] = RING_OVERLOADED;
+
+ if (ret == 0) {
+
+ /*
+ * Return mbufs to the pool,
+ * effectively dropping packets
+ */
+ for (i = 0; i < nb_dq_pkts; i++)
+ rte_pktmbuf_free(pkts[i]);
+ }
}
The thread's logic works mostly like receive_stage(),
qw_memzone = rte_memzone_lookup(QUOTA_WATERMARK_MEMZONE_NAME);
if (qw_memzone == NULL)
- rte_exit(EXIT_FAILURE, "Could't find memzone\n");
+ rte_exit(EXIT_FAILURE, "Couldn't find memzone\n");
quota = qw_memzone->addr;
- low_watermark = (unsigned int *) qw_memzone->addr + sizeof(int);
+ low_watermark = (unsigned int *) qw_memzone->addr + 1;
+ high_watermark = (unsigned int *) qw_memzone->addr + 2;
}
-
-.. |pipeline_overview| image:: img/pipeline_overview.png
-
-.. |ring_pipeline_perf_setup| image:: img/ring_pipeline_perf_setup.png
-
-.. |threads_pipelines| image:: img/threads_pipelines.png
git.droids-corp.org / dpdk.git / blobdiff