-.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Intel Corporation nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2010-2014 Intel Corporation.
Writing Efficient Code
======================
-This chapter provides some tips for developing efficient code using the Intel® DPDK.
+This chapter provides some tips for developing efficient code using the DPDK.
For additional and more general information,
please refer to the *Intel® 64 and IA-32 Architectures Optimization Reference Manual*
which is a valuable reference to writing efficient code.
Memory
------
-This section describes some key memory considerations when developing applications in the Intel® DPDK environment.
+This section describes some key memory considerations when developing applications in the DPDK environment.
Memory Copy: Do not Use libc in the Data Plane
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Many libc functions are available in the Intel® DPDK, via the Linux* application environment.
+Many libc functions are available in the DPDK, via the Linux* application environment.
This can ease the porting of applications and the development of the configuration plane.
However, many of these functions are not designed for performance.
Functions such as memcpy() or strcpy() should not be used in the data plane.
For specific functions that are called often,
it is also a good idea to provide a self-made optimized function, which should be declared as static inline.
-The Intel® DPDK API provides an optimized rte_memcpy() function.
+The DPDK API provides an optimized rte_memcpy() function.
Memory Allocation
~~~~~~~~~~~~~~~~~
~~~~
On a NUMA system, it is preferable to access local memory since remote memory access is slower.
-In the Intel® DPDK, the memzone, ring, rte_malloc and mempool APIs provide a way to create a pool on a specific socket.
+In the DPDK, the memzone, ring, rte_malloc and mempool APIs provide a way to create a pool on a specific socket.
Sometimes, it can be a good idea to duplicate data to optimize speed.
For read-mostly variables that are often accessed,
By default, the :ref:`Mempool Library <Mempool_Library>` spreads the addresses of objects among memory channels.
+Locking memory pages
+~~~~~~~~~~~~~~~~~~~~
+
+The underlying operating system is allowed to load/unload memory pages at its own discretion.
+These page loads could impact the performance, as the process is on hold when the kernel fetches them.
+
+To avoid these you could pre-load, and lock them into memory with the ``mlockall()`` call.
+
+.. code-block:: c
+
+ if (mlockall(MCL_CURRENT | MCL_FUTURE)) {
+ RTE_LOG(NOTICE, USER1, "mlockall() failed with error \"%s\"\n",
+ strerror(errno));
+ }
+
Communication Between lcores
----------------------------
To provide a message-based communication between lcores,
-it is advised to use the Intel® DPDK ring API, which provides a lockless ring implementation.
+it is advised to use the DPDK ring API, which provides a lockless ring implementation.
The ring supports bulk and burst access,
meaning that it is possible to read several elements from the ring with only one costly atomic operation
-(see Chapter 5 "Ring Library").
+(see :doc:`ring_lib`).
Performance is greatly improved when using bulk access operations.
The code algorithm that dequeues messages may be something similar to the following:
while (1) {
/* Process as many elements as can be dequeued. */
- count = rte_ring_dequeue_burst(ring, obj_table, MAX_BULK);
+ count = rte_ring_dequeue_burst(ring, obj_table, MAX_BULK, NULL);
if (unlikely(count == 0))
continue;
PMD Driver
----------
-The Intel® DPDK Poll Mode Driver (PMD) is also able to work in bulk/burst mode,
+The DPDK Poll Mode Driver (PMD) is also able to work in bulk/burst mode,
allowing the factorization of some code for each call in the send or receive function.
Avoid partial writes.
a low end-to-end latency, at the cost of lower throughput.
In order to achieve higher throughput,
-the Intel® DPDK attempts to aggregate the cost of processing each packet individually by processing packets in bursts.
+the DPDK attempts to aggregate the cost of processing each packet individually by processing packets in bursts.
Using the testpmd application as an example,
the burst size can be set on the command line to a value of 16 (also the default value).
Locks and Atomic Operations
---------------------------
-Atomic operations imply a lock prefix before the instruction,
+This section describes some key considerations when using locks and atomic
+operations in the DPDK environment.
+
+Locks
+~~~~~
+
+On x86, atomic operations imply a lock prefix before the instruction,
causing the processor's LOCK# signal to be asserted during execution of the following instruction.
This has a big impact on performance in a multicore environment.
Also, some locking techniques are more efficient than others.
For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks.
+Atomic Operations: Use C11 Atomic Builtins
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+DPDK generic rte_atomic operations are implemented by __sync builtins. These
+__sync builtins result in full barriers on aarch64, which are unnecessary
+in many use cases. They can be replaced by __atomic builtins that conform to
+the C11 memory model and provide finer memory order control.
+
+So replacing the rte_atomic operations with __atomic builtins might improve
+performance for aarch64 machines.
+
+Some typical optimization cases are listed below:
+
+Atomicity
+^^^^^^^^^
+
+Some use cases require atomicity alone, the ordering of the memory operations
+does not matter. For example, the packet statistics counters need to be
+incremented atomically but do not need any particular memory ordering.
+So, RELAXED memory ordering is sufficient.
+
+One-way Barrier
+^^^^^^^^^^^^^^^
+
+Some use cases allow for memory reordering in one way while requiring memory
+ordering in the other direction.
+
+For example, the memory operations before the spinlock lock are allowed to
+move to the critical section, but the memory operations in the critical section
+are not allowed to move above the lock. In this case, the full memory barrier
+in the compare-and-swap operation can be replaced with ACQUIRE memory order.
+On the other hand, the memory operations after the spinlock unlock are allowed
+to move to the critical section, but the memory operations in the critical
+section are not allowed to move below the unlock. So the full barrier in the
+store operation can use RELEASE memory order.
+
+Reader-Writer Concurrency
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Lock-free reader-writer concurrency is one of the common use cases in DPDK.
+
+The payload or the data that the writer wants to communicate to the reader,
+can be written with RELAXED memory order. However, the guard variable should
+be written with RELEASE memory order. This ensures that the store to guard
+variable is observable only after the store to payload is observable.
+
+Correspondingly, on the reader side, the guard variable should be read
+with ACQUIRE memory order. The payload or the data the writer communicated,
+can be read with RELAXED memory order. This ensures that, if the store to
+guard variable is observable, the store to payload is also observable.
+
Coding Considerations
---------------------
Setting the Target CPU Type
---------------------------
-The Intel® DPDK supports CPU microarchitecture-specific optimizations by means of CONFIG_RTE_MACHINE option
-in the Intel® DPDK configuration file.
-The degree of optimization depends on the compiler's ability to optimize for a specitic microarchitecture,
+The DPDK supports CPU microarchitecture-specific optimizations by means of RTE_MACHINE option.
+The degree of optimization depends on the compiler's ability to optimize for a specific microarchitecture,
therefore it is preferable to use the latest compiler versions whenever possible.
If the compiler version does not support the specific feature set (for example, the Intel® AVX instruction set),
Along with compiler optimizations,
a set of preprocessor defines are automatically added to the build process (regardless of the compiler version).
These defines correspond to the instruction sets that the target CPU should be able to support.
-For example, a binary compiled for any SSE4.2-capable processor will have RTE_MACHINE_CPUFLAG_SSE4_2 defined,
-thus enabling compile-time code path selection for different platforms.
git.droids-corp.org / dpdk.git / blobdiff