1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2010-2014 Intel Corporation.
9 The ring allows the management of queues.
10 Instead of having a linked list of infinite size, the rte_ring has the following properties:
14 * Maximum size is fixed, the pointers are stored in a table
16 * Lockless implementation
18 * Multi-consumer or single-consumer dequeue
20 * Multi-producer or single-producer enqueue
22 * Bulk dequeue - Dequeues the specified count of objects if successful; otherwise fails
24 * Bulk enqueue - Enqueues the specified count of objects if successful; otherwise fails
26 * Burst dequeue - Dequeue the maximum available objects if the specified count cannot be fulfilled
28 * Burst enqueue - Enqueue the maximum available objects if the specified count cannot be fulfilled
30 The advantages of this data structure over a linked list queue are as follows:
32 * Faster; only requires a single Compare-And-Swap instruction of sizeof(void \*) instead of several double-Compare-And-Swap instructions.
34 * Simpler than a full lockless queue.
36 * Adapted to bulk enqueue/dequeue operations.
37 As pointers are stored in a table, a dequeue of several objects will not produce as many cache misses as in a linked queue.
38 Also, a bulk dequeue of many objects does not cost more than a dequeue of a simple object.
44 * Having many rings costs more in terms of memory than a linked list queue. An empty ring contains at least N pointers.
46 A simplified representation of a Ring is shown in with consumer and producer head and tail pointers to objects stored in the data structure.
50 .. figure:: img/ring1.*
55 References for Ring Implementation in FreeBSD*
56 ----------------------------------------------
58 The following code was added in FreeBSD 8.0, and is used in some network device drivers (at least in Intel drivers):
60 * `bufring.h in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/sys/buf_ring.h?revision=199625&view=markup>`_
62 * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_
64 Lockless Ring Buffer in Linux*
65 ------------------------------
67 The following is a link describing the `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_.
75 A ring is identified by a unique name.
76 It is not possible to create two rings with the same name (rte_ring_create() returns NULL if this is attempted).
81 Use cases for the Ring library include:
83 * Communication between applications in the DPDK
85 * Used by memory pool allocator
87 Anatomy of a Ring Buffer
88 ------------------------
90 This section explains how a ring buffer operates.
91 The ring structure is composed of two head and tail couples; one is used by producers and one is used by the consumers.
92 The figures of the following sections refer to them as prod_head, prod_tail, cons_head and cons_tail.
94 Each figure represents a simplified state of the ring, which is a circular buffer.
95 The content of the function local variables is represented on the top of the figure,
96 and the content of ring structure is represented on the bottom of the figure.
98 Single Producer Enqueue
99 ~~~~~~~~~~~~~~~~~~~~~~~
101 This section explains what occurs when a producer adds an object to the ring.
102 In this example, only the producer head and tail (prod_head and prod_tail) are modified,
103 and there is only one producer.
105 The initial state is to have a prod_head and prod_tail pointing at the same location.
110 First, *ring->prod_head* and ring->cons_tail are copied in local variables.
111 The prod_next local variable points to the next element of the table, or several elements after in case of bulk enqueue.
113 If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error.
116 .. _figure_ring-enqueue1:
118 .. figure:: img/ring-enqueue1.*
126 The second step is to modify *ring->prod_head* in ring structure to point to the same location as prod_next.
128 A pointer to the added object is copied in the ring (obj4).
131 .. _figure_ring-enqueue2:
133 .. figure:: img/ring-enqueue2.*
141 Once the object is added in the ring, ring->prod_tail in the ring structure is modified to point to the same location as *ring->prod_head*.
142 The enqueue operation is finished.
145 .. _figure_ring-enqueue3:
147 .. figure:: img/ring-enqueue3.*
152 Single Consumer Dequeue
153 ~~~~~~~~~~~~~~~~~~~~~~~
155 This section explains what occurs when a consumer dequeues an object from the ring.
156 In this example, only the consumer head and tail (cons_head and cons_tail) are modified and there is only one consumer.
158 The initial state is to have a cons_head and cons_tail pointing at the same location.
163 First, ring->cons_head and ring->prod_tail are copied in local variables.
164 The cons_next local variable points to the next element of the table, or several elements after in the case of bulk dequeue.
166 If there are not enough objects in the ring (this is detected by checking prod_tail), it returns an error.
169 .. _figure_ring-dequeue1:
171 .. figure:: img/ring-dequeue1.*
179 The second step is to modify ring->cons_head in the ring structure to point to the same location as cons_next.
181 The pointer to the dequeued object (obj1) is copied in the pointer given by the user.
184 .. _figure_ring-dequeue2:
186 .. figure:: img/ring-dequeue2.*
194 Finally, ring->cons_tail in the ring structure is modified to point to the same location as ring->cons_head.
195 The dequeue operation is finished.
198 .. _figure_ring-dequeue3:
200 .. figure:: img/ring-dequeue3.*
205 Multiple Producers Enqueue
206 ~~~~~~~~~~~~~~~~~~~~~~~~~~
208 This section explains what occurs when two producers concurrently add an object to the ring.
209 In this example, only the producer head and tail (prod_head and prod_tail) are modified.
211 The initial state is to have a prod_head and prod_tail pointing at the same location.
213 Multiple Producers Enqueue First Step
214 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
216 On both cores, *ring->prod_head* and ring->cons_tail are copied in local variables.
217 The prod_next local variable points to the next element of the table,
218 or several elements after in the case of bulk enqueue.
220 If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error.
223 .. _figure_ring-mp-enqueue1:
225 .. figure:: img/ring-mp-enqueue1.*
227 Multiple producer enqueue first step
230 Multiple Producers Enqueue Second Step
231 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
233 The second step is to modify ring->prod_head in the ring structure to point to the same location as prod_next.
234 This operation is done using a Compare And Swap (CAS) instruction, which does the following operations atomically:
236 * If ring->prod_head is different to local variable prod_head,
237 the CAS operation fails, and the code restarts at first step.
239 * Otherwise, ring->prod_head is set to local prod_next,
240 the CAS operation is successful, and processing continues.
242 In the figure, the operation succeeded on core 1, and step one restarted on core 2.
245 .. _figure_ring-mp-enqueue2:
247 .. figure:: img/ring-mp-enqueue2.*
249 Multiple producer enqueue second step
252 Multiple Producers Enqueue Third Step
253 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
255 The CAS operation is retried on core 2 with success.
257 The core 1 updates one element of the ring(obj4), and the core 2 updates another one (obj5).
260 .. _figure_ring-mp-enqueue3:
262 .. figure:: img/ring-mp-enqueue3.*
264 Multiple producer enqueue third step
267 Multiple Producers Enqueue Fourth Step
268 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
270 Each core now wants to update ring->prod_tail.
271 A core can only update it if ring->prod_tail is equal to the prod_head local variable.
272 This is only true on core 1. The operation is finished on core 1.
275 .. _figure_ring-mp-enqueue4:
277 .. figure:: img/ring-mp-enqueue4.*
279 Multiple producer enqueue fourth step
282 Multiple Producers Enqueue Last Step
283 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
285 Once ring->prod_tail is updated by core 1, core 2 is allowed to update it too.
286 The operation is also finished on core 2.
289 .. _figure_ring-mp-enqueue5:
291 .. figure:: img/ring-mp-enqueue5.*
293 Multiple producer enqueue last step
296 Modulo 32-bit Indexes
297 ~~~~~~~~~~~~~~~~~~~~~
299 In the preceding figures, the prod_head, prod_tail, cons_head and cons_tail indexes are represented by arrows.
300 In the actual implementation, these values are not between 0 and size(ring)-1 as would be assumed.
301 The indexes are between 0 and 2^32 -1, and we mask their value when we access the pointer table (the ring itself).
302 32-bit modulo also implies that operations on indexes (such as, add/subtract) will automatically do 2^32 modulo
303 if the result overflows the 32-bit number range.
305 The following are two examples that help to explain how indexes are used in a ring.
309 To simplify the explanation, operations with modulo 16-bit are used instead of modulo 32-bit.
310 In addition, the four indexes are defined as unsigned 16-bit integers,
311 as opposed to unsigned 32-bit integers in the more realistic case.
314 .. _figure_ring-modulo1:
316 .. figure:: img/ring-modulo1.*
318 Modulo 32-bit indexes - Example 1
321 This ring contains 11000 entries.
324 .. _figure_ring-modulo2:
326 .. figure:: img/ring-modulo2.*
328 Modulo 32-bit indexes - Example 2
331 This ring contains 12536 entries.
335 For ease of understanding, we use modulo 65536 operations in the above examples.
336 In real execution cases, this is redundant for low efficiency, but is done automatically when the result overflows.
338 The code always maintains a distance between producer and consumer between 0 and size(ring)-1.
339 Thanks to this property, we can do subtractions between 2 index values in a modulo-32bit base:
340 that's why the overflow of the indexes is not a problem.
342 At any time, entries and free_entries are between 0 and size(ring)-1,
343 even if only the first term of subtraction has overflowed:
347 uint32_t entries = (prod_tail - cons_head);
348 uint32_t free_entries = (mask + cons_tail -prod_head);
353 * `bufring.h in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/sys/buf_ring.h?revision=199625&view=markup>`_ (version 8)
355 * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_ (version 8)
357 * `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_