2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 IPv4 Multicast Sample Application
32 =================================
34 The IPv4 Multicast application is a simple example of packet processing
35 using the Data Plane Development Kit (DPDK).
36 The application performs L3 multicasting.
41 The application demonstrates the use of zero-copy buffers for packet forwarding.
42 The initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
43 This guide highlights the differences between the two applications.
44 There are two key differences from the L2 Forwarding sample application:
46 * The IPv4 Multicast sample application makes use of indirect buffers.
48 * The forwarding decision is taken based on information read from the input packet's IPv4 header.
50 The lookup method is the Four-byte Key (FBK) hash-based method.
51 The lookup table is composed of pairs of destination IPv4 address (the FBK)
52 and a port mask associated with that IPv4 address.
54 For convenience and simplicity, this sample application does not take IANA-assigned multicast addresses into account,
55 but instead equates the last four bytes of the multicast group (that is, the last four bytes of the destination IP address)
56 with the mask of ports to multicast packets to.
57 Also, the application does not consider the Ethernet addresses;
58 it looks only at the IPv4 destination address for any given packet.
60 Building the Application
61 ------------------------
63 To compile the application:
65 #. Go to the sample application directory:
67 .. code-block:: console
69 export RTE_SDK=/path/to/rte_sdk
70 cd ${RTE_SDK}/examples/ipv4_multicast
72 #. Set the target (a default target is used if not specified). For example:
74 .. code-block:: console
76 export RTE_TARGET=x86_64-native-linuxapp-gcc
78 See the *DPDK Getting Started Guide* for possible RTE_TARGET values.
80 #. Build the application:
82 .. code-block:: console
88 The compiled application is written to the build subdirectory.
89 To have the application written to a different location,
90 the O=/path/to/build/directory option may be specified in the make command.
92 Running the Application
93 -----------------------
95 The application has a number of command line options:
97 .. code-block:: console
99 ./build/ipv4_multicast [EAL options] -- -p PORTMASK [-q NQ]
103 * -p PORTMASK: Hexadecimal bitmask of ports to configure
105 * -q NQ: determines the number of queues per lcore
109 Unlike the basic L2/L3 Forwarding sample applications,
110 NUMA support is not provided in the IPv4 Multicast sample application.
112 Typically, to run the IPv4 Multicast sample application, issue the following command (as root):
114 .. code-block:: console
116 ./build/ipv4_multicast -l 0-3 -n 3 -- -p 0x3 -q 1
120 * The -c option enables cores 0, 1, 2 and 3
122 * The -n option specifies 3 memory channels
124 * The -p option enables ports 0 and 1
126 * The -q option assigns 1 queue to each lcore
128 Refer to the *DPDK Getting Started Guide* for general information on running applications
129 and the Environment Abstraction Layer (EAL) options.
134 The following sections provide some explanation of the code.
135 As mentioned in the overview section,
136 the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
137 The following sections describe aspects that are specific to the IPv4 Multicast sample application.
139 Memory Pool Initialization
140 ~~~~~~~~~~~~~~~~~~~~~~~~~~
142 The IPv4 Multicast sample application uses three memory pools.
143 Two of the pools are for indirect buffers used for packet duplication purposes.
144 Memory pools for indirect buffers are initialized differently from the memory pool for direct buffers:
148 packet_pool = rte_pktmbuf_pool_create("packet_pool", NB_PKT_MBUF, 32,
149 0, PKT_MBUF_DATA_SIZE, rte_socket_id());
150 header_pool = rte_pktmbuf_pool_create("header_pool", NB_HDR_MBUF, 32,
151 0, HDR_MBUF_DATA_SIZE, rte_socket_id());
152 clone_pool = rte_pktmbuf_pool_create("clone_pool", NB_CLONE_MBUF, 32,
153 0, 0, rte_socket_id());
155 The reason for this is because indirect buffers are not supposed to hold any packet data and
156 therefore can be initialized with lower amount of reserved memory for each buffer.
161 The hash object is created and loaded with the pre-configured entries read from a global array:
167 init_mcast_hash(void)
170 mcast_hash_params.socket_id = rte_socket_id();
172 mcast_hash = rte_fbk_hash_create(&mcast_hash_params);
173 if (mcast_hash == NULL){
177 for (i = 0; i < N_MCAST_GROUPS; i ++){
178 if (rte_fbk_hash_add_key(mcast_hash, mcast_group_table[i].ip, mcast_group_table[i].port_mask) < 0) {
188 All forwarding is done inside the mcast_forward() function.
189 Firstly, the Ethernet* header is removed from the packet and the IPv4 address is extracted from the IPv4 header:
193 /* Remove the Ethernet header from the input packet */
195 iphdr = (struct ipv4_hdr *)rte_pktmbuf_adj(m, sizeof(struct ether_hdr));
196 RTE_ASSERT(iphdr != NULL);
197 dest_addr = rte_be_to_cpu_32(iphdr->dst_addr);
199 Then, the packet is checked to see if it has a multicast destination address and
200 if the routing table has any ports assigned to the destination address:
204 if (!IS_IPV4_MCAST(dest_addr) ||
205 (hash = rte_fbk_hash_lookup(mcast_hash, dest_addr)) <= 0 ||
206 (port_mask = hash & enabled_port_mask) == 0) {
211 Then, the number of ports in the destination portmask is calculated with the help of the bitcnt() function:
215 /* Get number of bits set. */
217 static inline uint32_t bitcnt(uint32_t v)
221 for (n = 0; v != 0; v &= v - 1, n++)
226 This is done to determine which forwarding algorithm to use.
227 This is explained in more detail in the next section.
229 Thereafter, a destination Ethernet address is constructed:
233 /* construct destination Ethernet address */
235 dst_eth_addr = ETHER_ADDR_FOR_IPV4_MCAST(dest_addr);
237 Since Ethernet addresses are also part of the multicast process, each outgoing packet carries the same destination Ethernet address.
238 The destination Ethernet address is constructed from the lower 23 bits of the multicast group OR-ed
239 with the Ethernet address 01:00:5e:00:00:00, as per RFC 1112:
243 #define ETHER_ADDR_FOR_IPV4_MCAST(x) \
244 (rte_cpu_to_be_64(0x01005e000000ULL | ((x) & 0x7fffff)) >> 16)
246 Then, packets are dispatched to the destination ports according to the portmask associated with a multicast group:
250 for (port = 0; use_clone != port_mask; port_mask >>= 1, port++) {
251 /* Prepare output packet and send it out. */
253 if ((port_mask & 1) != 0) {
254 if (likely ((mc = mcast_out_pkt(m, use_clone)) != NULL))
255 mcast_send_pkt(mc, &dst_eth_addr.as_addr, qconf, port);
256 else if (use_clone == 0)
261 The actual packet transmission is done in the mcast_send_pkt() function:
265 static inline void mcast_send_pkt(struct rte_mbuf *pkt, struct ether_addr *dest_addr, struct lcore_queue_conf *qconf, uint8_t port)
267 struct ether_hdr *ethdr;
270 /* Construct Ethernet header. */
272 ethdr = (struct ether_hdr *)rte_pktmbuf_prepend(pkt, (uint16_t) sizeof(*ethdr));
274 RTE_ASSERT(ethdr != NULL);
276 ether_addr_copy(dest_addr, ðdr->d_addr);
277 ether_addr_copy(&ports_eth_addr[port], ðdr->s_addr);
278 ethdr->ether_type = rte_be_to_cpu_16(ETHER_TYPE_IPv4);
280 /* Put new packet into the output queue */
282 len = qconf->tx_mbufs[port].len;
283 qconf->tx_mbufs[port].m_table[len] = pkt;
284 qconf->tx_mbufs[port].len = ++len;
286 /* Transmit packets */
288 if (unlikely(MAX_PKT_BURST == len))
289 send_burst(qconf, port);
295 This is the most important part of the application since it demonstrates the use of zero- copy buffer cloning.
296 There are two approaches for creating the outgoing packet and although both are based on the data zero-copy idea,
297 there are some differences in the detail.
299 The first approach creates a clone of the input packet, for example,
300 walk though all segments of the input packet and for each of segment,
301 create a new buffer and attach that new buffer to the segment
302 (refer to rte_pktmbuf_clone() in the rte_mbuf library for more details).
303 A new buffer is then allocated for the packet header and is prepended to the cloned buffer.
305 The second approach does not make a clone, it just increments the reference counter for all input packet segment,
306 allocates a new buffer for the packet header and prepends it to the input packet.
308 Basically, the first approach reuses only the input packet's data, but creates its own copy of packet's metadata.
309 The second approach reuses both input packet's data and metadata.
311 The advantage of first approach is that each outgoing packet has its own copy of the metadata,
312 so we can safely modify the data pointer of the input packet.
313 That allows us to skip creation if the output packet is for the last destination port
314 and instead modify input packet's header in place.
315 For example, for N destination ports, we need to invoke mcast_out_pkt() (N-1) times.
317 The advantage of the second approach is that there is less work to be done for each outgoing packet,
318 that is, the "clone" operation is skipped completely.
319 However, there is a price to pay.
320 The input packet's metadata must remain intact, so for N destination ports,
321 we need to invoke mcast_out_pkt() (N) times.
323 Therefore, for a small number of outgoing ports (and segments in the input packet),
324 first approach is faster.
325 As the number of outgoing ports (and/or input segments) grows, the second approach becomes more preferable.
327 Depending on the number of segments or the number of ports in the outgoing portmask,
328 either the first (with cloning) or the second (without cloning) approach is taken:
332 use_clone = (port_num <= MCAST_CLONE_PORTS && m->pkt.nb_segs <= MCAST_CLONE_SEGS);
334 It is the mcast_out_pkt() function that performs the packet duplication (either with or without actually cloning the buffers):
338 static inline struct rte_mbuf *mcast_out_pkt(struct rte_mbuf *pkt, int use_clone)
340 struct rte_mbuf *hdr;
342 /* Create new mbuf for the header. */
344 if (unlikely ((hdr = rte_pktmbuf_alloc(header_pool)) == NULL))
347 /* If requested, then make a new clone packet. */
349 if (use_clone != 0 && unlikely ((pkt = rte_pktmbuf_clone(pkt, clone_pool)) == NULL)) {
350 rte_pktmbuf_free(hdr);
354 /* prepend new header */
358 /* update header's fields */
360 hdr->pkt.pkt_len = (uint16_t)(hdr->pkt.data_len + pkt->pkt.pkt_len);
361 hdr->pkt.nb_segs = (uint8_t)(pkt->pkt.nb_segs + 1);
363 /* copy metadata from source packet */
365 hdr->pkt.in_port = pkt->pkt.in_port;
366 hdr->pkt.vlan_macip = pkt->pkt.vlan_macip;
367 hdr->pkt.hash = pkt->pkt.hash;
368 hdr->ol_flags = pkt->ol_flags;
369 rte_mbuf_sanity_check(hdr, RTE_MBUF_PKT, 1);