2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 IPv4 Multicast Sample Application
32 =================================
34 The IPv4 Multicast application is a simple example of packet processing
35 using the Data Plane Development Kit (DPDK).
36 The application performs L3 multicasting.
41 The application demonstrates the use of zero-copy buffers for packet forwarding.
42 The initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
43 This guide highlights the differences between the two applications.
44 There are two key differences from the L2 Forwarding sample application:
46 * The IPv4 Multicast sample application makes use of indirect buffers.
48 * The forwarding decision is taken based on information read from the input packet's IPv4 header.
50 The lookup method is the Four-byte Key (FBK) hash-based method.
51 The lookup table is composed of pairs of destination IPv4 address (the FBK)
52 and a port mask associated with that IPv4 address.
56 The max port mask supported in the given hash table is 0xf, so only first
57 four ports can be supported.
58 If using non-consecutive ports, use the destination IPv4 address accordingly.
60 For convenience and simplicity, this sample application does not take IANA-assigned multicast addresses into account,
61 but instead equates the last four bytes of the multicast group (that is, the last four bytes of the destination IP address)
62 with the mask of ports to multicast packets to.
63 Also, the application does not consider the Ethernet addresses;
64 it looks only at the IPv4 destination address for any given packet.
66 Compiling the Application
67 -------------------------
69 To compile the sample application see :doc:`compiling`.
71 The application is located in the ``ipv4_multicast`` sub-directory.
73 Running the Application
74 -----------------------
76 The application has a number of command line options:
78 .. code-block:: console
80 ./build/ipv4_multicast [EAL options] -- -p PORTMASK [-q NQ]
84 * -p PORTMASK: Hexadecimal bitmask of ports to configure
86 * -q NQ: determines the number of queues per lcore
90 Unlike the basic L2/L3 Forwarding sample applications,
91 NUMA support is not provided in the IPv4 Multicast sample application.
93 Typically, to run the IPv4 Multicast sample application, issue the following command (as root):
95 .. code-block:: console
97 ./build/ipv4_multicast -l 0-3 -n 3 -- -p 0x3 -q 1
101 * The -l option enables cores 0, 1, 2 and 3
103 * The -n option specifies 3 memory channels
105 * The -p option enables ports 0 and 1
107 * The -q option assigns 1 queue to each lcore
109 Refer to the *DPDK Getting Started Guide* for general information on running applications
110 and the Environment Abstraction Layer (EAL) options.
115 The following sections provide some explanation of the code.
116 As mentioned in the overview section,
117 the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
118 The following sections describe aspects that are specific to the IPv4 Multicast sample application.
120 Memory Pool Initialization
121 ~~~~~~~~~~~~~~~~~~~~~~~~~~
123 The IPv4 Multicast sample application uses three memory pools.
124 Two of the pools are for indirect buffers used for packet duplication purposes.
125 Memory pools for indirect buffers are initialized differently from the memory pool for direct buffers:
129 packet_pool = rte_pktmbuf_pool_create("packet_pool", NB_PKT_MBUF, 32,
130 0, PKT_MBUF_DATA_SIZE, rte_socket_id());
131 header_pool = rte_pktmbuf_pool_create("header_pool", NB_HDR_MBUF, 32,
132 0, HDR_MBUF_DATA_SIZE, rte_socket_id());
133 clone_pool = rte_pktmbuf_pool_create("clone_pool", NB_CLONE_MBUF, 32,
134 0, 0, rte_socket_id());
136 The reason for this is because indirect buffers are not supposed to hold any packet data and
137 therefore can be initialized with lower amount of reserved memory for each buffer.
142 The hash object is created and loaded with the pre-configured entries read from a global array:
148 init_mcast_hash(void)
151 mcast_hash_params.socket_id = rte_socket_id();
153 mcast_hash = rte_fbk_hash_create(&mcast_hash_params);
154 if (mcast_hash == NULL){
158 for (i = 0; i < N_MCAST_GROUPS; i ++){
159 if (rte_fbk_hash_add_key(mcast_hash, mcast_group_table[i].ip, mcast_group_table[i].port_mask) < 0) {
169 All forwarding is done inside the mcast_forward() function.
170 Firstly, the Ethernet* header is removed from the packet and the IPv4 address is extracted from the IPv4 header:
174 /* Remove the Ethernet header from the input packet */
176 iphdr = (struct ipv4_hdr *)rte_pktmbuf_adj(m, sizeof(struct ether_hdr));
177 RTE_ASSERT(iphdr != NULL);
178 dest_addr = rte_be_to_cpu_32(iphdr->dst_addr);
180 Then, the packet is checked to see if it has a multicast destination address and
181 if the routing table has any ports assigned to the destination address:
185 if (!IS_IPV4_MCAST(dest_addr) ||
186 (hash = rte_fbk_hash_lookup(mcast_hash, dest_addr)) <= 0 ||
187 (port_mask = hash & enabled_port_mask) == 0) {
192 Then, the number of ports in the destination portmask is calculated with the help of the bitcnt() function:
196 /* Get number of bits set. */
198 static inline uint32_t bitcnt(uint32_t v)
202 for (n = 0; v != 0; v &= v - 1, n++)
207 This is done to determine which forwarding algorithm to use.
208 This is explained in more detail in the next section.
210 Thereafter, a destination Ethernet address is constructed:
214 /* construct destination Ethernet address */
216 dst_eth_addr = ETHER_ADDR_FOR_IPV4_MCAST(dest_addr);
218 Since Ethernet addresses are also part of the multicast process, each outgoing packet carries the same destination Ethernet address.
219 The destination Ethernet address is constructed from the lower 23 bits of the multicast group OR-ed
220 with the Ethernet address 01:00:5e:00:00:00, as per RFC 1112:
224 #define ETHER_ADDR_FOR_IPV4_MCAST(x) \
225 (rte_cpu_to_be_64(0x01005e000000ULL | ((x) & 0x7fffff)) >> 16)
227 Then, packets are dispatched to the destination ports according to the portmask associated with a multicast group:
231 for (port = 0; use_clone != port_mask; port_mask >>= 1, port++) {
232 /* Prepare output packet and send it out. */
234 if ((port_mask & 1) != 0) {
235 if (likely ((mc = mcast_out_pkt(m, use_clone)) != NULL))
236 mcast_send_pkt(mc, &dst_eth_addr.as_addr, qconf, port);
237 else if (use_clone == 0)
242 The actual packet transmission is done in the mcast_send_pkt() function:
246 static inline void mcast_send_pkt(struct rte_mbuf *pkt, struct ether_addr *dest_addr, struct lcore_queue_conf *qconf, uint16_t port)
248 struct ether_hdr *ethdr;
251 /* Construct Ethernet header. */
253 ethdr = (struct ether_hdr *)rte_pktmbuf_prepend(pkt, (uint16_t) sizeof(*ethdr));
255 RTE_ASSERT(ethdr != NULL);
257 ether_addr_copy(dest_addr, ðdr->d_addr);
258 ether_addr_copy(&ports_eth_addr[port], ðdr->s_addr);
259 ethdr->ether_type = rte_be_to_cpu_16(ETHER_TYPE_IPv4);
261 /* Put new packet into the output queue */
263 len = qconf->tx_mbufs[port].len;
264 qconf->tx_mbufs[port].m_table[len] = pkt;
265 qconf->tx_mbufs[port].len = ++len;
267 /* Transmit packets */
269 if (unlikely(MAX_PKT_BURST == len))
270 send_burst(qconf, port);
276 This is the most important part of the application since it demonstrates the use of zero- copy buffer cloning.
277 There are two approaches for creating the outgoing packet and although both are based on the data zero-copy idea,
278 there are some differences in the detail.
280 The first approach creates a clone of the input packet, for example,
281 walk though all segments of the input packet and for each of segment,
282 create a new buffer and attach that new buffer to the segment
283 (refer to rte_pktmbuf_clone() in the rte_mbuf library for more details).
284 A new buffer is then allocated for the packet header and is prepended to the cloned buffer.
286 The second approach does not make a clone, it just increments the reference counter for all input packet segment,
287 allocates a new buffer for the packet header and prepends it to the input packet.
289 Basically, the first approach reuses only the input packet's data, but creates its own copy of packet's metadata.
290 The second approach reuses both input packet's data and metadata.
292 The advantage of first approach is that each outgoing packet has its own copy of the metadata,
293 so we can safely modify the data pointer of the input packet.
294 That allows us to skip creation if the output packet is for the last destination port
295 and instead modify input packet's header in place.
296 For example, for N destination ports, we need to invoke mcast_out_pkt() (N-1) times.
298 The advantage of the second approach is that there is less work to be done for each outgoing packet,
299 that is, the "clone" operation is skipped completely.
300 However, there is a price to pay.
301 The input packet's metadata must remain intact, so for N destination ports,
302 we need to invoke mcast_out_pkt() (N) times.
304 Therefore, for a small number of outgoing ports (and segments in the input packet),
305 first approach is faster.
306 As the number of outgoing ports (and/or input segments) grows, the second approach becomes more preferable.
308 Depending on the number of segments or the number of ports in the outgoing portmask,
309 either the first (with cloning) or the second (without cloning) approach is taken:
313 use_clone = (port_num <= MCAST_CLONE_PORTS && m->pkt.nb_segs <= MCAST_CLONE_SEGS);
315 It is the mcast_out_pkt() function that performs the packet duplication (either with or without actually cloning the buffers):
319 static inline struct rte_mbuf *mcast_out_pkt(struct rte_mbuf *pkt, int use_clone)
321 struct rte_mbuf *hdr;
323 /* Create new mbuf for the header. */
325 if (unlikely ((hdr = rte_pktmbuf_alloc(header_pool)) == NULL))
328 /* If requested, then make a new clone packet. */
330 if (use_clone != 0 && unlikely ((pkt = rte_pktmbuf_clone(pkt, clone_pool)) == NULL)) {
331 rte_pktmbuf_free(hdr);
335 /* prepend new header */
339 /* update header's fields */
341 hdr->pkt.pkt_len = (uint16_t)(hdr->pkt.data_len + pkt->pkt.pkt_len);
342 hdr->pkt.nb_segs = pkt->pkt.nb_segs + 1;
344 /* copy metadata from source packet */
346 hdr->pkt.in_port = pkt->pkt.in_port;
347 hdr->pkt.vlan_macip = pkt->pkt.vlan_macip;
348 hdr->pkt.hash = pkt->pkt.hash;
349 hdr->ol_flags = pkt->ol_flags;
350 rte_mbuf_sanity_check(hdr, RTE_MBUF_PKT, 1);