2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 IP Reassembly Sample Application
32 ================================
34 The L3 Forwarding application is a simple example of packet processing using the DPDK.
35 The application performs L3 forwarding with reassembly for fragmented IPv4 and IPv6 packets.
40 The application demonstrates the use of the DPDK libraries to implement packet forwarding
41 with reassembly for IPv4 and IPv6 fragmented packets.
42 The initialization and run- time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
43 The main difference from the L2 Forwarding sample application is that
44 it reassembles fragmented IPv4 and IPv6 packets before forwarding.
45 The maximum allowed size of reassembled packet is 9.5 KB.
47 There are two key differences from the L2 Forwarding sample application:
49 * The first difference is that the forwarding decision is taken based on information read from the input packet's IP header.
51 * The second difference is that the application differentiates between IP and non-IP traffic by means of offload flags.
53 The Longest Prefix Match (LPM for IPv4, LPM6 for IPv6) table is used to store/lookup an outgoing port number, associated with that IPv4 address. Any unmatched packets are forwarded to the originating port.Compiling the Application
54 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
57 Compiling the Application
58 -------------------------
60 To compile the sample application see :doc:`compiling`.
62 The application is located in the ``ip_reassembly`` sub-directory.
65 Running the Application
66 -----------------------
68 The application has a number of command line options:
70 .. code-block:: console
72 ./build/ip_reassembly [EAL options] -- -p PORTMASK [-q NQ] [--maxflows=FLOWS>] [--flowttl=TTL[(s|ms)]]
76 * -p PORTMASK: Hexadecimal bitmask of ports to configure
78 * -q NQ: Number of RX queues per lcore
80 * --maxflows=FLOWS: determines maximum number of active fragmented flows (1-65535). Default value: 4096.
82 * --flowttl=TTL[(s|ms)]: determines maximum Time To Live for fragmented packet.
83 If all fragments of the packet wouldn't appear within given time-out,
84 then they are considered as invalid and will be dropped.
85 Valid range is 1ms - 3600s. Default value: 1s.
87 To run the example in linuxapp environment with 2 lcores (2,4) over 2 ports(0,2) with 1 RX queue per lcore:
89 .. code-block:: console
91 ./build/ip_reassembly -l 2,4 -n 3 -- -p 5
92 EAL: coremask set to 14
93 EAL: Detected lcore 0 on socket 0
94 EAL: Detected lcore 1 on socket 1
95 EAL: Detected lcore 2 on socket 0
96 EAL: Detected lcore 3 on socket 1
97 EAL: Detected lcore 4 on socket 0
100 Initializing port 0 on lcore 2... Address:00:1B:21:76:FA:2C, rxq=0 txq=2,0 txq=4,1
101 done: Link Up - speed 10000 Mbps - full-duplex
102 Skipping disabled port 1
103 Initializing port 2 on lcore 4... Address:00:1B:21:5C:FF:54, rxq=0 txq=2,0 txq=4,1
104 done: Link Up - speed 10000 Mbps - full-duplex
105 Skipping disabled port 3IP_FRAG: Socket 0: adding route 100.10.0.0/16 (port 0)
106 IP_RSMBL: Socket 0: adding route 100.20.0.0/16 (port 1)
109 IP_RSMBL: Socket 0: adding route 0101:0101:0101:0101:0101:0101:0101:0101/48 (port 0)
110 IP_RSMBL: Socket 0: adding route 0201:0101:0101:0101:0101:0101:0101:0101/48 (port 1)
113 IP_RSMBL: entering main loop on lcore 4
114 IP_RSMBL: -- lcoreid=4 portid=2
115 IP_RSMBL: entering main loop on lcore 2
116 IP_RSMBL: -- lcoreid=2 portid=0
118 To run the example in linuxapp environment with 1 lcore (4) over 2 ports(0,2) with 2 RX queues per lcore:
120 .. code-block:: console
122 ./build/ip_reassembly -l 4 -n 3 -- -p 5 -q 2
124 To test the application, flows should be set up in the flow generator that match the values in the
125 l3fwd_ipv4_route_array and/or l3fwd_ipv6_route_array table.
127 Please note that in order to test this application,
128 the traffic generator should be generating valid fragmented IP packets.
129 For IPv6, the only supported case is when no other extension headers other than
130 fragment extension header are present in the packet.
132 The default l3fwd_ipv4_route_array table is:
136 struct l3fwd_ipv4_route l3fwd_ipv4_route_array[] = {
137 {IPv4(100, 10, 0, 0), 16, 0},
138 {IPv4(100, 20, 0, 0), 16, 1},
139 {IPv4(100, 30, 0, 0), 16, 2},
140 {IPv4(100, 40, 0, 0), 16, 3},
141 {IPv4(100, 50, 0, 0), 16, 4},
142 {IPv4(100, 60, 0, 0), 16, 5},
143 {IPv4(100, 70, 0, 0), 16, 6},
144 {IPv4(100, 80, 0, 0), 16, 7},
147 The default l3fwd_ipv6_route_array table is:
151 struct l3fwd_ipv6_route l3fwd_ipv6_route_array[] = {
152 {{1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 0},
153 {{2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 1},
154 {{3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 2},
155 {{4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 3},
156 {{5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 4},
157 {{6, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 5},
158 {{7, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 6},
159 {{8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 7},
162 For example, for the fragmented input IPv4 packet with destination address: 100.10.1.1,
163 a reassembled IPv4 packet be sent out from port #0 to the destination address 100.10.1.1
164 once all the fragments are collected.
169 The following sections provide some explanation of the sample application code.
170 As mentioned in the overview section, the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`.
171 The following sections describe aspects that are specific to the IP reassemble sample application.
173 IPv4 Fragment Table Initialization
174 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
176 This application uses the rte_ip_frag library. Please refer to Programmer's Guide for more detailed explanation of how to use this library.
177 Fragment table maintains information about already received fragments of the packet.
178 Each IP packet is uniquely identified by triple <Source IP address>, <Destination IP address>, <ID>.
179 To avoid lock contention, each RX queue has its own Fragment Table,
180 e.g. the application can't handle the situation when different fragments of the same packet arrive through different RX queues.
181 Each table entry can hold information about packet consisting of up to RTE_LIBRTE_IP_FRAG_MAX_FRAGS fragments.
185 frag_cycles = (rte_get_tsc_hz() + MS_PER_S - 1) / MS_PER_S * max_flow_ttl;
187 if ((qconf->frag_tbl[queue] = rte_ip_frag_tbl_create(max_flow_num, IPV4_FRAG_TBL_BUCKET_ENTRIES, max_flow_num, frag_cycles, socket)) == NULL)
189 RTE_LOG(ERR, IP_RSMBL, "ip_frag_tbl_create(%u) on " "lcore: %u for queue: %u failed\n", max_flow_num, lcore, queue);
193 Mempools Initialization
194 ~~~~~~~~~~~~~~~~~~~~~~~
196 The reassembly application demands a lot of mbuf's to be allocated.
197 At any given time up to (2 \* max_flow_num \* RTE_LIBRTE_IP_FRAG_MAX_FRAGS \* <maximum number of mbufs per packet>)
198 can be stored inside Fragment Table waiting for remaining fragments.
199 To keep mempool size under reasonable limits and to avoid situation when one RX queue can starve other queues,
200 each RX queue uses its own mempool.
204 nb_mbuf = RTE_MAX(max_flow_num, 2UL * MAX_PKT_BURST) * RTE_LIBRTE_IP_FRAG_MAX_FRAGS;
205 nb_mbuf *= (port_conf.rxmode.max_rx_pkt_len + BUF_SIZE - 1) / BUF_SIZE;
206 nb_mbuf *= 2; /* ipv4 and ipv6 */
207 nb_mbuf += RTE_TEST_RX_DESC_DEFAULT + RTE_TEST_TX_DESC_DEFAULT;
208 nb_mbuf = RTE_MAX(nb_mbuf, (uint32_t)NB_MBUF);
210 snprintf(buf, sizeof(buf), "mbuf_pool_%u_%u", lcore, queue);
212 if ((rxq->pool = rte_mempool_create(buf, nb_mbuf, MBUF_SIZE, 0, sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, NULL,
213 rte_pktmbuf_init, NULL, socket, MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET)) == NULL) {
215 RTE_LOG(ERR, IP_RSMBL, "mempool_create(%s) failed", buf);
219 Packet Reassembly and Forwarding
220 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
222 For each input packet, the packet forwarding operation is done by the l3fwd_simple_forward() function.
223 If the packet is an IPv4 or IPv6 fragment, then it calls rte_ipv4_reassemble_packet() for IPv4 packets,
224 or rte_ipv6_reassemble_packet() for IPv6 packets.
225 These functions either return a pointer to valid mbuf that contains reassembled packet,
226 or NULL (if the packet can't be reassembled for some reason).
227 Then l3fwd_simple_forward() continues with the code for the packet forwarding decision
228 (that is, the identification of the output interface for the packet) and
229 actual transmit of the packet.
231 The rte_ipv4_reassemble_packet() or rte_ipv6_reassemble_packet() are responsible for:
233 #. Searching the Fragment Table for entry with packet's <IP Source Address, IP Destination Address, Packet ID>
235 #. If the entry is found, then check if that entry already timed-out.
236 If yes, then free all previously received fragments,
237 and remove information about them from the entry.
239 #. If no entry with such key is found, then try to create a new one by one of two ways:
241 #. Use as empty entry
243 #. Delete a timed-out entry, free mbufs associated with it mbufs and store a new entry with specified key in it.
245 #. Update the entry with new fragment information and check
246 if a packet can be reassembled (the packet's entry contains all fragments).
248 #. If yes, then, reassemble the packet, mark table's entry as empty and return the reassembled mbuf to the caller.
250 #. If no, then just return a NULL to the caller.
252 If at any stage of packet processing a reassembly function encounters an error
253 (can't insert new entry into the Fragment table, or invalid/timed-out fragment),
254 then it will free all associated with the packet fragments,
255 mark the table entry as invalid and return NULL to the caller.
257 Debug logging and Statistics Collection
258 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260 The RTE_LIBRTE_IP_FRAG_TBL_STAT controls statistics collection for the IP Fragment Table.
261 This macro is disabled by default.
262 To make ip_reassembly print the statistics to the standard output,
263 the user must send either an USR1, INT or TERM signal to the process.
264 For all of these signals, the ip_reassembly process prints Fragment table statistics for each RX queue,
265 plus the INT and TERM will cause process termination as usual.