2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 L3 Forwarding with Access Control Sample Application
32 ====================================================
34 The L3 Forwarding with Access Control application is a simple example of packet processing using the DPDK.
35 The application performs a security check on received packets.
36 Packets that are in the Access Control List (ACL), which is loaded during initialization, are dropped.
37 Others are forwarded to the correct port.
42 The application demonstrates the use of the ACL library in the DPDK to implement access control
43 and packet L3 forwarding.
44 The application loads two types of rules at initialization:
46 * Route information rules, which are used for L3 forwarding
48 * Access Control List (ACL) rules that blacklist (or block) packets with a specific characteristic
50 When packets are received from a port,
51 the application extracts the necessary information from the TCP/IP header of the received packet and
52 performs a lookup in the rule database to figure out whether the packets should be dropped (in the ACL range)
53 or forwarded to desired ports.
54 The initialization and run-time paths are similar to those of the L3 forwarding application
55 (see Chapter 10, "L3 Forwarding Sample Application" for more information).
56 However, there are significant differences in the two applications.
57 For example, the original L3 forwarding application uses either LPM or
58 an exact match algorithm to perform forwarding port lookup,
59 while this application uses the ACL library to perform both ACL and route entry lookup.
60 The following sections provide more detail.
62 Classification for both IPv4 and IPv6 packets is supported in this application.
63 The application also assumes that all the packets it processes are TCP/UDP packets and
64 always extracts source/destination port information from the packets.
69 The application implements packet classification for the IPv4/IPv6 5-tuple syntax specifically.
70 The 5-tuple syntax consist of a source IP address, a destination IP address,
71 a source port, a destination port and a protocol identifier.
72 The fields in the 5-tuple syntax have the following formats:
74 * **Source IP address and destination IP address**
75 : Each is either a 32-bit field (for IPv4), or a set of 4 32-bit fields (for IPv6) represented by a value and a mask length.
76 For example, an IPv4 range of 192.168.1.0 to 192.168.1.255 could be represented by a value = [192, 168, 1, 0] and a mask length = 24.
78 * **Source port and destination port**
79 : Each is a 16-bit field, represented by a lower start and a higher end.
80 For example, a range of ports 0 to 8192 could be represented by lower = 0 and higher = 8192.
82 * **Protocol identifier**
83 : An 8-bit field, represented by a value and a mask, that covers a range of values.
84 To verify that a value is in the range, use the following expression: "(VAL & mask) == value"
86 The trick in how to represent a range with a mask and value is as follows.
87 A range can be enumerated in binary numbers with some bits that are never changed and some bits that are dynamically changed.
88 Set those bits that dynamically changed in mask and value with 0.
89 Set those bits that never changed in the mask with 1, in value with number expected.
90 For example, a range of 6 to 7 is enumerated as 0b110 and 0b111.
91 Bit 1-7 are bits never changed and bit 0 is the bit dynamically changed.
92 Therefore, set bit 0 in mask and value with 0, set bits 1-7 in mask with 1, and bits 1-7 in value with number 0b11.
93 So, mask is 0xfe, value is 0x6.
97 The library assumes that each field in the rule is in LSB or Little Endian order when creating the database.
98 It internally converts them to MSB or Big Endian order.
99 When performing a lookup, the library assumes the input is in MSB or Big Endian order.
104 In this sample application, each rule is a combination of the following:
106 * 5-tuple field: This field has a format described in Section.
108 * priority field: A weight to measure the priority of the rules.
109 The rule with the higher priority will ALWAYS be returned if the specific input has multiple matches in the rule database.
110 Rules with lower priority will NEVER be returned in any cases.
112 * userdata field: A user-defined field that could be any value.
113 It can be the forwarding port number if the rule is a route table entry or it can be a pointer to a mapping address
114 if the rule is used for address mapping in the NAT application.
115 The key point is that it is a useful reserved field for user convenience.
120 The application needs to acquire ACL and route rules before it runs.
121 Route rules are mandatory, while ACL rules are optional.
122 To simplify the complexity of the priority field for each rule, all ACL and route entries are assumed to be in the same file.
123 To read data from the specified file successfully, the application assumes the following:
125 * Each rule occupies a single line.
127 * Only the following four rule line types are valid in this application:
129 * ACL rule line, which starts with a leading character '@'
131 * Route rule line, which starts with a leading character 'R'
133 * Comment line, which starts with a leading character '#'
135 * Empty line, which consists of a space, form-feed ('\f'), newline ('\n'),
136 carriage return ('\r'), horizontal tab ('\t'), or vertical tab ('\v').
138 Other lines types are considered invalid.
140 * Rules are organized in descending order of priority,
141 which means rules at the head of the file always have a higher priority than those further down in the file.
143 * A typical IPv4 ACL rule line should have a format as shown below:
146 .. _figure_ipv4_acl_rule:
148 .. figure:: img/ipv4_acl_rule.*
150 A typical IPv4 ACL rule
153 IPv4 addresses are specified in CIDR format as specified in RFC 4632.
154 They consist of the dot notation for the address and a prefix length separated by '/'.
155 For example, 192.168.0.34/32, where the address is 192.168.0.34 and the prefix length is 32.
157 Ports are specified as a range of 16-bit numbers in the format MIN:MAX,
158 where MIN and MAX are the inclusive minimum and maximum values of the range.
159 The range 0:65535 represents all possible ports in a range.
160 When MIN and MAX are the same value, a single port is represented, for example, 20:20.
162 The protocol identifier is an 8-bit value and a mask separated by '/'.
163 For example: 6/0xfe matches protocol values 6 and 7.
165 * Route rules start with a leading character 'R' and have the same format as ACL rules except an extra field at the tail
166 that indicates the forwarding port number.
171 .. _figure_example_rules:
173 .. figure:: img/example_rules.*
178 Each rule is explained as follows:
180 * Rule 1 (the first line) tells the application to drop those packets with source IP address = [1.2.3.*],
181 destination IP address = [192.168.0.36], protocol = [6]/[7]
183 * Rule 2 (the second line) is similar to Rule 1, except the source IP address is ignored.
184 It tells the application to forward packets with destination IP address = [192.168.0.36],
185 protocol = [6]/[7], destined to port 1.
187 * Rule 3 (the third line) tells the application to forward all packets to port 0.
188 This is something like a default route entry.
190 As described earlier, the application assume rules are listed in descending order of priority,
191 therefore Rule 1 has the highest priority, then Rule 2, and finally,
192 Rule 3 has the lowest priority.
194 Consider the arrival of the following three packets:
196 * Packet 1 has source IP address = [1.2.3.4], destination IP address = [192.168.0.36], and protocol = [6]
198 * Packet 2 has source IP address = [1.2.4.4], destination IP address = [192.168.0.36], and protocol = [6]
200 * Packet 3 has source IP address = [1.2.3.4], destination IP address = [192.168.0.36], and protocol = [8]
204 * Packet 1 matches all of the rules
206 * Packet 2 matches Rule 2 and Rule 3
208 * Packet 3 only matches Rule 3
210 For priority reasons, Packet 1 matches Rule 1 and is dropped.
211 Packet 2 matches Rule 2 and is forwarded to port 1.
212 Packet 3 matches Rule 3 and is forwarded to port 0.
214 For more details on the rule file format,
215 please refer to rule_ipv4.db and rule_ipv6.db files (inside <RTE_SDK>/examples/l3fwd-acl/).
220 Once the application starts, it transitions through three phases:
222 * **Initialization Phase**
223 - Perform the following tasks:
225 * Parse command parameters. Check the validity of rule file(s) name(s), number of logical cores, receive and transmit queues.
226 Bind ports, queues and logical cores. Check ACL search options, and so on.
228 * Call Environmental Abstraction Layer (EAL) and Poll Mode Driver (PMD) functions to initialize the environment and detect possible NICs.
229 The EAL creates several threads and sets affinity to a specific hardware thread CPU based on the configuration specified
230 by the command line arguments.
232 * Read the rule files and format the rules into the representation that the ACL library can recognize.
233 Call the ACL library function to add the rules into the database and compile them as a trie of pattern sets.
234 Note that application maintains a separate AC contexts for IPv4 and IPv6 rules.
237 - Process the incoming packets from a port. Packets are processed in three steps:
239 * Retrieval: Gets a packet from the receive queue. Each logical core may process several queues for different ports.
240 This depends on the configuration specified by command line arguments.
242 * Lookup: Checks that the packet type is supported (IPv4/IPv6) and performs a 5-tuple lookup over corresponding AC context.
243 If an ACL rule is matched, the packets will be dropped and return back to step 1.
244 If a route rule is matched, it indicates the packet is not in the ACL list and should be forwarded.
245 If there is no matches for the packet, then the packet is dropped.
247 * Forwarding: Forwards the packet to the corresponding port.
249 * **Final Phase** - Perform the following tasks:
251 Calls the EAL, PMD driver and ACL library to free resource, then quits.
253 Compiling the Application
254 -------------------------
256 To compile the application:
258 #. Go to the sample application directory:
260 .. code-block:: console
262 export RTE_SDK=/path/to/rte_sdk
263 cd ${RTE_SDK}/examples/l3fwd-acl
265 #. Set the target (a default target is used if not specified). For example:
267 .. code-block:: console
269 export RTE_TARGET=x86_64-native-linuxapp-gcc
271 See the *DPDK IPL Getting Started Guide* for possible RTE_TARGET values.
273 #. Build the application:
275 .. code-block:: console
279 Running the Application
280 -----------------------
282 The application has a number of command line options:
284 .. code-block:: console
286 ./build/l3fwd-acl [EAL options] -- -p PORTMASK [-P] --config(port,queue,lcore)[,(port,queue,lcore)] --rule_ipv4 FILENAME rule_ipv6 FILENAME [--scalar] [--enable-jumbo [--max-pkt-len PKTLEN]] [--no-numa]
291 * -p PORTMASK: Hexadecimal bitmask of ports to configure
293 * -P: Sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
294 Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
296 * --config (port,queue,lcore)[,(port,queue,lcore)]: determines which queues from which ports are mapped to which cores
298 * --rule_ipv4 FILENAME: Specifies the IPv4 ACL and route rules file
300 * --rule_ipv6 FILENAME: Specifies the IPv6 ACL and route rules file
302 * --scalar: Use a scalar function to perform rule lookup
304 * --enable-jumbo: optional, enables jumbo frames
306 * --max-pkt-len: optional, maximum packet length in decimal (64-9600)
308 * --no-numa: optional, disables numa awareness
310 As an example, consider a dual processor socket platform where cores 0, 2, 4, 6, 8 and 10 appear on socket 0,
311 while cores 1, 3, 5, 7, 9 and 11 appear on socket 1.
312 Let's say that the user wants to use memory from both NUMA nodes,
313 the platform has only two ports and the user wants to use two cores from each processor socket to do the packet processing.
315 To enable L3 forwarding between two ports, using two cores from each processor,
316 while also taking advantage of local memory access by optimizing around NUMA,
317 the user must enable two queues from each port,
318 pin to the appropriate cores and allocate memory from the appropriate NUMA node.
319 This is achieved using the following command:
321 .. code-block:: console
323 ./build/l3fwd-acl -c f -n 4 -- -p 0x3 --config="(0,0,0),(0,1,2),(1,0,1),(1,1,3)" --rule_ipv4="./rule_ipv4.db" -- rule_ipv6="./rule_ipv6.db" --scalar
327 * The -c option enables cores 0, 1, 2, 3
329 * The -p option enables ports 0 and 1
331 * The --config option enables two queues on each port and maps each (port,queue) pair to a specific core.
332 Logic to enable multiple RX queues using RSS and to allocate memory from the correct NUMA nodes is included in the application
333 and is done transparently.
334 The following table shows the mapping in this example:
336 +----------+------------+-----------+------------------------------------------------+
337 | **Port** | **Queue** | **lcore** | **Description** |
339 +==========+============+===========+================================================+
340 | 0 | 0 | 0 | Map queue 0 from port 0 to lcore 0. |
342 +----------+------------+-----------+------------------------------------------------+
343 | 0 | 1 | 2 | Map queue 1 from port 0 to lcore 2. |
345 +----------+------------+-----------+------------------------------------------------+
346 | 1 | 0 | 1 | Map queue 0 from port 1 to lcore 1. |
348 +----------+------------+-----------+------------------------------------------------+
349 | 1 | 1 | 3 | Map queue 1 from port 1 to lcore 3. |
351 +----------+------------+-----------+------------------------------------------------+
353 * The --rule_ipv4 option specifies the reading of IPv4 rules sets from the ./ rule_ipv4.db file.
355 * The --rule_ipv6 option specifies the reading of IPv6 rules sets from the ./ rule_ipv6.db file.
357 * The --scalar option specifies the performing of rule lookup with a scalar function.
362 The following sections provide some explanation of the sample application code.
363 The aspects of port, device and CPU configuration are similar to those of the L3 forwarding application
364 (see Chapter 10, "L3 Forwarding Sample Application" for more information).
365 The following sections describe aspects that are specific to L3 forwarding with access control.
367 Parse Rules from File
368 ~~~~~~~~~~~~~~~~~~~~~
370 As described earlier, both ACL and route rules are assumed to be saved in the same file.
371 The application parses the rules from the file and adds them to the database by calling the ACL library function.
372 It ignores empty and comment lines, and parses and validates the rules it reads.
373 If errors are detected, the application exits with messages to identify the errors encountered.
375 The application needs to consider the userdata and priority fields.
376 The ACL rules save the index to the specific rules in the userdata field,
377 while route rules save the forwarding port number.
378 In order to differentiate the two types of rules, ACL rules add a signature in the userdata field.
379 As for the priority field, the application assumes rules are organized in descending order of priority.
380 Therefore, the code only decreases the priority number with each rule it parses.
382 Setting Up the ACL Context
383 ~~~~~~~~~~~~~~~~~~~~~~~~~~
385 For each supported AC rule format (IPv4 5-tuple, IPv6 6-tuple) application creates a separate context handler
386 from the ACL library for each CPU socket on the board and adds parsed rules into that context.
388 Note, that for each supported rule type,
389 application needs to calculate the expected offset of the fields from the start of the packet.
390 That's why only packets with fixed IPv4/ IPv6 header are supported.
391 That allows to perform ACL classify straight over incoming packet buffer -
392 no extra protocol field retrieval need to be performed.
394 Subsequently, the application checks whether NUMA is enabled.
395 If it is, the application records the socket IDs of the CPU cores involved in the task.
397 Finally, the application creates contexts handler from the ACL library,
398 adds rules parsed from the file into the database and build an ACL trie.
399 It is important to note that the application creates an independent copy of each database for each socket CPU
400 involved in the task to reduce the time for remote memory access.