2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
5 Redistribution and use in source and binary forms, with or without
6 modification, are permitted provided that the following conditions
9 * Redistributions of source code must retain the above copyright
10 notice, this list of conditions and the following disclaimer.
11 * Redistributions in binary form must reproduce the above copyright
12 notice, this list of conditions and the following disclaimer in
13 the documentation and/or other materials provided with the
15 * Neither the name of Intel Corporation nor the names of its
16 contributors may be used to endorse or promote products derived
17 from this software without specific prior written permission.
19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 Exception Path Sample Application
32 =================================
34 The Exception Path sample application is a simple example that demonstrates the use of the DPDK
35 to set up an exception path for packets to go through the Linux* kernel.
36 This is done by using virtual TAP network interfaces.
37 These can be read from and written to by the DPDK application and
38 appear to the kernel as a standard network interface.
43 The application creates two threads for each NIC port being used.
44 One thread reads from the port and writes the data unmodified to a thread-specific TAP interface.
45 The second thread reads from a TAP interface and writes the data unmodified to the NIC port.
47 The packet flow through the exception path application is as shown in the following figure.
51 **Figure 1. Packet Flow**
53 .. image2_png has been replaced
55 |exception_path_example|
57 To make throughput measurements, kernel bridges must be setup to forward data between the bridges appropriately.
59 Compiling the Application
60 -------------------------
62 #. Go to example directory:
64 .. code-block:: console
66 export RTE_SDK=/path/to/rte_sdk
67 cd ${RTE_SDK}/examples/exception_path
69 #. Set the target (a default target will be used if not specified).
72 .. code-block:: console
74 export RTE_TARGET=x86_64-native-linuxapp-gcc
76 This application is intended as a linuxapp only.
77 See the *DPDK Getting Started Guide* for possible RTE_TARGET values.
79 #. Build the application:
81 .. code-block:: console
85 Running the Application
86 -----------------------
88 The application requires a number of command line options:
90 .. code-block:: console
92 .build/exception_path [EAL options] -- -p PORTMASK -i IN_CORES -o OUT_CORES
96 * -p PORTMASK: A hex bitmask of ports to use
98 * -i IN_CORES: A hex bitmask of cores which read from NIC
100 * -o OUT_CORES: A hex bitmask of cores which write to NIC
102 Refer to the *DPDK Getting Started Guide* for general information on running applications
103 and the Environment Abstraction Layer (EAL) options.
105 The number of bits set in each bitmask must be the same.
106 The coremask -c parameter of the EAL options should include IN_CORES and OUT_CORES.
107 The same bit must not be set in IN_CORES and OUT_CORES.
108 The affinities between ports and cores are set beginning with the least significant bit of each mask, that is,
109 the port represented by the lowest bit in PORTMASK is read from by the core represented by the lowest bit in IN_CORES,
110 and written to by the core represented by the lowest bit in OUT_CORES.
112 For example to run the application with two ports and four cores:
114 .. code-block:: console
116 ./build/exception_path -c f -n 4 -- -p 3 -i 3 -o c
121 While the application is running, statistics on packets sent and
122 received can be displayed by sending the SIGUSR1 signal to the application from another terminal:
124 .. code-block:: console
126 killall -USR1 exception_path
128 The statistics can be reset by sending a SIGUSR2 signal in a similar way.
133 The following sections provide some explanation of the code.
138 Setup of the mbuf pool, driver and queues is similar to the setup done in the L2 Forwarding sample application
139 (see Chapter 9 "L2 forwarding Sample Application (in Real and Virtualized Environments" for details).
140 In addition, the TAP interfaces must also be created.
141 A TAP interface is created for each lcore that is being used.
142 The code for creating the TAP interface is as follows:
147 * Create a tap network interface, or use existing one with same name.
148 * If name[0]='\0' then a name is automatically assigned and returned in name.
151 static int tap_create(char *name)
156 fd = open("/dev/net/tun", O_RDWR);
160 memset(&ifr, 0, sizeof(ifr));
162 /* TAP device without packet information */
164 ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
166 rte_snprinf(ifr.ifr_name, IFNAMSIZ, name);
168 ret = ioctl(fd, TUNSETIFF, (void *) &ifr);
177 rte_snprintf(name, IFNAMSIZ, ifr.ifr_name);
182 The other step in the initialization process that is unique to this sample application
183 is the association of each port with two cores:
185 * One core to read from the port and write to a TAP interface
187 * A second core to read from a TAP interface and write to the port
189 This is done using an array called port_ids[], which is indexed by the lcore IDs.
190 The population of this array is shown below:
197 RTE_LCORE_FOREACH(i) {
198 if (input_cores_mask & (1ULL << i)) {
199 /* Skip ports that are not enabled */
200 while ((ports_mask & (1 << rx_port)) == 0) {
202 if (rx_port > (sizeof(ports_mask) * 8))
203 goto fail; /* not enough ports */
205 port_ids[i] = rx_port++;
206 } else if (output_cores_mask & (1ULL << i)) {
207 /* Skip ports that are not enabled */
208 while ((ports_mask & (1 << tx_port)) == 0) {
210 if (tx_port > (sizeof(ports_mask) * 8))
211 goto fail; /* not enough ports */
213 port_ids[i] = tx_port++;
220 After the initialization steps are complete, the main_loop() function is run on each lcore.
221 This function first checks the lcore_id against the user provided input_cores_mask and output_cores_mask to see
222 if this core is reading from or writing to a TAP interface.
224 For the case that reads from a NIC port, the packet reception is the same as in the L2 Forwarding sample application
225 (see Section 9.4.6, "Receive, Process and Transmit Packets").
226 The packet transmission is done by calling write() with the file descriptor of the appropriate TAP interface
227 and then explicitly freeing the mbuf back to the pool.
231 /* Loop forever reading from NIC and writing to tap */
234 struct rte_mbuf *pkts_burst[PKT_BURST_SZ];
237 const unsigned nb_rx = rte_eth_rx_burst(port_ids[lcore_id], 0, pkts_burst, PKT_BURST_SZ);
239 lcore_stats[lcore_id].rx += nb_rx;
241 for (i = 0; likely(i < nb_rx); i++) {
242 struct rte_mbuf *m = pkts_burst[i];
243 int ret = write(tap_fd, rte_pktmbuf_mtod(m, void*),
245 rte_pktmbuf_data_len(m));
248 lcore_stats[lcore_id].dropped++;
250 lcore_stats[lcore_id].tx++;
254 For the other case that reads from a TAP interface and writes to a NIC port,
255 packets are retrieved by doing a read() from the file descriptor of the appropriate TAP interface.
256 This fills in the data into the mbuf, then other fields are set manually.
257 The packet can then be transmitted as normal.
261 /* Loop forever reading from tap and writing to NIC */
265 struct rte_mbuf *m = rte_pktmbuf_alloc(pktmbuf_pool);
270 ret = read(tap_fd, m->pkt.data, MAX_PACKET_SZ); lcore_stats[lcore_id].rx++;
271 if (unlikely(ret < 0)) {
272 FATAL_ERROR("Reading from %s interface failed", tap_name);
277 m->pkt.data_len = (uint16_t)ret;
279 ret = rte_eth_tx_burst(port_ids[lcore_id], 0, &m, 1);
280 if (unlikely(ret < 1)) {
282 lcore_stats[lcore_id].dropped++;
285 lcore_stats[lcore_id].tx++;
289 To set up loops for measuring throughput, TAP interfaces can be connected using bridging.
290 The steps to do this are described in the section that follows.
292 Managing TAP Interfaces and Bridges
293 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
295 The Exception Path sample application creates TAP interfaces with names of the format tap_dpdk_nn,
296 where nn is the lcore ID. These TAP interfaces need to be configured for use:
298 .. code-block:: console
300 ifconfig tap_dpdk_00 up
302 To set up a bridge between two interfaces so that packets sent to one interface can be read from another,
305 .. code-block:: console
308 brctl addif br0 tap_dpdk_00
309 brctl addif br0 tap_dpdk_03
312 The TAP interfaces created by this application exist only when the application is running,
313 so the steps above need to be repeated each time the application is run.
314 To avoid this, persistent TAP interfaces can be created using openvpn:
316 .. code-block:: console
318 openvpn --mktun --dev tap_dpdk_00
320 If this method is used, then the steps above have to be done only once and
321 the same TAP interfaces can be reused each time the application is run.
322 To remove bridges and persistent TAP interfaces, the following commands are used:
324 .. code-block:: console
328 openvpn --rmtun --dev tap_dpdk_00
330 .. |exception_path_example| image:: img/exception_path_example.*