1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2016-2017 Intel Corporation.
4 IPsec Security Gateway Sample Application
5 =========================================
7 The IPsec Security Gateway application is an example of a "real world"
8 application using DPDK cryptodev framework.
13 The application demonstrates the implementation of a Security Gateway
14 (not IPsec compliant, see the Constraints section below) using DPDK based on RFC4301,
15 RFC4303, RFC3602 and RFC2404.
17 Internet Key Exchange (IKE) is not implemented, so only manual setting of
18 Security Policies and Security Associations is supported.
20 The Security Policies (SP) are implemented as ACL rules, the Security
21 Associations (SA) are stored in a table and the routing is implemented
24 The application classifies the ports as *Protected* and *Unprotected*.
25 Thus, traffic received on an Unprotected or Protected port is consider
26 Inbound or Outbound respectively.
28 The application also supports complete IPsec protocol offload to hardware
29 (Look aside crypto accelerator or using ethernet device). It also support
30 inline ipsec processing by the supported ethernet device during transmission.
31 These modes can be selected during the SA creation configuration.
33 In case of complete protocol offload, the processing of headers(ESP and outer
34 IP header) is done by the hardware and the application does not need to
35 add/remove them during outbound/inbound processing.
37 For inline offloaded outbound traffic, the application will not do the LPM
38 lookup for routing, as the port on which the packet has to be forwarded will be
39 part of the SA. Security parameters will be configured on that port only, and
40 sending the packet on other ports could result in unencrypted packets being
43 The Path for IPsec Inbound traffic is:
45 * Read packets from the port.
46 * Classify packets between IPv4 and ESP.
47 * Perform Inbound SA lookup for ESP packets based on their SPI.
48 * Perform Verification/Decryption (Not needed in case of inline ipsec).
49 * Remove ESP and outer IP header (Not needed in case of protocol offload).
50 * Inbound SP check using ACL of decrypted packets and any other IPv4 packets.
52 * Write packet to port.
54 The Path for the IPsec Outbound traffic is:
56 * Read packets from the port.
57 * Perform Outbound SP check using ACL of all IPv4 traffic.
58 * Perform Outbound SA lookup for packets that need IPsec protection.
59 * Add ESP and outer IP header (Not needed in case protocol offload).
60 * Perform Encryption/Digest (Not needed in case of inline ipsec).
62 * Write packet to port.
68 * No IPv6 options headers.
70 * Supported algorithms: AES-CBC, AES-CTR, AES-GCM, 3DES-CBC, HMAC-SHA1 and NULL.
71 * Each SA must be handle by a unique lcore (*1 RX queue per port*).
73 Compiling the Application
74 -------------------------
76 To compile the sample application see :doc:`compiling`.
78 The application is located in the ``ipsec-secgw`` sub-directory.
80 #. [Optional] Build the application for debugging:
81 This option adds some extra flags, disables compiler optimizations and
87 Running the Application
88 -----------------------
90 The application has a number of command line options::
93 ./build/ipsec-secgw [EAL options] --
94 -p PORTMASK -P -u PORTMASK -j FRAMESIZE
95 -l -w REPLAY_WINOW_SIZE -e -a
97 --config (port,queue,lcore)[,(port,queue,lcore]
107 * ``-p PORTMASK``: Hexadecimal bitmask of ports to configure.
109 * ``-P``: *optional*. Sets all ports to promiscuous mode so that packets are
110 accepted regardless of the packet's Ethernet MAC destination address.
111 Without this option, only packets with the Ethernet MAC destination address
112 set to the Ethernet address of the port are accepted (default is enabled).
114 * ``-u PORTMASK``: hexadecimal bitmask of unprotected ports
116 * ``-j FRAMESIZE``: *optional*. data buffer size (in bytes),
117 in other words maximum data size for one segment.
118 Packets with length bigger then FRAMESIZE still can be received,
119 but will be segmented.
120 Default value: RTE_MBUF_DEFAULT_BUF_SIZE (2176)
121 Minimum value: RTE_MBUF_DEFAULT_BUF_SIZE (2176)
122 Maximum value: UINT16_MAX (65535).
124 * ``-l``: enables code-path that uses librte_ipsec.
126 * ``-w REPLAY_WINOW_SIZE``: specifies the IPsec sequence number replay window
127 size for each Security Association (available only with librte_ipsec
130 * ``-e``: enables Security Association extended sequence number processing
131 (available only with librte_ipsec code path).
133 * ``-a``: enables Security Association sequence number atomic behavior
134 (available only with librte_ipsec code path).
136 * ``-c``: specifies the SAD cache size. Stores the most recent SA in a per
137 lcore cache. Cache represents flat array containing SA's indexed by SPI.
138 Zero value disables cache.
141 * ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues
142 from which ports are mapped to which cores.
144 * ``--single-sa SAIDX``: use a single SA for outbound traffic, bypassing the SP
145 on both Inbound and Outbound. This option is meant for debugging/performance
148 * ``--rxoffload MASK``: RX HW offload capabilities to enable/use on this port
149 (bitmask of DEV_RX_OFFLOAD_* values). It is an optional parameter and
150 allows user to disable some of the RX HW offload capabilities.
151 By default all HW RX offloads are enabled.
153 * ``--txoffload MASK``: TX HW offload capabilities to enable/use on this port
154 (bitmask of DEV_TX_OFFLOAD_* values). It is an optional parameter and
155 allows user to disable some of the TX HW offload capabilities.
156 By default all HW TX offloads are enabled.
158 * ``--mtu MTU``: MTU value (in bytes) on all attached ethernet ports.
159 Outgoing packets with length bigger then MTU will be fragmented.
160 Incoming packets with length bigger then MTU will be discarded.
163 * ``--frag-ttl FRAG_TTL_NS``: fragment lifetime (in nanoseconds).
164 If packet is not reassembled within this time, received fragments
165 will be discarded. Fragment lifetime should be decreased when
166 there is a high fragmented traffic loss in high bandwidth networks.
167 Should be lower for low number of reassembly buckets.
168 Valid values: from 1 ns to 10 s. Default value: 10000000 (10 s).
170 * ``--reassemble NUM``: max number of entries in reassemble fragment table.
171 Zero value disables reassembly functionality.
174 * ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
175 configuration items for running the application (See Configuration file
176 syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
177 **ONLY** the UNIX format configuration file is accepted.
180 The mapping of lcores to port/queues is similar to other l3fwd applications.
182 For example, given the following command line::
184 ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
185 --vdev "crypto_null" -- -p 0xf -P -u 0x3 \
186 --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \
187 -f /path/to/config_file \
189 where each options means:
191 * The ``-l`` option enables cores 20 and 21.
193 * The ``-n`` option sets memory 4 channels.
195 * The ``--socket-mem`` to use 2GB on socket 1.
197 * The ``--vdev "crypto_null"`` option creates virtual NULL cryptodev PMD.
199 * The ``-p`` option enables ports (detected) 0, 1, 2 and 3.
201 * The ``-P`` option enables promiscuous mode.
203 * The ``-u`` option sets ports 1 and 2 as unprotected, leaving 2 and 3 as protected.
205 * The ``--config`` option enables one queue per port with the following mapping:
207 +----------+-----------+-----------+---------------------------------------+
208 | **Port** | **Queue** | **lcore** | **Description** |
210 +----------+-----------+-----------+---------------------------------------+
211 | 0 | 0 | 20 | Map queue 0 from port 0 to lcore 20. |
213 +----------+-----------+-----------+---------------------------------------+
214 | 1 | 0 | 20 | Map queue 0 from port 1 to lcore 20. |
216 +----------+-----------+-----------+---------------------------------------+
217 | 2 | 0 | 21 | Map queue 0 from port 2 to lcore 21. |
219 +----------+-----------+-----------+---------------------------------------+
220 | 3 | 0 | 21 | Map queue 0 from port 3 to lcore 21. |
222 +----------+-----------+-----------+---------------------------------------+
224 * The ``-f /path/to/config_file`` option enables the application read and
225 parse the configuration file specified, and configures the application
226 with a given set of SP, SA and Routing entries accordingly. The syntax of
227 the configuration file will be explained below in more detail. Please
228 **note** the parser only accepts UNIX format text file. Other formats
229 such as DOS/MAC format will cause a parse error.
231 Refer to the *DPDK Getting Started Guide* for general information on running
232 applications and the Environment Abstraction Layer (EAL) options.
234 The application would do a best effort to "map" crypto devices to cores, with
235 hardware devices having priority. Basically, hardware devices if present would
236 be assigned to a core before software ones.
237 This means that if the application is using a single core and both hardware
238 and software crypto devices are detected, hardware devices will be used.
240 A way to achieve the case where you want to force the use of virtual crypto
241 devices is to whitelist the Ethernet devices needed and therefore implicitly
242 blacklisting all hardware crypto devices.
244 For example, something like the following command line:
246 .. code-block:: console
248 ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
249 -w 81:00.0 -w 81:00.1 -w 81:00.2 -w 81:00.3 \
250 --vdev "crypto_aesni_mb" --vdev "crypto_null" \
252 -p 0xf -P -u 0x3 --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \
259 The following sections provide the syntax of configurations to initialize
260 your SP, SA, Routing and Neighbour tables.
261 Configurations shall be specified in the configuration file to be passed to
262 the application. The file is then parsed by the application. The successful
263 parsing will result in the appropriate rules being applied to the tables
267 Configuration File Syntax
268 ~~~~~~~~~~~~~~~~~~~~~~~~~
270 As mention in the overview, the Security Policies are ACL rules.
271 The application parsers the rules specified in the configuration file and
272 passes them to the ACL table, and replicates them per socket in use.
274 Following are the configuration file syntax.
279 The parse treats one line in the configuration file as one configuration
280 item (unless the line concatenation symbol exists). Every configuration
281 item shall follow the syntax of either SP, SA, Routing or Neighbour
282 rules specified below.
284 The configuration parser supports the following special symbols:
286 * Comment symbol **#**. Any character from this symbol to the end of
287 line is treated as comment and will not be parsed.
289 * Line concatenation symbol **\\**. This symbol shall be placed in the end
290 of the line to be concatenated to the line below. Multiple lines'
291 concatenation is supported.
297 The SP rule syntax is shown as follows:
299 .. code-block:: console
301 sp <ip_ver> <dir> esp <action> <priority> <src_ip> <dst_ip>
302 <proto> <sport> <dport>
305 where each options means:
309 * IP protocol version
315 * *ipv4*: IP protocol version 4
316 * *ipv6*: IP protocol version 6
320 * The traffic direction
326 * *in*: inbound traffic
327 * *out*: outbound traffic
337 * *protect <SA_idx>*: the specified traffic is protected by SA rule
339 * *bypass*: the specified traffic traffic is bypassed
340 * *discard*: the specified traffic is discarded
346 * Optional: Yes, default priority 0 will be used
352 * The source IP address and mask
354 * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
358 * *src X.X.X.X/Y* for IPv4
359 * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
363 * The destination IP address and mask
365 * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
369 * *dst X.X.X.X/Y* for IPv4
370 * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
374 * The protocol start and end range
376 * Optional: yes, default range of 0 to 0 will be used
378 * Syntax: *proto X:Y*
382 * The source port start and end range
384 * Optional: yes, default range of 0 to 0 will be used
386 * Syntax: *sport X:Y*
390 * The destination port start and end range
392 * Optional: yes, default range of 0 to 0 will be used
394 * Syntax: *dport X:Y*
398 .. code-block:: console
400 sp ipv4 out esp protect 105 pri 1 dst 192.168.115.0/24 sport 0:65535 \
403 sp ipv6 in esp bypass pri 1 dst 0000:0000:0000:0000:5555:5555:\
404 0000:0000/96 sport 0:65535 dport 0:65535
410 The successfully parsed SA rules will be stored in an array table.
412 The SA rule syntax is shown as follows:
414 .. code-block:: console
416 sa <dir> <spi> <cipher_algo> <cipher_key> <auth_algo> <auth_key>
417 <mode> <src_ip> <dst_ip> <action_type> <port_id> <fallback>
419 where each options means:
423 * The traffic direction
429 * *in*: inbound traffic
430 * *out*: outbound traffic
438 * Syntax: unsigned integer number
444 * Optional: Yes, unless <aead_algo> is not used
448 * *null*: NULL algorithm
449 * *aes-128-cbc*: AES-CBC 128-bit algorithm
450 * *aes-256-cbc*: AES-CBC 256-bit algorithm
451 * *aes-128-ctr*: AES-CTR 128-bit algorithm
452 * *3des-cbc*: 3DES-CBC 192-bit algorithm
454 * Syntax: *cipher_algo <your algorithm>*
458 * Cipher key, NOT available when 'null' algorithm is used
460 * Optional: Yes, unless <aead_algo> is not used.
461 Must be followed by <cipher_algo> option
463 * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
464 The number of bytes should be as same as the specified cipher algorithm
467 For example: *cipher_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
472 * Authentication algorithm
474 * Optional: Yes, unless <aead_algo> is not used
478 * *null*: NULL algorithm
479 * *sha1-hmac*: HMAC SHA1 algorithm
483 * Authentication key, NOT available when 'null' or 'aes-128-gcm' algorithm
486 * Optional: Yes, unless <aead_algo> is not used.
487 Must be followed by <auth_algo> option
489 * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
490 The number of bytes should be as same as the specified authentication
493 For example: *auth_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
500 * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used
504 * *aes-128-gcm*: AES-GCM 128-bit algorithm
506 * Syntax: *cipher_algo <your algorithm>*
510 * Cipher key, NOT available when 'null' algorithm is used
512 * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used.
513 Must be followed by <aead_algo> option
515 * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
516 The number of bytes should be as same as the specified AEAD algorithm
519 For example: *aead_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
530 * *ipv4-tunnel*: Tunnel mode for IPv4 packets
531 * *ipv6-tunnel*: Tunnel mode for IPv6 packets
532 * *transport*: transport mode
538 * The source IP address. This option is not available when
539 transport mode is used
541 * Optional: Yes, default address 0.0.0.0 will be used
545 * *src X.X.X.X* for IPv4
546 * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6
550 * The destination IP address. This option is not available when
551 transport mode is used
553 * Optional: Yes, default address 0.0.0.0 will be used
557 * *dst X.X.X.X* for IPv4
558 * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6
562 * Action type to specify the security action. This option specify
563 the SA to be performed with look aside protocol offload to HW
564 accelerator or protocol offload on ethernet device or inline
565 crypto processing on the ethernet device during transmission.
567 * Optional: Yes, default type *no-offload*
571 * *lookaside-protocol-offload*: look aside protocol offload to HW accelerator
572 * *inline-protocol-offload*: inline protocol offload on ethernet device
573 * *inline-crypto-offload*: inline crypto processing on ethernet device
574 * *no-offload*: no offloading to hardware
578 * Port/device ID of the ethernet/crypto accelerator for which the SA is
579 configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
580 port will be used for routing. The routing table will not be referred in
583 * Optional: No, if *type* is not *no-offload*
587 * *port_id X* X is a valid device number in decimal
591 * Action type for ingress IPsec packets that inline processor failed to
592 process. Only a combination of *inline-crypto-offload* as a primary
593 session and *lookaside-none* as a fall-back session is supported at the
596 If used in conjunction with IPsec window, its width needs be increased
597 due to different processing times of inline and lookaside modes which
598 results in packet reordering.
604 * *lookaside-none*: use automatically chosen cryptodev to process packets
608 * *fallback lookaside-none*
612 .. code-block:: console
614 sa out 5 cipher_algo null auth_algo null mode ipv4-tunnel \
615 src 172.16.1.5 dst 172.16.2.5
617 sa out 25 cipher_algo aes-128-cbc \
618 cipher_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \
619 auth_algo sha1-hmac \
620 auth_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \
622 src 1111:1111:1111:1111:1111:1111:1111:5555 \
623 dst 2222:2222:2222:2222:2222:2222:2222:5555
625 sa in 105 aead_algo aes-128-gcm \
626 aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
627 mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5
629 sa out 5 cipher_algo aes-128-cbc cipher_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \
630 auth_algo sha1-hmac auth_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \
631 mode ipv4-tunnel src 172.16.1.5 dst 172.16.2.5 \
632 type lookaside-protocol-offload port_id 4
634 sa in 35 aead_algo aes-128-gcm \
635 aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
636 mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 \
637 type inline-crypto-offload port_id 0
642 The Routing rule syntax is shown as follows:
644 .. code-block:: console
646 rt <ip_ver> <src_ip> <dst_ip> <port>
649 where each options means:
653 * IP protocol version
659 * *ipv4*: IP protocol version 4
660 * *ipv6*: IP protocol version 6
664 * The source IP address and mask
666 * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
670 * *src X.X.X.X/Y* for IPv4
671 * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
675 * The destination IP address and mask
677 * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
681 * *dst X.X.X.X/Y* for IPv4
682 * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
686 * The traffic output port id
688 * Optional: yes, default output port 0 will be used
694 .. code-block:: console
696 rt ipv4 dst 172.16.1.5/32 port 0
698 rt ipv6 dst 1111:1111:1111:1111:1111:1111:1111:5555/116 port 0
700 Neighbour rule syntax
701 ^^^^^^^^^^^^^^^^^^^^^
703 The Neighbour rule syntax is shown as follows:
705 .. code-block:: console
707 neigh <port> <dst_mac>
710 where each options means:
722 * The destination ethernet address to use for that port
730 Example Neighbour rules:
732 .. code-block:: console
734 neigh port 0 DE:AD:BE:EF:01:02
739 The test directory contains scripts for testing the various encryption
742 The purpose of the scripts is to automate ipsec-secgw testing
743 using another system running linux as a DUT.
745 The user must setup the following environment variables:
747 * ``SGW_PATH``: path to the ipsec-secgw binary to test.
749 * ``REMOTE_HOST``: IP address/hostname of the DUT.
751 * ``REMOTE_IFACE``: interface name for the test-port on the DUT.
753 * ``ETH_DEV``: ethernet device to be used on the SUT by DPDK ('-w <pci-id>')
755 Also the user can optionally setup:
757 * ``SGW_LCORE``: lcore to run ipsec-secgw on (default value is 0)
759 * ``CRYPTO_DEV``: crypto device to be used ('-w <pci-id>'). If none specified
760 appropriate vdevs will be created by the script
762 * ``MULTI_SEG_TEST``: ipsec-secgw option to enable reassembly support and
763 specify size of reassembly table (e.g.
764 ``MULTI_SEG_TEST='--reassemble 128'``). This option must be set for
765 fallback session tests.
767 Note that most of the tests require the appropriate crypto PMD/device to be
773 Two servers are required for the tests, SUT and DUT.
775 Make sure the user from the SUT can ssh to the DUT without entering the password.
776 To enable this feature keys must be setup on the DUT.
778 ``ssh-keygen`` will make a private & public key pair on the SUT.
780 ``ssh-copy-id`` <user name>@<target host name> on the SUT will copy the public
781 key to the DUT. It will ask for credentials so that it can upload the public key.
783 The SUT and DUT are connected through at least 2 NIC ports.
785 One NIC port is expected to be managed by linux on both machines and will be
786 used as a control path.
788 The second NIC port (test-port) should be bound to DPDK on the SUT, and should
789 be managed by linux on the DUT.
791 The script starts ``ipsec-secgw`` with 2 NIC devices: ``test-port`` and
794 It then configures the local tap interface and the remote interface and IPsec
795 policies in the following way:
797 Traffic going over the test-port in both directions has to be protected by IPsec.
799 Traffic going over the TAP port in both directions does not have to be protected.
803 DUT OS(NIC1)--(IPsec)-->(NIC1)ipsec-secgw(TAP)--(plain)-->(TAP)SUT OS
805 SUT OS(TAP)--(plain)-->(TAP)psec-secgw(NIC1)--(IPsec)-->(NIC1)DUT OS
807 It then tries to perform some data transfer using the scheme described above.
812 In the ipsec-secgw/test directory
814 to run one test for IPv4 or IPv6
816 /bin/bash linux_test(4|6).sh <ipsec_mode>
818 to run all tests for IPv4 or IPv6
820 /bin/bash run_test.sh -4|-6
822 For the list of available modes please refer to run_test.sh.