doc: add event mode to ipsec-secgw
[dpdk.git] / doc / guides / sample_app_ug / ipsec_secgw.rst
index fcb33c2..038f593 100644 (file)
@@ -1,32 +1,6 @@
-..  BSD LICENSE
-    Copyright(c) 2016 Intel Corporation. All rights reserved.
-    All rights reserved.
-
-    Redistribution and use in source and binary forms, with or without
-    modification, are permitted provided that the following conditions
-    are met:
-
-    * Redistributions of source code must retain the above copyright
-    notice, this list of conditions and the following disclaimer.
-    * Redistributions in binary form must reproduce the above copyright
-    notice, this list of conditions and the following disclaimer in
-    the documentation and/or other materials provided with the
-    distribution.
-    * Neither the name of Intel Corporation nor the names of its
-    contributors may be used to endorse or promote products derived
-    from this software without specific prior written permission.
-
-    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2016-2017 Intel Corporation.
+    Copyright (C) 2020 Marvell International Ltd.
 
 IPsec Security Gateway Sample Application
 =========================================
@@ -52,13 +26,28 @@ The application classifies the ports as *Protected* and *Unprotected*.
 Thus, traffic received on an Unprotected or Protected port is consider
 Inbound or Outbound respectively.
 
+The application also supports complete IPsec protocol offload to hardware
+(Look aside crypto accelerator or using ethernet device). It also support
+inline ipsec processing by the supported ethernet device during transmission.
+These modes can be selected during the SA creation configuration.
+
+In case of complete protocol offload, the processing of headers(ESP and outer
+IP header) is done by the hardware and the application does not need to
+add/remove them during outbound/inbound processing.
+
+For inline offloaded outbound traffic, the application will not do the LPM
+lookup for routing, as the port on which the packet has to be forwarded will be
+part of the SA. Security parameters will be configured on that port only, and
+sending the packet on other ports could result in unencrypted packets being
+sent out.
+
 The Path for IPsec Inbound traffic is:
 
 *  Read packets from the port.
 *  Classify packets between IPv4 and ESP.
 *  Perform Inbound SA lookup for ESP packets based on their SPI.
-*  Perform Verification/Decryption.
-*  Remove ESP and outer IP header
+*  Perform Verification/Decryption (Not needed in case of inline ipsec).
+*  Remove ESP and outer IP header (Not needed in case of protocol offload).
 *  Inbound SP check using ACL of decrypted packets and any other IPv4 packets.
 *  Routing.
 *  Write packet to port.
@@ -68,42 +57,64 @@ The Path for the IPsec Outbound traffic is:
 *  Read packets from the port.
 *  Perform Outbound SP check using ACL of all IPv4 traffic.
 *  Perform Outbound SA lookup for packets that need IPsec protection.
-*  Add ESP and outer IP header.
-*  Perform Encryption/Digest.
+*  Add ESP and outer IP header (Not needed in case protocol offload).
+*  Perform Encryption/Digest (Not needed in case of inline ipsec).
 *  Routing.
 *  Write packet to port.
 
+The application supports two modes of operation: poll mode and event mode.
+
+* In the poll mode a core receives packets from statically configured list
+  of eth ports and eth ports' queues.
+
+* In the event mode a core receives packets as events. After packet processing
+  is done core submits them back as events to an event device. This enables
+  multicore scaling and HW assisted scheduling by making use of the event device
+  capabilities. The event mode configuration is predefined. All packets reaching
+  given eth port will arrive at the same event queue. All event queues are mapped
+  to all event ports. This allows all cores to receive traffic from all ports.
+  Since the underlying event device might have varying capabilities, the worker
+  threads can be drafted differently to maximize performance. For example, if an
+  event device - eth device pair has Tx internal port, then application can call
+  rte_event_eth_tx_adapter_enqueue() instead of regular rte_event_enqueue_burst().
+  So a thread which assumes that the device pair has internal port will not be the
+  right solution for another pair. The infrastructure added for the event mode aims
+  to help application to have multiple worker threads by maximizing performance from
+  every type of event device without affecting existing paths/use cases. The worker
+  to be used will be determined by the operating conditions and the underlying device
+  capabilities. **Currently the application provides non-burst, internal port worker
+  threads and supports inline protocol only.** It also provides infrastructure for
+  non-internal port however does not define any worker threads.
+
+Additionally the event mode introduces two submodes of processing packets:
+
+* Driver submode: This submode has bare minimum changes in the application to support
+  IPsec. There are no lookups, no routing done in the application. And for inline
+  protocol use case, the worker thread resembles l2fwd worker thread as the IPsec
+  processing is done entirely in HW. This mode can be used to benchmark the raw
+  performance of the HW. The driver submode is selected with --single-sa option
+  (used also by poll mode). When --single-sa option is used in conjution with event
+  mode then index passed to --single-sa is ignored.
+
+* App submode: This submode has all the features currently implemented with the
+  application (non librte_ipsec path). All the lookups, routing follows existing
+  methods and report numbers that can be compared against regular poll mode
+  benchmark numbers.
 
 Constraints
 -----------
 
 *  No IPv6 options headers.
 *  No AH mode.
-*  Currently only EAS-CBC, HMAC-SHA1 and NULL.
+*  Supported algorithms: AES-CBC, AES-CTR, AES-GCM, 3DES-CBC, HMAC-SHA1 and NULL.
 *  Each SA must be handle by a unique lcore (*1 RX queue per port*).
-*  No chained mbufs.
-
 
 Compiling the Application
 -------------------------
 
-To compile the application:
-
-#. Go to the sample application directory::
-
-      export RTE_SDK=/path/to/rte_sdk
-      cd ${RTE_SDK}/examples/ipsec-secgw
-
-#. Set the target (a default target is used if not specified). For example::
-
-
-      export RTE_TARGET=x86_64-native-linuxapp-gcc
+To compile the sample application see :doc:`compiling`.
 
-   See the *DPDK Getting Started Guide* for possible RTE_TARGET values.
-
-#. Build the application::
-
-       make
+The application is located in the ``ipsec-secgw`` sub-directory.
 
 #. [Optional] Build the application for debugging:
    This option adds some extra flags, disables compiler optimizations and
@@ -119,10 +130,21 @@ The application has a number of command line options::
 
 
    ./build/ipsec-secgw [EAL options] --
-                        -p PORTMASK -P -u PORTMASK
-                        --config (port,queue,lcore)[,(port,queue,lcore]
+                        -p PORTMASK -P -u PORTMASK -j FRAMESIZE
+                        -l -w REPLAY_WINOW_SIZE -e -a
+                        -c SAD_CACHE_SIZE
+                        -s NUMBER_OF_MBUFS_IN_PACKET_POOL
+                        -f CONFIG_FILE_PATH
+                        --config (port,queue,lcore)[,(port,queue,lcore)]
                         --single-sa SAIDX
-                       --ep0|--ep1
+                        --cryptodev_mask MASK
+                        --transfer-mode MODE
+                        --event-schedule-type TYPE
+                        --rxoffload MASK
+                        --txoffload MASK
+                        --reassemble NUM
+                        --mtu MTU
+                        --frag-ttl FRAG_TTL_NS
 
 Where:
 
@@ -135,31 +157,100 @@ Where:
 
 *   ``-u PORTMASK``: hexadecimal bitmask of unprotected ports
 
-*   ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues
-    from which ports are mapped to which cores.
-
-*   ``--single-sa SAIDX``: use a single SA for outbound traffic, bypassing the SP
-    on both Inbound and Outbound. This option is meant for debugging/performance
-    purposes.
-
-*   ``--ep0``: configure the app as Endpoint 0.
+*   ``-j FRAMESIZE``: *optional*. data buffer size (in bytes),
+    in other words maximum data size for one segment.
+    Packets with length bigger then FRAMESIZE still can be received,
+    but will be segmented.
+    Default value: RTE_MBUF_DEFAULT_BUF_SIZE (2176)
+    Minimum value: RTE_MBUF_DEFAULT_BUF_SIZE (2176)
+    Maximum value: UINT16_MAX (65535).
+
+*   ``-l``: enables code-path that uses librte_ipsec.
+
+*   ``-w REPLAY_WINOW_SIZE``: specifies the IPsec sequence number replay window
+    size for each Security Association (available only with librte_ipsec
+    code path).
+
+*   ``-e``: enables Security Association extended sequence number processing
+    (available only with librte_ipsec code path).
+
+*   ``-a``: enables Security Association sequence number atomic behavior
+    (available only with librte_ipsec code path).
+
+*   ``-c``: specifies the SAD cache size. Stores the most recent SA in a per
+    lcore cache. Cache represents flat array containing SA's indexed by SPI.
+    Zero value disables cache.
+    Default value: 128.
+
+*   ``-s``: sets number of mbufs in packet pool, if not provided number of mbufs
+    will be calculated based on number of cores, eth ports and crypto queues.
+
+*   ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
+    configuration items for running the application (See Configuration file
+    syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
+    **ONLY** the UNIX format configuration file is accepted.
+
+*   ``--config (port,queue,lcore)[,(port,queue,lcore)]``: in poll mode determines
+    which queues from which ports are mapped to which cores. In event mode this
+    option is not used as packets are dynamically scheduled to cores by HW.
+
+*   ``--single-sa SAIDX``: in poll mode use a single SA for outbound traffic,
+    bypassing the SP on both Inbound and Outbound. This option is meant for
+    debugging/performance purposes. In event mode selects driver submode, SA index
+    value is ignored.
+
+*   ``--cryptodev_mask MASK``: hexadecimal bitmask of the crypto devices
+    to configure.
+
+*   ``--transfer-mode MODE``: sets operating mode of the application
+    "poll"  : packet transfer via polling (default)
+    "event" : Packet transfer via event device
+
+*   ``--event-schedule-type TYPE``: queue schedule type, applies only when
+    --transfer-mode is set to event.
+    "ordered"  : Ordered (default)
+    "atomic"   : Atomic
+    "parallel" : Parallel
+    When --event-schedule-type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
+    device will ensure the ordering. Ordering will be lost when tried in PARALLEL.
+
+*   ``--rxoffload MASK``: RX HW offload capabilities to enable/use on this port
+    (bitmask of DEV_RX_OFFLOAD_* values). It is an optional parameter and
+    allows user to disable some of the RX HW offload capabilities.
+    By default all HW RX offloads are enabled.
+
+*   ``--txoffload MASK``: TX HW offload capabilities to enable/use on this port
+    (bitmask of DEV_TX_OFFLOAD_* values). It is an optional parameter and
+    allows user to disable some of the TX HW offload capabilities.
+    By default all HW TX offloads are enabled.
+
+*   ``--reassemble NUM``: max number of entries in reassemble fragment table.
+    Zero value disables reassembly functionality.
+    Default value: 0.
+
+*   ``--mtu MTU``: MTU value (in bytes) on all attached ethernet ports.
+    Outgoing packets with length bigger then MTU will be fragmented.
+    Incoming packets with length bigger then MTU will be discarded.
+    Default value: 1500.
+
+*   ``--frag-ttl FRAG_TTL_NS``: fragment lifetime (in nanoseconds).
+    If packet is not reassembled within this time, received fragments
+    will be discarded. Fragment lifetime should be decreased when
+    there is a high fragmented traffic loss in high bandwidth networks.
+    Should be lower for low number of reassembly buckets.
+    Valid values: from 1 ns to 10 s. Default value: 10000000 (10 s).
 
-*   ``--ep1``: configure the app as Endpoint 1.
-
-Either one of ``--ep0`` or ``--ep1`` **must** be specified.
-The main purpose of these options is to easily configure two systems
-back-to-back that would forward traffic through an IPsec tunnel (see
-:ref:`figure_ipsec_endpoints`).
 
 The mapping of lcores to port/queues is similar to other l3fwd applications.
 
-For example, given the following command line::
+For example, given the following command line to run application in poll mode::
 
     ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048       \
-           --vdev "cryptodev_null_pmd" -- -p 0xf -P -u 0x3      \
-           --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" --ep0 \
+           --vdev "crypto_null" -- -p 0xf -P -u 0x3             \
+           --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)"       \
+           -f /path/to/config_file --transfer-mode poll         \
 
-where each options means:
+where each option means:
 
 *   The ``-l`` option enables cores 20 and 21.
 
@@ -167,13 +258,13 @@ where each options means:
 
 *   The ``--socket-mem`` to use 2GB on socket 1.
 
-*   The ``--vdev "cryptodev_null_pmd"`` option creates virtual NULL cryptodev PMD.
+*   The ``--vdev "crypto_null"`` option creates virtual NULL cryptodev PMD.
 
 *   The ``-p`` option enables ports (detected) 0, 1, 2 and 3.
 
 *   The ``-P`` option enables promiscuous mode.
 
-*   The ``-u`` option sets ports 1 and 2 as unprotected, leaving 2 and 3 as protected.
+*   The ``-u`` option sets ports 0 and 1 as unprotected, leaving 2 and 3 as protected.
 
 *   The ``--config`` option enables one queue per port with the following mapping:
 
@@ -194,8 +285,39 @@ where each options means:
     |          |           |           |                                       |
     +----------+-----------+-----------+---------------------------------------+
 
-*   The ``--ep0`` options configures the app with a given set of SP, SA and Routing
-    entries as explained below in more detail.
+*   The ``-f /path/to/config_file`` option enables the application read and
+    parse the configuration file specified, and configures the application
+    with a given set of SP, SA and Routing entries accordingly. The syntax of
+    the configuration file will be explained below in more detail. Please
+    **note** the parser only accepts UNIX format text file. Other formats
+    such as DOS/MAC format will cause a parse error.
+
+*   The ``--transfer-mode`` option selects poll mode for processing packets.
+
+Similarly for example, given the following command line to run application in
+event app mode::
+
+    ./build/ipsec-secgw -c 0x3 -- -P -p 0x3 -u 0x1       \
+           -f /path/to/config_file --transfer-mode event \
+           --event-schedule-type parallel                \
+
+where each option means:
+
+*   The ``-c`` option selects cores 0 and 1 to run on.
+
+*   The ``-P`` option enables promiscuous mode.
+
+*   The ``-p`` option enables ports (detected) 0 and 1.
+
+*   The ``-u`` option sets ports 0 as unprotected, leaving 1 as protected.
+
+*   The ``-f /path/to/config_file`` option has the same behavior as in poll
+    mode example.
+
+*   The ``--transfer-mode`` option selects event mode for processing packets.
+
+*   The ``--event-schedule-type`` option selects parallel ordering of event queues.
+
 
 Refer to the *DPDK Getting Started Guide* for general information on running
 applications and the Environment Abstraction Layer (EAL) options.
@@ -216,499 +338,576 @@ For example, something like the following command line:
 
     ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
             -w 81:00.0 -w 81:00.1 -w 81:00.2 -w 81:00.3 \
-            --vdev "cryptodev_aesni_mb_pmd" --vdev "cryptodev_null_pmd" \
+            --vdev "crypto_aesni_mb" --vdev "crypto_null" \
            -- \
             -p 0xf -P -u 0x3 --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \
-            --ep0
+            -f sample.cfg
 
 
 Configurations
 --------------
 
-The following sections provide some details on the default values used to
-initialize the SP, SA and Routing tables.
-Currently all configuration information is hard coded into the application.
+The following sections provide the syntax of configurations to initialize
+your SP, SA, Routing and Neighbour tables.
+Configurations shall be specified in the configuration file to be passed to
+the application. The file is then parsed by the application. The successful
+parsing will result in the appropriate rules being applied to the tables
+accordingly.
 
-The following image illustrate a few of the concepts regarding IPSec, such
-as protected/unprotected and inbound/outbound traffic, from the point of
-view of two back-to-back endpoints:
 
-.. _figure_ipsec_endpoints:
+Configuration File Syntax
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. figure:: img/ipsec_endpoints.*
+As mention in the overview, the Security Policies are ACL rules.
+The application parsers the rules specified in the configuration file and
+passes them to the ACL table, and replicates them per socket in use.
 
-   IPSec Inbound/Outbound traffic
+Following are the configuration file syntax.
 
-Note that the above image only displays unidirectional traffic per port
-for illustration purposes.
-The application supports bidirectional traffic on all ports,
+General rule syntax
+^^^^^^^^^^^^^^^^^^^
 
+The parse treats one line in the configuration file as one configuration
+item (unless the line concatenation symbol exists). Every configuration
+item shall follow the syntax of either SP, SA, Routing or Neighbour
+rules specified below.
 
-Security Policy Initialization
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The configuration parser supports the following special symbols:
 
-As mention in the overview, the Security Policies are ACL rules.
-The application defines two ACLs, one each of Inbound and Outbound, and
-it replicates them per socket in use.
-
-Following are the default rules which show only the relevant information,
-assuming ANY value is valid for the fields not mentioned (src ip, proto,
-src/dst ports).
-
-.. _table_ipsec_endpoint_outbound_sp:
-
-.. table:: Endpoint 0 Outbound Security Policies
-
-   +-----------------------------------+------------+
-   | **Dst**                           | **SA idx** |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.105.0/24                  | 5          |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.106.0/24                  | 6          |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.175.0/24                  | 10         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.176.0/24                  | 11         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.200.0/24                  | 15         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.201.0/24                  | 16         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.55.0/24                   | 25         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.56.0/24                   | 26         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.240.0/24                  | BYPASS     |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.241.0/24                  | BYPASS     |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:0:0:5555:5555:0:0/96          | 5          |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:0:0:6666:6666:0:0/96          | 6          |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:1111:1111:0:0:0:0/96          | 10         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:1111:1111:1111:1111:0:0/96    | 11         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:0:0:aaaa:aaaa:0:0/96          | 25         |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 0:0:0:0:bbbb:bbbb:0:0/96          | 26         |
-   |                                   |            |
-   +-----------------------------------+------------+
-
-.. _table_ipsec_endpoint_inbound_sp:
-
-.. table:: Endpoint 0 Inbound Security Policies
-
-   +-----------------------------------+------------+
-   | **Dst**                           | **SA idx** |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.115.0/24                  | 105        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.116.0/24                  | 106        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.185.0/24                  | 110        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.186.0/24                  | 111        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.210.0/24                  | 115        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.211.0/24                  | 116        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.65.0/24                   | 125        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.66.0/24                   | 126        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.245.0/24                  | BYPASS     |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | 192.168.246.0/24                  | BYPASS     |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:0:0:5555:5555:0:0/96       | 105        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:0:0:6666:6666:0:0/96       | 106        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:1111:1111:0:0:0:0/96       | 110        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:1111:1111:1111:1111:0:0/96 | 111        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:0:0:aaaa:aaaa:0:0/96       | 125        |
-   |                                   |            |
-   +-----------------------------------+------------+
-   | ffff:0:0:0:bbbb:bbbb:0:0/96       | 126        |
-   |                                   |            |
-   +-----------------------------------+------------+
-
-For Endpoint 1, we use the same policies in reverse, meaning the Inbound SP
-entries are set as Outbound and vice versa.
-
-
-Security Association Initialization
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The SAs are kept in a array table.
-
-For Inbound, the SPI is used as index modulo the table size.
-This means that on a table for 100 SA, SPI 5 and 105 would use the same index
-and that is not currently supported.
-
-Notice that it is not an issue for Outbound traffic as we store the index and
-not the SPI in the Security Policy.
-
-All SAs configured with AES-CBC and HMAC-SHA1 share the same values for cipher
-block size and key, and authentication digest size and key.
-
-The following are the default values:
-
-.. _table_ipsec_endpoint_outbound_sa:
-
-.. table:: Endpoint 0 Outbound Security Associations
-
-   +---------+----------+------------+-----------+----------------+----------------+
-   | **SPI** | **Mode** | **Cipher** | **Auth**  | **Tunnel src** | **Tunnel dst** |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 5       | Tunnel   | AES-CBC    | HMAC-SHA1 | 172.16.1.5     | 172.16.2.5     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 6       | Tunnel   | AES-CBC    | HMAC-SHA1 | 172.16.1.6     | 172.16.2.6     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 10      | Trans    | AES-CBC    | HMAC-SHA1 | N/A            | N/A            |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 11      | Trans    | AES-CBC    | HMAC-SHA1 | N/A            | N/A            |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 15      | Tunnel   | NULL       | NULL      | 172.16.1.5     | 172.16.2.5     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 16      | Tunnel   | NULL       | NULL      | 172.16.1.6     | 172.16.2.6     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 25      | Tunnel   | AES-CBC    | HMAC-SHA1 | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:5555      | 2222:5555      |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 26      | Tunnel   | AES-CBC    | HMAC-SHA1 | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:1111:     | 2222:2222:     |
-   |         |          |            |           | 1111:6666      | 2222:6666      |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-
-.. _table_ipsec_endpoint_inbound_sa:
-
-.. table:: Endpoint 0 Inbound Security Associations
-
-   +---------+----------+------------+-----------+----------------+----------------+
-   | **SPI** | **Mode** | **Cipher** | **Auth**  | **Tunnel src** | **Tunnel dst** |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 105     | Tunnel   | AES-CBC    | HMAC-SHA1 | 172.16.2.5     | 172.16.1.5     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 106     | Tunnel   | AES-CBC    | HMAC-SHA1 | 172.16.2.6     | 172.16.1.6     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 110     | Trans    | AES-CBC    | HMAC-SHA1 | N/A            | N/A            |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 111     | Trans    | AES-CBC    | HMAC-SHA1 | N/A            | N/A            |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 115     | Tunnel   | NULL       | NULL      | 172.16.2.5     | 172.16.1.5     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 116     | Tunnel   | NULL       | NULL      | 172.16.2.6     | 172.16.1.6     |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 125     | Tunnel   | AES-CBC    | HMAC-SHA1 | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:5555      | 1111:5555      |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-   | 126     | Tunnel   | AES-CBC    | HMAC-SHA1 | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:2222:     | 1111:1111:     |
-   |         |          |            |           | 2222:6666      | 1111:6666      |
-   |         |          |            |           |                |                |
-   +---------+----------+------------+-----------+----------------+----------------+
-
-For Endpoint 1, we use the same policies in reverse, meaning the Inbound SP
-entries are set as Outbound and vice versa.
-
-
-Routing Initialization
-~~~~~~~~~~~~~~~~~~~~~~
-
-The Routing is implemented using an LPM table.
-
-Following default values:
-
-.. _table_ipsec_endpoint_outbound_routing:
-
-.. table:: Endpoint 0 Routing Table
-
-   +------------------+----------+
-   | **Dst addr**     | **Port** |
-   |                  |          |
-   +------------------+----------+
-   | 172.16.2.5/32    | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 172.16.2.6/32    | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.175.0/24 | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.176.0/24 | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.240.0/24 | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.241.0/24 | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.115.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.116.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.65.0/24  | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.66.0/24  | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.185.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.186.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.210.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.211.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.245.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.246.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 2222:2222:       | 0        |
-   | 2222:2222:       |          |
-   | 2222:2222:       |          |
-   | 2222:5555/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 2222:2222:       | 1        |
-   | 2222:2222:       |          |
-   | 2222:2222:       |          |
-   | 2222:6666/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 0        |
-   | 1111:1111:       |          |
-   | 0000:0000:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 1        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 2        |
-   | 0000:0000:       |          |
-   | aaaa:aaaa:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 3        |
-   | 0000:0000:       |          |
-   | bbbb:bbbb:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 2        |
-   | 0000:0000:       |          |
-   | 5555:5555:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 3        |
-   | 0000:0000:       |          |
-   | 6666:6666:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 2        |
-   | 1111:1111:       |          |
-   | 0000:0000:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 3        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-
-.. _table_ipsec_endpoint_inbound_routing:
-
-.. table:: Endpoint 1 Routing Table
-
-   +------------------+----------+
-   | **Dst addr**     | **Port** |
-   |                  |          |
-   +------------------+----------+
-   | 172.16.1.5/32    | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 172.16.1.6/32    | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.185.0/24 | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.186.0/24 | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.245.0/24 | 0        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.246.0/24 | 1        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.105.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.106.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.55.0/24  | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.56.0/24  | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.175.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.176.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.200.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.201.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.240.0/24 | 2        |
-   |                  |          |
-   +------------------+----------+
-   | 192.168.241.0/24 | 3        |
-   |                  |          |
-   +------------------+----------+
-   | 1111:1111:       | 0        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 1111:5555/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 1111:1111:       | 1        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 1111:6666/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 0        |
-   | 1111:1111:       |          |
-   | 0000:0000:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | ffff:0000:       | 1        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 2        |
-   | 0000:0000:       |          |
-   | aaaa:aaaa:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 3        |
-   | 0000:0000:       |          |
-   | bbbb:bbbb:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 2        |
-   | 0000:0000:       |          |
-   | 5555:5555:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 3        |
-   | 0000:0000:       |          |
-   | 6666:6666:       |          |
-   | 0000:0/116       |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 2        |
-   | 1111:1111:       |          |
-   | 0000:0000:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
-   | 0000:0000:       | 3        |
-   | 1111:1111:       |          |
-   | 1111:1111:       |          |
-   | 0000:0000/116    |          |
-   |                  |          |
-   +------------------+----------+
+ * Comment symbol **#**. Any character from this symbol to the end of
+   line is treated as comment and will not be parsed.
+
+ * Line concatenation symbol **\\**. This symbol shall be placed in the end
+   of the line to be concatenated to the line below. Multiple lines'
+   concatenation is supported.
+
+
+SP rule syntax
+^^^^^^^^^^^^^^
+
+The SP rule syntax is shown as follows:
+
+.. code-block:: console
+
+    sp <ip_ver> <dir> esp <action> <priority> <src_ip> <dst_ip>
+    <proto> <sport> <dport>
+
+
+where each options means:
+
+``<ip_ver>``
+
+ * IP protocol version
+
+ * Optional: No
+
+ * Available options:
+
+   * *ipv4*: IP protocol version 4
+   * *ipv6*: IP protocol version 6
+
+``<dir>``
+
+ * The traffic direction
+
+ * Optional: No
+
+ * Available options:
+
+   * *in*: inbound traffic
+   * *out*: outbound traffic
+
+``<action>``
+
+ * IPsec action
+
+ * Optional: No
+
+ * Available options:
+
+   * *protect <SA_idx>*: the specified traffic is protected by SA rule
+     with id SA_idx
+   * *bypass*: the specified traffic traffic is bypassed
+   * *discard*: the specified traffic is discarded
+
+``<priority>``
+
+ * Rule priority
+
+ * Optional: Yes, default priority 0 will be used
+
+ * Syntax: *pri <id>*
+
+``<src_ip>``
+
+ * The source IP address and mask
+
+ * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
+
+ * Syntax:
+
+   * *src X.X.X.X/Y* for IPv4
+   * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
+
+``<dst_ip>``
+
+ * The destination IP address and mask
+
+ * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
+
+ * Syntax:
+
+   * *dst X.X.X.X/Y* for IPv4
+   * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
+
+``<proto>``
+
+ * The protocol start and end range
+
+ * Optional: yes, default range of 0 to 0 will be used
+
+ * Syntax: *proto X:Y*
+
+``<sport>``
+
+ * The source port start and end range
+
+ * Optional: yes, default range of 0 to 0 will be used
+
+ * Syntax: *sport X:Y*
+
+``<dport>``
+
+ * The destination port start and end range
+
+ * Optional: yes, default range of 0 to 0 will be used
+
+ * Syntax: *dport X:Y*
+
+Example SP rules:
+
+.. code-block:: console
+
+    sp ipv4 out esp protect 105 pri 1 dst 192.168.115.0/24 sport 0:65535 \
+    dport 0:65535
+
+    sp ipv6 in esp bypass pri 1 dst 0000:0000:0000:0000:5555:5555:\
+    0000:0000/96 sport 0:65535 dport 0:65535
+
+
+SA rule syntax
+^^^^^^^^^^^^^^
+
+The successfully parsed SA rules will be stored in an array table.
+
+The SA rule syntax is shown as follows:
+
+.. code-block:: console
+
+    sa <dir> <spi> <cipher_algo> <cipher_key> <auth_algo> <auth_key>
+    <mode> <src_ip> <dst_ip> <action_type> <port_id> <fallback>
+
+where each options means:
+
+``<dir>``
+
+ * The traffic direction
+
+ * Optional: No
+
+ * Available options:
+
+   * *in*: inbound traffic
+   * *out*: outbound traffic
+
+``<spi>``
+
+ * The SPI number
+
+ * Optional: No
+
+ * Syntax: unsigned integer number
+
+``<cipher_algo>``
+
+ * Cipher algorithm
+
+ * Optional: Yes, unless <aead_algo> is not used
+
+ * Available options:
+
+   * *null*: NULL algorithm
+   * *aes-128-cbc*: AES-CBC 128-bit algorithm
+   * *aes-256-cbc*: AES-CBC 256-bit algorithm
+   * *aes-128-ctr*: AES-CTR 128-bit algorithm
+   * *3des-cbc*: 3DES-CBC 192-bit algorithm
+
+ * Syntax: *cipher_algo <your algorithm>*
+
+``<cipher_key>``
+
+ * Cipher key, NOT available when 'null' algorithm is used
+
+ * Optional: Yes, unless <aead_algo> is not used.
+   Must be followed by <cipher_algo> option
+
+ * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
+   The number of bytes should be as same as the specified cipher algorithm
+   key size.
+
+   For example: *cipher_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
+   A1:B2:C3:D4*
+
+``<auth_algo>``
+
+ * Authentication algorithm
+
+ * Optional: Yes, unless <aead_algo> is not used
+
+ * Available options:
+
+    * *null*: NULL algorithm
+    * *sha1-hmac*: HMAC SHA1 algorithm
+
+``<auth_key>``
+
+ * Authentication key, NOT available when 'null' or 'aes-128-gcm' algorithm
+   is used.
+
+ * Optional: Yes, unless <aead_algo> is not used.
+   Must be followed by <auth_algo> option
+
+ * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
+   The number of bytes should be as same as the specified authentication
+   algorithm key size.
+
+   For example: *auth_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
+   A1:B2:C3:D4*
+
+``<aead_algo>``
+
+ * AEAD algorithm
+
+ * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used
+
+ * Available options:
+
+   * *aes-128-gcm*: AES-GCM 128-bit algorithm
+
+ * Syntax: *cipher_algo <your algorithm>*
+
+``<aead_key>``
+
+ * Cipher key, NOT available when 'null' algorithm is used
+
+ * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used.
+   Must be followed by <aead_algo> option
+
+ * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'.
+   The number of bytes should be as same as the specified AEAD algorithm
+   key size.
+
+   For example: *aead_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:
+   A1:B2:C3:D4*
+
+``<mode>``
+
+ * The operation mode
+
+ * Optional: No
+
+ * Available options:
+
+   * *ipv4-tunnel*: Tunnel mode for IPv4 packets
+   * *ipv6-tunnel*: Tunnel mode for IPv6 packets
+   * *transport*: transport mode
+
+ * Syntax: mode XXX
+
+``<src_ip>``
+
+ * The source IP address. This option is not available when
+   transport mode is used
+
+ * Optional: Yes, default address 0.0.0.0 will be used
+
+ * Syntax:
+
+   * *src X.X.X.X* for IPv4
+   * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6
+
+``<dst_ip>``
+
+ * The destination IP address. This option is not available when
+   transport mode is used
+
+ * Optional: Yes, default address 0.0.0.0 will be used
+
+ * Syntax:
+
+   * *dst X.X.X.X* for IPv4
+   * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6
+
+``<type>``
+
+ * Action type to specify the security action. This option specify
+   the SA to be performed with look aside protocol offload to HW
+   accelerator or protocol offload on ethernet device or inline
+   crypto processing on the ethernet device during transmission.
+
+ * Optional: Yes, default type *no-offload*
+
+ * Available options:
+
+   * *lookaside-protocol-offload*: look aside protocol offload to HW accelerator
+   * *inline-protocol-offload*: inline protocol offload on ethernet device
+   * *inline-crypto-offload*: inline crypto processing on ethernet device
+   * *no-offload*: no offloading to hardware
+
+ ``<port_id>``
+
+ * Port/device ID of the ethernet/crypto accelerator for which the SA is
+   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
+   port will be used for routing. The routing table will not be referred in
+   this case.
+
+ * Optional: No, if *type* is not *no-offload*
+
+ * Syntax:
+
+   * *port_id X* X is a valid device number in decimal
+
+ ``<fallback>``
+
+ * Action type for ingress IPsec packets that inline processor failed to
+   process. Only a combination of *inline-crypto-offload* as a primary
+   session and *lookaside-none* as a fall-back session is supported at the
+   moment.
+
+   If used in conjunction with IPsec window, its width needs be increased
+   due to different processing times of inline and lookaside modes which
+   results in packet reordering.
+
+ * Optional: Yes.
+
+ * Available options:
+
+   * *lookaside-none*: use automatically chosen cryptodev to process packets
+
+ * Syntax:
+
+   * *fallback lookaside-none*
+
+Example SA rules:
+
+.. code-block:: console
+
+    sa out 5 cipher_algo null auth_algo null mode ipv4-tunnel \
+    src 172.16.1.5 dst 172.16.2.5
+
+    sa out 25 cipher_algo aes-128-cbc \
+    cipher_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \
+    auth_algo sha1-hmac \
+    auth_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \
+    mode ipv6-tunnel \
+    src 1111:1111:1111:1111:1111:1111:1111:5555 \
+    dst 2222:2222:2222:2222:2222:2222:2222:5555
+
+    sa in 105 aead_algo aes-128-gcm \
+    aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
+    mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5
+
+    sa out 5 cipher_algo aes-128-cbc cipher_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \
+    auth_algo sha1-hmac auth_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \
+    mode ipv4-tunnel src 172.16.1.5 dst 172.16.2.5 \
+    type lookaside-protocol-offload port_id 4
+
+    sa in 35 aead_algo aes-128-gcm \
+    aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
+    mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 \
+    type inline-crypto-offload port_id 0
+
+Routing rule syntax
+^^^^^^^^^^^^^^^^^^^
+
+The Routing rule syntax is shown as follows:
+
+.. code-block:: console
+
+    rt <ip_ver> <src_ip> <dst_ip> <port>
+
+
+where each options means:
+
+``<ip_ver>``
+
+ * IP protocol version
+
+ * Optional: No
+
+ * Available options:
+
+   * *ipv4*: IP protocol version 4
+   * *ipv6*: IP protocol version 6
+
+``<src_ip>``
+
+ * The source IP address and mask
+
+ * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
+
+ * Syntax:
+
+   * *src X.X.X.X/Y* for IPv4
+   * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
+
+``<dst_ip>``
+
+ * The destination IP address and mask
+
+ * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used
+
+ * Syntax:
+
+   * *dst X.X.X.X/Y* for IPv4
+   * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6
+
+``<port>``
+
+ * The traffic output port id
+
+ * Optional: yes, default output port 0 will be used
+
+ * Syntax: *port X*
+
+Example SP rules:
+
+.. code-block:: console
+
+    rt ipv4 dst 172.16.1.5/32 port 0
+
+    rt ipv6 dst 1111:1111:1111:1111:1111:1111:1111:5555/116 port 0
+
+Neighbour rule syntax
+^^^^^^^^^^^^^^^^^^^^^
+
+The Neighbour rule syntax is shown as follows:
+
+.. code-block:: console
+
+    neigh <port> <dst_mac>
+
+
+where each options means:
+
+``<port>``
+
+ * The output port id
+
+ * Optional: No
+
+ * Syntax: *port X*
+
+``<dst_mac>``
+
+ * The destination ethernet address to use for that port
+
+ * Optional: No
+
+ * Syntax:
+
+   * XX:XX:XX:XX:XX:XX
+
+Example Neighbour rules:
+
+.. code-block:: console
+
+    neigh port 0 DE:AD:BE:EF:01:02
+
+Test directory
+--------------
+
+The test directory contains scripts for testing the various encryption
+algorithms.
+
+The purpose of the scripts is to automate ipsec-secgw testing
+using another system running linux as a DUT.
+
+The user must setup the following environment variables:
+
+*   ``SGW_PATH``: path to the ipsec-secgw binary to test.
+
+*   ``REMOTE_HOST``: IP address/hostname of the DUT.
+
+*   ``REMOTE_IFACE``: interface name for the test-port on the DUT.
+
+*   ``ETH_DEV``: ethernet device to be used on the SUT by DPDK ('-w <pci-id>')
+
+Also the user can optionally setup:
+
+*   ``SGW_LCORE``: lcore to run ipsec-secgw on (default value is 0)
+
+*   ``CRYPTO_DEV``: crypto device to be used ('-w <pci-id>'). If none specified
+    appropriate vdevs will be created by the script
+
+*   ``MULTI_SEG_TEST``: ipsec-secgw option to enable reassembly support and
+    specify size of reassembly table (e.g.
+    ``MULTI_SEG_TEST='--reassemble 128'``). This option must be set for
+    fallback session tests.
+
+Note that most of the tests require the appropriate crypto PMD/device to be
+available.
+
+Server configuration
+~~~~~~~~~~~~~~~~~~~~
+
+Two servers are required for the tests, SUT and DUT.
+
+Make sure the user from the SUT can ssh to the DUT without entering the password.
+To enable this feature keys must be setup on the DUT.
+
+``ssh-keygen`` will make a private & public key pair on the SUT.
+
+``ssh-copy-id`` <user name>@<target host name> on the SUT will copy the public
+key to the DUT. It will ask for credentials so that it can upload the public key.
+
+The SUT and DUT are connected through at least 2 NIC ports.
+
+One NIC port is expected to be managed by linux on both machines and will be
+used as a control path.
+
+The second NIC port (test-port) should be bound to DPDK on the SUT, and should
+be managed by linux on the DUT.
+
+The script starts ``ipsec-secgw`` with 2 NIC devices: ``test-port`` and
+``tap vdev``.
+
+It then configures the local tap interface and the remote interface and IPsec
+policies in the following way:
+
+Traffic going over the test-port in both directions has to be protected by IPsec.
+
+Traffic going over the TAP port in both directions does not have to be protected.
+
+i.e:
+
+DUT OS(NIC1)--(IPsec)-->(NIC1)ipsec-secgw(TAP)--(plain)-->(TAP)SUT OS
+
+SUT OS(TAP)--(plain)-->(TAP)psec-secgw(NIC1)--(IPsec)-->(NIC1)DUT OS
+
+It then tries to perform some data transfer using the scheme described above.
+
+usage
+~~~~~
+
+In the ipsec-secgw/test directory
+
+to run one test for IPv4 or IPv6
+
+/bin/bash linux_test(4|6).sh <ipsec_mode>
+
+to run all tests for IPv4 or IPv6
+
+/bin/bash run_test.sh -4|-6
+
+For the list of available modes please refer to run_test.sh.