1 .. SPDX-License-Identifier: BSD-3-Clause
2 Copyright(c) 2017 Intel Corporation.
4 Generic Segmentation Offload Library
5 ====================================
9 Generic Segmentation Offload (GSO) is a widely used software implementation of
10 TCP Segmentation Offload (TSO), which reduces per-packet processing overhead.
11 Much like TSO, GSO gains performance by enabling upper layer applications to
12 process a smaller number of large packets (e.g. MTU size of 64KB), instead of
13 processing higher numbers of small packets (e.g. MTU size of 1500B), thus
14 reducing per-packet overhead.
16 For example, GSO allows guest kernel stacks to transmit over-sized TCP segments
17 that far exceed the kernel interface's MTU; this eliminates the need to segment
18 packets within the guest, and improves the data-to-overhead ratio of both the
19 guest-host link, and PCI bus. The expectation of the guest network stack in this
20 scenario is that segmentation of egress frames will take place either in the NIC
21 HW, or where that hardware capability is unavailable, either in the host
22 application, or network stack.
24 Bearing that in mind, the GSO library enables DPDK applications to segment
25 packets in software. Note however, that GSO is implemented as a standalone
26 library, and not via a 'fallback' mechanism (i.e. for when TSO is unsupported
27 in the underlying hardware); that is, applications must explicitly invoke the
28 GSO library to segment packets, they also must call ``rte_pktmbuf_free()``
29 to free mbuf GSO segments attached after calling ``rte_gso_segment()``.
30 The size of GSO segments (``segsz``) is configurable by the application.
35 #. The GSO library doesn't check if input packets have correct checksums.
37 #. In addition, the GSO library doesn't re-calculate checksums for segmented
38 packets (that task is left to the application).
40 #. IP fragments are unsupported by the GSO library.
42 #. The egress interface's driver must support multi-segment packets.
44 #. Currently, the GSO library supports the following IPv4 packet types:
51 See `Supported GSO Packet Types`_ for further details.
56 The ``rte_gso_segment()`` function is the GSO library's primary
59 Before performing segmentation, an application must create a GSO context object
60 ``(struct rte_gso_ctx)``, which provides the library with some of the
61 information required to understand how the packet should be segmented. Refer to
62 `How to Segment a Packet`_ for additional details on same. Once the GSO context
63 has been created, and populated, the application can then use the
64 ``rte_gso_segment()`` function to segment packets.
66 The GSO library typically stores each segment that it creates in two parts: the
67 first part contains a copy of the original packet's headers, while the second
68 part contains a pointer to an offset within the original packet. This mechanism
69 is explained in more detail in `GSO Output Segment Format`_.
71 The GSO library supports both single- and multi-segment input mbufs.
73 GSO Output Segment Format
74 ~~~~~~~~~~~~~~~~~~~~~~~~~
75 To reduce the number of expensive memcpy operations required when segmenting a
76 packet, the GSO library typically stores each segment that it creates as a
77 two-part mbuf (technically, this is termed a 'two-segment' mbuf; however, since
78 the elements produced by the API are also called 'segments', for clarity the
79 term 'part' is used here instead).
81 The first part of each output segment is a direct mbuf and contains a copy of
82 the original packet's headers, which must be prepended to each output segment.
83 These headers are copied from the original packet into each output segment.
85 The second part of each output segment, represents a section of data from the
86 original packet, i.e. a data segment. Rather than copy the data directly from
87 the original packet into the output segment (which would impact performance
88 considerably), the second part of each output segment is an indirect mbuf,
89 which contains no actual data, but simply points to an offset within the
92 The combination of the 'header' segment and the 'data' segment constitutes a
93 single logical output GSO segment of the original packet. This is illustrated
94 in :numref:`figure_gso-output-segment-format`.
96 .. _figure_gso-output-segment-format:
98 .. figure:: img/gso-output-segment-format.*
101 Two-part GSO output segment
103 In one situation, the output segment may contain additional 'data' segments.
104 This only occurs when:
106 - the input packet on which GSO is to be performed is represented by a
109 - the output segment is required to contain data that spans the boundaries
110 between segments of the input multi-segment mbuf.
112 The GSO library traverses each segment of the input packet, and produces
113 numerous output segments; for optimal performance, the number of output
114 segments is kept to a minimum. Consequently, the GSO library maximizes the
115 amount of data contained within each output segment; i.e. each output segment
116 ``segsz`` bytes of data. The only exception to this is in the case of the very
117 final output segment; if ``pkt_len`` % ``segsz``, then the final segment is
118 smaller than the rest.
120 In order for an output segment to meet its MSS, it may need to include data from
121 multiple input segments. Due to the nature of indirect mbufs (each indirect mbuf
122 can point to only one direct mbuf), the solution here is to add another indirect
123 mbuf to the output segment; this additional segment then points to the next
124 input segment. If necessary, this chaining process is repeated, until the sum of
125 all of the data 'contained' in the output segment reaches ``segsz``. This
126 ensures that the amount of data contained within each output segment is uniform,
127 with the possible exception of the last segment, as previously described.
129 :numref:`figure_gso-three-seg-mbuf` illustrates an example of a three-part
130 output segment. In this example, the output segment needs to include data from
131 the end of one input segment, and the beginning of another. To achieve this,
132 an additional indirect mbuf is chained to the second part of the output segment,
133 and is attached to the next input segment (i.e. it points to the data in the
136 .. _figure_gso-three-seg-mbuf:
138 .. figure:: img/gso-three-seg-mbuf.*
141 Three-part GSO output segment
143 Supported GSO Packet Types
144 --------------------------
148 TCP/IPv4 GSO supports segmentation of suitably large TCP/IPv4 packets, which
149 may also contain an optional VLAN tag.
153 UDP/IPv4 GSO supports segmentation of suitably large UDP/IPv4 packets, which
154 may also contain an optional VLAN tag. UDP GSO is the same as IP fragmentation.
155 Specifically, UDP GSO treats the UDP header as a part of the payload and
156 does not modify it during segmentation. Therefore, after UDP GSO, only the
157 first output packet has the original UDP header, and others just have l2
162 VXLAN packets GSO supports segmentation of suitably large VXLAN packets,
163 which contain an outer IPv4 header, inner TCP/IPv4 or UDP/IPv4 headers, and
164 optional inner and/or outer VLAN tag(s).
168 GRE GSO supports segmentation of suitably large GRE packets, which contain
169 an outer IPv4 header, inner TCP/IPv4 headers, and an optional VLAN tag.
171 How to Segment a Packet
172 -----------------------
174 To segment an outgoing packet, an application must:
176 #. First create a GSO context ``(struct rte_gso_ctx)``; this contains:
178 - a pointer to the mbuf pool for allocating the direct buffers, which are
179 used to store the GSO segments' packet headers.
181 - a pointer to the mbuf pool for allocating indirect buffers, which are
182 used to locate GSO segments' packet payloads.
186 An application may use the same pool for both direct and indirect
187 buffers. However, since indirect mbufs simply store a pointer, the
188 application may reduce its memory consumption by creating a separate memory
189 pool, containing smaller elements, for the indirect pool.
192 - the size of each output segment, including packet headers and payload,
195 - the bit mask of required GSO types. The GSO library uses the same macros as
196 those that describe a physical device's TX offloading capabilities (i.e.
197 ``DEV_TX_OFFLOAD_*_TSO``) for gso_types. For example, if an application
198 wants to segment TCP/IPv4 packets, it should set gso_types to
199 ``DEV_TX_OFFLOAD_TCP_TSO``. The only other supported values currently
200 supported for gso_types are ``DEV_TX_OFFLOAD_VXLAN_TNL_TSO``, and
201 ``DEV_TX_OFFLOAD_GRE_TNL_TSO``; a combination of these macros is also
204 - a flag, that indicates whether the IPv4 headers of output segments should
205 contain fixed or incremental ID values.
207 2. Set the appropriate ol_flags in the mbuf.
209 - The GSO library use the value of an mbuf's ``ol_flags`` attribute to
210 determine how a packet should be segmented. It is the application's
211 responsibility to ensure that these flags are set.
213 - For example, in order to segment TCP/IPv4 packets, the application should
214 add the ``PKT_TX_IPV4`` and ``PKT_TX_TCP_SEG`` flags to the mbuf's
217 - If checksum calculation in hardware is required, the application should
218 also add the ``PKT_TX_TCP_CKSUM`` and ``PKT_TX_IP_CKSUM`` flags.
220 #. Check if the packet should be processed. Packets with one of the
221 following properties are not processed and are returned immediately:
223 - Packet length is less than ``segsz`` (i.e. GSO is not required).
225 - Packet type is not supported by GSO library (see
226 `Supported GSO Packet Types`_).
228 - Application has not enabled GSO support for the packet type.
230 - Packet's ol_flags have been incorrectly set.
232 #. Allocate space in which to store the output GSO segments. If the amount of
233 space allocated by the application is insufficient, segmentation will fail.
235 #. Invoke the GSO segmentation API, ``rte_gso_segment()``.
237 #. Call ``rte_pktmbuf_free()`` to free mbuf ``rte_gso_segment()`` segments.
239 #. If required, update the L3 and L4 checksums of the newly-created segments.
240 For tunneled packets, the outer IPv4 headers' checksums should also be
241 updated. Alternatively, the application may offload checksum calculation