From 9da82e8d8ba5783a155c4a73499c6d26550b1e1d Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Fri, 10 Jul 2020 12:39:41 +0000 Subject: [PATCH] mbuf: introduce accurate packet Tx scheduling There is the requirement on some networks for precise traffic timing management. The ability to send (and, generally speaking, receive) the packets at the very precisely specified moment of time provides the opportunity to support the connections with Time Division Multiplexing using the contemporary general purpose NIC without involving an auxiliary hardware. For example, the supporting of O-RAN Fronthaul interface is one of the promising features for potentially usage of the precise time management for the egress packets. The main objective of this patchset is to specify the way how applications can provide the moment of time at what the packet transmission must be started and to describe in preliminary the supporting this feature from mlx5 PMD side [1]. The new dynamic timestamp field is proposed, it provides some timing information, the units and time references (initial phase) are not explicitly defined but are maintained always the same for a given port. Some devices allow to query rte_eth_read_clock() that will return the current device timestamp. The dynamic timestamp flag tells whether the field contains actual timestamp value. For the packets being sent this value can be used by PMD to schedule packet sending. The device clock is opaque entity, the units and frequency are vendor specific and might depend on hardware capabilities and configurations. If might (or not) be synchronized with real time via PTP, might (or not) be synchronous with CPU clock (for example if NIC and CPU share the same clock source there might be no any drift between the NIC and CPU clocks), etc. After PKT_RX_TIMESTAMP flag and fixed timestamp field supposed deprecation and obsoleting, these dynamic flag and field might be used to manage the timestamps on receiving datapath as well. Having the dedicated flags for Rx/Tx timestamps allows applications not to perform explicit flags reset on forwarding and not to promote received timestamps to the transmitting datapath by default. The static PKT_RX_TIMESTAMP is considered as candidate to become the dynamic flag and this move should be discussed. When PMD sees the "rte_dynfield_timestamp" set on the packet being sent it tries to synchronize the time of packet appearing on the wire with the specified packet timestamp. If the specified one is in the past it should be ignored, if one is in the distant future it should be capped with some reasonable value (in range of seconds). These specific cases ("too late" and "distant future") can be optionally reported via device xstats to assist applications to detect the time-related problems. There is no any packet reordering according timestamps is supposed, neither within packet burst, nor between packets, it is an entirely application responsibility to generate packets and its timestamps in desired order. The timestamps can be put only in the first packet in the burst providing the entire burst scheduling. PMD reports the ability to synchronize packet sending on timestamp with new offload flag: This is palliative and might be replaced with new eth_dev API about reporting/managing the supported dynamic flags and its related features. This API would break ABI compatibility and can't be introduced at the moment, so is postponed to 20.11. For testing purposes it is proposed to update testpmd "txonly" forwarding mode routine. With this update testpmd application generates the packets and sets the dynamic timestamps according to specified time pattern if it sees the "rte_dynfield_timestamp" is registered. The new testpmd command is proposed to configure sending pattern: set tx_times , - the delay between the packets within the burst specified in the device clock units. The number of packets in the burst is defined by txburst parameter - the delay between the bursts in the device clock units As the result the bursts of packet will be transmitted with specific delays between the packets within the burst and specific delay between the bursts. The rte_eth_read_clock is supposed to be engaged to get the current device clock value and provide the reference for the timestamps. [1] http://patches.dpdk.org/patch/73714/ Signed-off-by: Viacheslav Ovsiienko Acked-by: Olivier Matz Reviewed-by: Ferruh Yigit --- doc/guides/rel_notes/release_20_08.rst | 7 ++++++ lib/librte_ethdev/rte_ethdev.c | 1 + lib/librte_ethdev/rte_ethdev.h | 4 ++++ lib/librte_mbuf/rte_mbuf_dyn.h | 31 ++++++++++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst index d72344a180..2ca12fa5ca 100644 --- a/doc/guides/rel_notes/release_20_08.rst +++ b/doc/guides/rel_notes/release_20_08.rst @@ -81,6 +81,13 @@ New Features Added the RegEx library which provides an API for offload of regular expressions search operations to hardware or software accelerator devices. + +* **Introduced send packet scheduling on the timestamps.** + + Added the new mbuf dynamic field and flag to provide timestamp on what packet + transmitting can be synchronized. The device Tx offload flag is added to + indicate the PMD supports send scheduling. + * **Updated PCAP driver.** Updated PCAP driver with new features and improvements, including: diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 5ff52046dc..598ff80a6e 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -160,6 +160,7 @@ static const struct { RTE_TX_OFFLOAD_BIT2STR(UDP_TNL_TSO), RTE_TX_OFFLOAD_BIT2STR(IP_TNL_TSO), RTE_TX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM), + RTE_TX_OFFLOAD_BIT2STR(SEND_ON_TIMESTAMP), }; #undef RTE_TX_OFFLOAD_BIT2STR diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 03356cdb24..57e4a6ca58 100644 --- a/lib/librte_ethdev/rte_ethdev.h +++ b/lib/librte_ethdev/rte_ethdev.h @@ -1287,6 +1287,10 @@ struct rte_eth_conf { /** Device supports outer UDP checksum */ #define DEV_TX_OFFLOAD_OUTER_UDP_CKSUM 0x00100000 +/** Device supports send on timestamp */ +#define DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP 0x00200000 + + #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001 /**< Device supports Rx queue setup after device started*/ #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002 diff --git a/lib/librte_mbuf/rte_mbuf_dyn.h b/lib/librte_mbuf/rte_mbuf_dyn.h index 96c363137e..8407230ecf 100644 --- a/lib/librte_mbuf/rte_mbuf_dyn.h +++ b/lib/librte_mbuf/rte_mbuf_dyn.h @@ -250,4 +250,35 @@ void rte_mbuf_dyn_dump(FILE *out); #define RTE_MBUF_DYNFIELD_METADATA_NAME "rte_flow_dynfield_metadata" #define RTE_MBUF_DYNFLAG_METADATA_NAME "rte_flow_dynflag_metadata" +/** + * The timestamp dynamic field provides some timing information, the + * units and time references (initial phase) are not explicitly defined + * but are maintained always the same for a given port. Some devices allow + * to query rte_eth_read_clock() that will return the current device + * timestamp. The dynamic Tx timestamp flag tells whether the field contains + * actual timestamp value for the packets being sent, this value can be + * used by PMD to schedule packet sending. + * + * After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation + * and obsoleting, the dedicated Rx timestamp flag is supposed to be + * introduced and the shared dynamic timestamp field will be used + * to handle the timestamps on receiving datapath as well. + */ +#define RTE_MBUF_DYNFIELD_TIMESTAMP_NAME "rte_dynfield_timestamp" + +/** + * When PMD sees the RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME flag set on the + * packet being sent it tries to synchronize the time of packet appearing + * on the wire with the specified packet timestamp. If the specified one + * is in the past it should be ignored, if one is in the distant future + * it should be capped with some reasonable value (in range of seconds). + * + * There is no any packet reordering according to timestamps is supposed, + * neither for packet within the burst, nor for the whole bursts, it is + * an entirely application responsibility to generate packets and its + * timestamps in desired order. The timestamps might be put only in + * the first packet in the burst providing the entire burst scheduling. + */ +#define RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME "rte_dynflag_tx_timestamp" + #endif -- 2.20.1