For more details, e.g. how to convert an mbuf to an SGL, please refer to an
example usage in the IPsec library implementation.
+Cryptodev Raw Data-path APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Crypto Raw data-path APIs are a set of APIs designed to enable external
+libraries/applications to leverage the cryptographic processing provided by
+DPDK crypto PMDs through the cryptodev API but in a manner that is not
+dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc)
+in their data-path implementation.
+
+The raw data-path APIs have the following advantages:
+
+- External data structure friendly design. The new APIs uses the operation
+ descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
+ IOVA addresses as input. Moreover, the APIs does not require the user to
+ allocate the descriptor from mempool, nor requiring mbufs to describe input
+ data's virtual and IOVA addresses. All these features made the translation
+ from user's own data structure into the descriptor easier and more efficient.
+
+- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
+ user more control to the enqueue and dequeue operations, including the
+ capability of precious enqueue/dequeue count, abandoning enqueue or dequeue
+ at any time, and operation status translation and set on the fly.
+
+Cryptodev PMDs which support the raw data-path APIs will have
+``RTE_CRYPTODEV_FF_SYM_RAW_DP`` feature flag presented. To use this feature,
+the user shall create a local ``struct rte_crypto_raw_dp_ctx`` buffer and
+extend to at least the length returned by ``rte_cryptodev_get_raw_dp_ctx_size``
+function call. The created buffer is then initialized using
+``rte_cryptodev_configure_raw_dp_ctx`` function with the ``is_update``
+parameter as 0. The library and the crypto device driver will then set the
+buffer and attach either the cryptodev sym session, the rte_security session,
+or the cryptodev xform for session-less operation into the ctx buffer, and
+set the corresponding enqueue and dequeue function handlers based on the
+algorithm information stored in the session or xform. When the ``is_update``
+parameter passed into ``rte_cryptodev_configure_raw_dp_ctx`` is 1, the driver
+will not initialize the buffer but only update the session or xform and
+the function handlers accordingly.
+
+After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is now
+ready for enqueue and dequeue operation. There are two different enqueue
+functions: ``rte_cryptodev_raw_enqueue`` to enqueue single raw data
+operation, and ``rte_cryptodev_raw_enqueue_burst`` to enqueue a descriptor
+with multiple operations. In case of the application uses similar approach to
+``struct rte_crypto_sym_vec`` to manage its data burst but with different
+data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may be
+less efficient as this is a situation where the application has to loop over
+all crypto operations to assemble the ``struct rte_crypto_sym_vec`` descriptor
+from its own data structure, and then the driver will loop over them again to
+translate every operation in the descriptor to the driver's specific queue data.
+The ``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data
+burst instead.
+
+The ``rte_cryptodev_raw_enqueue`` and ``rte_cryptodev_raw_enqueue_burst``
+functions will return or set the enqueue status. ``rte_cryptodev_raw_enqueue``
+will return the status directly, ``rte_cryptodev_raw_enqueue_burst`` will
+return the number of operations enqueued or stored (explained as follows) and
+set the ``enqueue_status`` buffer provided by the user. The possible
+enqueue status values are:
+
+- ``1``: the operation(s) is/are enqueued successfully.
+- ``0``: the operation(s) is/are cached successfully in the crypto device queue
+ but is not actually enqueued. The user shall call
+ ``rte_cryptodev_raw_enqueue_done`` function after the expected operations
+ are stored. The crypto device will then start enqueuing all of them at
+ once.
+- The negative integer: error occurred during enqueue.
+
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter ``is_update``
+set as 0 twice without the enqueue function returning or setting enqueue status
+to 1 or ``rte_cryptodev_raw_enqueue_done`` function being called in between will
+invalidate any operation stored in the device queue but not enqueued. This
+feature is useful when the user wants to abandon partially enqueued operations
+for a failed enqueue burst operation and try enqueuing in a whole later.
+
+Similar as enqueue, there are two dequeue functions:
+``rte_cryptodev_raw_dequeue`` for dequeing single operation, and
+``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of operations (e.g.
+all operations in a ``struct rte_crypto_sym_vec`` descriptor). The
+``rte_cryptodev_raw_dequeue_burst`` function allows the user to provide callback
+functions to retrieve dequeue count from the enqueued user data and write the
+expected status value to the user data on the fly. The dequeue functions also
+set the dequeue status:
+
+- ``1``: the operation(s) is/are dequeued successfully.
+- ``0``: the operation(s) is/are completed but is not actually dequeued (hence
+ still kept in the device queue). The user shall call the
+ ``rte_cryptodev_raw_dequeue_done`` function after the expected number of
+ operations (e.g. all operations in a descriptor) are dequeued. The crypto
+ device driver will then free them from the queue at once.
+- The negative integer: error occurred during dequeue.
+
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter ``is_update``
+set as 0 twice without the dequeue functions execution changed dequeue_status
+to 1 or ``rte_cryptodev_raw_dequeue_done`` function being called in between will
+revert the crypto device queue's dequeue effort to the moment when the
+``struct rte_crypto_raw_dp_ctx`` buffer is initialized. This feature is useful
+when the user wants to abandon partially dequeued data and try dequeuing again
+later in a whole.
+
+There are a few limitations to the raw data path APIs:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the raw data-path API's enqueue with rte_cryptodev_enqueue_burst,
+ or vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
Sample code
-----------
/**< Support symmetric session-less operations */
#define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA (1ULL << 23)
/**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_SYM_RAW_DP (1ULL << 24)
+/**< Support accelerator specific symmetric raw data-path APIs */
/**
* Get the name of a crypto device feature flag
struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
struct rte_crypto_sym_vec *vec);
+/**
+ * Get the size of the raw data-path context buffer.
+ *
+ * @param dev_id The device identifier.
+ *
+ * @return
+ * - If the device supports raw data-path APIs, return the context size.
+ * - If the device does not support the APIs, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_get_raw_dp_ctx_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.
+ */
+union rte_cryptodev_session_ctx {
+ struct rte_cryptodev_sym_session *crypto_sess;
+ struct rte_crypto_sym_xform *xform;
+ struct rte_security_session *sec_sess;
+};
+
+/**
+ * Enqueue a vectorized operation descriptor into the device queue but the
+ * driver may or may not start processing until rte_cryptodev_raw_enqueue_done()
+ * is called.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param drv_ctx Driver specific context data.
+ * @param vec Vectorized operation descriptor.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param user_data The array of user data for dequeue later.
+ * @param enqueue_status Driver written value to specify the
+ * enqueue status. Possible values:
+ * - 1: The number of operations returned are
+ * enqueued successfully.
+ * - 0: The number of operations returned are
+ * cached into the queue but are not processed
+ * until rte_cryptodev_raw_enqueue_done() is
+ * called.
+ * - negative integer: Error occurred.
+ * @return
+ * - The number of operations in the descriptor successfully enqueued or
+ * cached into the queue but not enqueued yet, depends on the
+ * "enqueue_status" value.
+ */
+typedef uint32_t (*cryptodev_sym_raw_enqueue_burst_t)(
+ void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec,
+ union rte_crypto_sym_ofs ofs, void *user_data[], int *enqueue_status);
+
+/**
+ * Enqueue single raw data vector into the device queue but the driver may or
+ * may not start processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param drv_ctx Driver specific context data.
+ * @param data_vec The buffer data vector.
+ * @param n_data_vecs Number of buffer data vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param iv IV virtual and IOVA addresses
+ * @param digest digest virtual and IOVA addresses
+ * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses,
+ * depends on the algorithm used.
+ * @param user_data The user data.
+ * @return
+ * - 1: The data vector is enqueued successfully.
+ * - 0: The data vector is cached into the queue but is not processed
+ * until rte_cryptodev_raw_enqueue_done() is called.
+ * - negative integer: failure.
+ */
+typedef int (*cryptodev_sym_raw_enqueue_t)(
+ void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec,
+ uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+ struct rte_crypto_va_iova_ptr *iv,
+ struct rte_crypto_va_iova_ptr *digest,
+ struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+ void *user_data);
+
+/**
+ * Inform the cryptodev queue pair to start processing or finish dequeuing all
+ * enqueued/dequeued operations.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param drv_ctx Driver specific context data.
+ * @param n The total number of processed operations.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+typedef int (*cryptodev_sym_raw_operation_done_t)(void *qp, uint8_t *drv_ctx,
+ uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from the user
+ * data stored in the first processed operation.
+ *
+ * @param user_data Dequeued user data.
+ * @return
+ * - The number of operations to be dequeued.
+ **/
+typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void *user_data);
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation, such
+ * as filling status.
+ *
+ * @param user_data Dequeued user data.
+ * @param index Index number of the processed descriptor.
+ * @param is_op_success Operation status provided by the driver.
+ **/
+typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
+ uint32_t index, uint8_t is_op_success);
+
+/**
+ * Dequeue a burst of symmetric crypto processing.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param drv_ctx Driver specific context data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue operation count.
+ * @param post_dequeue User provided callback function to
+ * post-process a dequeued operation.
+ * @param out_user_data User data pointer array to be retrieve
+ * from device queue. In case of
+ * *is_user_data_array* is set there
+ * should be enough room to store all
+ * user data.
+ * @param is_user_data_array Set 1 if every dequeued user data will
+ * be written into out_user_data array.
+ * Set 0 if only the first user data will
+ * be written into out_user_data array.
+ * @param n_success Driver written value to specific the
+ * total successful operations count.
+ * @param dequeue_status Driver written value to specify the
+ * dequeue status. Possible values:
+ * - 1: Successfully dequeued the number
+ * of operations returned. The user
+ * data previously set during enqueue
+ * is stored in the "out_user_data".
+ * - 0: The number of operations returned
+ * are completed and the user data is
+ * stored in the "out_user_data", but
+ * they are not freed from the queue
+ * until
+ * rte_cryptodev_raw_dequeue_done()
+ * is called.
+ * - negative integer: Error occurred.
+ * @return
+ * - The number of operations dequeued or completed but not freed from the
+ * queue, depends on "dequeue_status" value.
+ */
+typedef uint32_t (*cryptodev_sym_raw_dequeue_burst_t)(void *qp,
+ uint8_t *drv_ctx,
+ rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+ rte_cryptodev_raw_post_dequeue_t post_dequeue,
+ void **out_user_data, uint8_t is_user_data_array,
+ uint32_t *n_success, int *dequeue_status);
+
+/**
+ * Dequeue a symmetric crypto processing.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param drv_ctx Driver specific context data.
+ * @param dequeue_status Driver written value to specify the
+ * dequeue status. Possible values:
+ * - 1: Successfully dequeued a operation.
+ * The user data is returned.
+ * - 0: The first operation in the queue
+ * is completed and the user data
+ * previously set during enqueue is
+ * returned, but it is not freed from
+ * the queue until
+ * rte_cryptodev_raw_dequeue_done() is
+ * called.
+ * - negative integer: Error occurred.
+ * @param op_status Driver written value to specify
+ * operation status.
+ * @return
+ * - The user data pointer retrieved from device queue or NULL if no
+ * operation is ready for dequeue.
+ */
+typedef void * (*cryptodev_sym_raw_dequeue_t)(
+ void *qp, uint8_t *drv_ctx, int *dequeue_status,
+ enum rte_crypto_op_status *op_status);
+
+/**
+ * Context data for raw data-path API crypto process. The buffer of this
+ * structure is to be allocated by the user application with the size equal
+ * or bigger than rte_cryptodev_get_raw_dp_ctx_size() returned value.
+ */
+struct rte_crypto_raw_dp_ctx {
+ void *qp_data;
+
+ cryptodev_sym_raw_enqueue_t enqueue;
+ cryptodev_sym_raw_enqueue_burst_t enqueue_burst;
+ cryptodev_sym_raw_operation_done_t enqueue_done;
+ cryptodev_sym_raw_dequeue_t dequeue;
+ cryptodev_sym_raw_dequeue_burst_t dequeue_burst;
+ cryptodev_sym_raw_operation_done_t dequeue_done;
+
+ /* Driver specific context data */
+ __extension__ uint8_t drv_ctx_data[];
+};
+
+/**
+ * Configure raw data-path context data.
+ *
+ * NOTE:
+ * After the context data is configured, the user should call
+ * rte_cryptodev_raw_attach_session() before using it in
+ * rte_cryptodev_raw_enqueue/dequeue function call.
+ *
+ * @param dev_id The device identifier.
+ * @param qp_id The index of the queue pair from which to
+ * retrieve processed packets. The value must be
+ * in the range [0, nb_queue_pair - 1] previously
+ * supplied to rte_cryptodev_configure().
+ * @param ctx The raw data-path context data.
+ * @param sess_type session type.
+ * @param session_ctx Session context data.
+ * @param is_update Set 0 if it is to initialize the ctx.
+ * Set 1 if ctx is initialized and only to update
+ * session context data.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_configure_raw_dp_ctx(uint8_t dev_id, uint16_t qp_id,
+ struct rte_crypto_raw_dp_ctx *ctx,
+ enum rte_crypto_op_sess_type sess_type,
+ union rte_cryptodev_session_ctx session_ctx,
+ uint8_t is_update);
+
+/**
+ * Enqueue a vectorized operation descriptor into the device queue but the
+ * driver may or may not start processing until rte_cryptodev_raw_enqueue_done()
+ * is called.
+ *
+ * @param ctx The initialized raw data-path context data.
+ * @param vec Vectorized operation descriptor.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param user_data The array of user data for dequeue later.
+ * @param enqueue_status Driver written value to specify the
+ * enqueue status. Possible values:
+ * - 1: The number of operations returned are
+ * enqueued successfully.
+ * - 0: The number of operations returned are
+ * cached into the queue but are not processed
+ * until rte_cryptodev_raw_enqueue_done() is
+ * called.
+ * - negative integer: Error occurred.
+ * @return
+ * - The number of operations in the descriptor successfully enqueued or
+ * cached into the queue but not enqueued yet, depends on the
+ * "enqueue_status" value.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+ struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+ void **user_data, int *enqueue_status);
+
+/**
+ * Enqueue single raw data vector into the device queue but the driver may or
+ * may not start processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param ctx The initialized raw data-path context data.
+ * @param data_vec The buffer data vector.
+ * @param n_data_vecs Number of buffer data vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param iv IV virtual and IOVA addresses
+ * @param digest digest virtual and IOVA addresses
+ * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses,
+ * depends on the algorithm used.
+ * @param user_data The user data.
+ * @return
+ * - 1: The data vector is enqueued successfully.
+ * - 0: The data vector is cached into the queue but is not processed
+ * until rte_cryptodev_raw_enqueue_done() is called.
+ * - negative integer: failure.
+ */
+__rte_experimental
+static __rte_always_inline int
+rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx,
+ struct rte_crypto_vec *data_vec, uint16_t n_data_vecs,
+ union rte_crypto_sym_ofs ofs,
+ struct rte_crypto_va_iova_ptr *iv,
+ struct rte_crypto_va_iova_ptr *digest,
+ struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+ void *user_data)
+{
+ return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec,
+ n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data);
+}
+
+/**
+ * Start processing all enqueued operations from last
+ * rte_cryptodev_configure_raw_dp_ctx() call.
+ *
+ * @param ctx The initialized raw data-path context data.
+ * @param n The number of operations cached.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+ uint32_t n);
+
+/**
+ * Dequeue a burst of symmetric crypto processing.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue operation count.
+ * @param post_dequeue User provided callback function to
+ * post-process a dequeued operation.
+ * @param out_user_data User data pointer array to be retrieve
+ * from device queue. In case of
+ * *is_user_data_array* is set there
+ * should be enough room to store all
+ * user data.
+ * @param is_user_data_array Set 1 if every dequeued user data will
+ * be written into out_user_data array.
+ * Set 0 if only the first user data will
+ * be written into out_user_data array.
+ * @param n_success Driver written value to specific the
+ * total successful operations count.
+ * @param dequeue_status Driver written value to specify the
+ * dequeue status. Possible values:
+ * - 1: Successfully dequeued the number
+ * of operations returned. The user
+ * data previously set during enqueue
+ * is stored in the "out_user_data".
+ * - 0: The number of operations returned
+ * are completed and the user data is
+ * stored in the "out_user_data", but
+ * they are not freed from the queue
+ * until
+ * rte_cryptodev_raw_dequeue_done()
+ * is called.
+ * - negative integer: Error occurred.
+ * @return
+ * - The number of operations dequeued or completed but not freed from the
+ * queue, depends on "dequeue_status" value.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+ rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+ rte_cryptodev_raw_post_dequeue_t post_dequeue,
+ void **out_user_data, uint8_t is_user_data_array,
+ uint32_t *n_success, int *dequeue_status);
+
+/**
+ * Dequeue a symmetric crypto processing.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param dequeue_status Driver written value to specify the
+ * dequeue status. Possible values:
+ * - 1: Successfully dequeued a operation.
+ * The user data is returned.
+ * - 0: The first operation in the queue
+ * is completed and the user data
+ * previously set during enqueue is
+ * returned, but it is not freed from
+ * the queue until
+ * rte_cryptodev_raw_dequeue_done() is
+ * called.
+ * - negative integer: Error occurred.
+ * @param op_status Driver written value to specify
+ * operation status.
+ * @return
+ * - The user data pointer retrieved from device queue or NULL if no
+ * operation is ready for dequeue.
+ */
+__rte_experimental
+static __rte_always_inline void *
+rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx,
+ int *dequeue_status, enum rte_crypto_op_status *op_status)
+{
+ return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data, dequeue_status,
+ op_status);
+}
+
+/**
+ * Inform the queue pair dequeue operations is finished.
+ *
+ * @param ctx The initialized raw data-path context data.
+ * @param n The number of operations.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+ uint32_t n);
+
#ifdef __cplusplus
}
#endif
git.droids-corp.org / dpdk.git / commitdiff