From: Jerin Jacob Date: Wed, 22 Apr 2020 19:03:49 +0000 (+0530) Subject: doc: add trace library guide X-Git-Url: http://git.droids-corp.org/?p=dpdk.git;a=commitdiff_plain;h=9f8e1810f62bc6549fa7782e550f68130fc4ce8e doc: add trace library guide Add programmer's guide for trace library support. Signed-off-by: Jerin Jacob Signed-off-by: Sunil Kumar Kori Acked-by: David Marchand --- diff --git a/MAINTAINERS b/MAINTAINERS index c456126e2a..e18b2dba7e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -200,6 +200,7 @@ M: Sunil Kumar Kori F: lib/librte_eal/include/rte_trace*.h F: lib/librte_eal/common/eal_common_trace*.c F: lib/librte_eal/common/eal_trace.h +F: doc/guides/prog_guide/trace_lib.rst F: app/test/test_trace* Memory Allocation diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf51..1d0cd49cd7 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -12,6 +12,7 @@ Programmer's Guide overview env_abstraction_layer service_cores + trace_lib rcu_lib ring_lib stack_lib diff --git a/doc/guides/prog_guide/trace_lib.rst b/doc/guides/prog_guide/trace_lib.rst new file mode 100644 index 0000000000..6a2016c7dc --- /dev/null +++ b/doc/guides/prog_guide/trace_lib.rst @@ -0,0 +1,358 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2020 Marvell International Ltd. + +Trace Library +============= + +Overview +-------- + +*Tracing* is a technique used to understand what goes on in a running software +system. The software used for tracing is called a *tracer*, which is +conceptually similar to a tape recorder. +When recording, specific instrumentation points placed in the software source +code generate events that are saved on a giant tape: a trace file. +The trace file then later can be opened in *trace viewers* to visualize and +analyze the trace events with timestamps and multi-core views. +Such a mechanism will be useful for resolving a wide range of problems such as +multi-core synchronization issues, latency measurements, finding out the +post analysis information like CPU idle time, etc that would otherwise be +extremely challenging to get. + +Tracing is often compared to *logging*. However, tracers and loggers are two +different tools, serving two different purposes. +Tracers are designed to record much lower-level events that occur much more +frequently than log messages, often in the range of thousands per second, with +very little execution overhead. +Logging is more appropriate for a very high-level analysis of less frequent +events: user accesses, exceptional conditions (errors and warnings, for +example), database transactions, instant messaging communications, and such. +Simply put, logging is one of the many use cases that can be satisfied with +tracing. + +DPDK tracing library features +----------------------------- + +- A framework to add tracepoints in control and fast path APIs with minimum + impact on performance. + Typical trace overhead is ~20 cycles and instrumentation overhead is 1 cycle. +- Enable and disable the tracepoints at runtime. +- Save the trace buffer to the filesystem at any point in time. +- Support ``overwrite`` and ``discard`` trace mode operations. +- String-based tracepoint object lookup. +- Enable and disable a set of tracepoints based on regular expression and/or + globbing. +- Generate trace in ``Common Trace Format (CTF)``. ``CTF`` is an open-source + trace format and is compatible with ``LTTng``. + For detailed information, refer to + `Common Trace Format `_. + +How to add a tracepoint? +------------------------ + +This section steps you through the details of adding a simple tracepoint. + +.. _create_provider_header_file: + +Create the tracepoint provider header file +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + #include + + RTE_TRACE_POINT( + app_trace_string, + RTE_TRACE_POINT_ARGS(const char *str), + rte_trace_point_emit_string(str); + ) + +The above macro creates ``app_trace_string`` tracepoint. +The user can choose any name for the tracepoint. +However, when adding a tracepoint in the DPDK library, the +``rte__trace_[_]`` naming convention must be +followed. +The examples are ``rte_eal_trace_generic_str``, ``rte_mempool_trace_create``. + +The ``RTE_TRACE_POINT`` macro expands from above definition as the following +function template: + +.. code-block:: c + + static __rte_always_inline void + app_trace_string(const char *str) + { + /* Trace subsystem hooks */ + ... + rte_trace_point_emit_string(str); + } + +The consumer of this tracepoint can invoke +``app_trace_string(const char *str)`` to emit the trace event to the trace +buffer. + +Register the tracepoint +~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + /* Select tracepoint register macros */ + #define RTE_TRACE_POINT_REGISTER_SELECT + + #include + + RTE_TRACE_POINT_DEFINE(app_trace_string); + + RTE_INIT(app_trace_init) + { + RTE_TRACE_POINT_REGISTER(app_trace_string, app.trace.string); + } + +The above code snippet registers the ``app_trace_string`` tracepoint to +trace library. Here, the ``my_tracepoint_provider.h`` is the header file +that the user created in the first step :ref:`create_provider_header_file`. + +The second argument for the ``RTE_TRACE_POINT_REGISTER`` is the name for the +tracepoint. This string will be used for tracepoint lookup or regular +expression and/or glob based tracepoint operations. +There is no requirement for the tracepoint function and its name to be similar. +However, it is recommended to have a similar name for a better naming +convention. + +The user must register the tracepoint before the ``rte_eal_init`` invocation. +The user can use the ``RTE_INIT`` construction scheme to achieve this. + +.. note:: + + The ``RTE_TRACE_POINT_REGISTER_SELECT`` must be defined before including the + header for the tracepoint registration to work properly. + +.. note:: + + The ``RTE_TRACE_POINT_DEFINE`` defines the placeholder for the + ``rte_trace_point_t`` tracepoint object. The user must export a + ``__`` symbol in the library ``.map`` file for this + tracepoint to be used out of the library, in shared builds. + For example, ``__app_trace_string`` will be the exported symbol in the + above example. + +Fast path tracepoint +-------------------- + +In order to avoid performance impact in fast path code, the library introduced +``RTE_TRACE_POINT_FP``. When adding the tracepoint in fast path code, +the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. + +``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using +``CONFIG_RTE_ENABLE_TRACE_FP`` configuration parameter. +The ``enable_trace_fp`` option shall be used for the same for meson build. + +Event record mode +----------------- + +Event record mode is an attribute of trace buffers. Trace library exposes the +following modes: + +Overwrite + When the trace buffer is full, new trace events overwrites the existing + captured events in the trace buffer. +Discard + When the trace buffer is full, new trace events will be discarded. + +The mode can be configured either using EAL command line parameter +``--trace-mode`` on application boot up or use ``rte_trace_mode_set()`` API to +configure at runtime. + +Trace file location +------------------- + +On ``rte_trace_save()`` or ``rte_eal_cleanup()`` invocation, the library saves +the trace buffers to the filesystem. By default, the trace files are stored in +``$HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/``. +It can be overridden by the ``--trace-dir=`` EAL command line +option. + +For more information, refer to :doc:`../linux_gsg/linux_eal_parameters` for +trace EAL command line options. + +View and analyze the recorded events +------------------------------------ + +Once the trace directory is available, the user can view/inspect the recorded +events. + +There are many tools you can use to read DPDK traces: + +1. ``babeltrace`` is a command-line utility that converts trace formats; it +supports the format that DPDK trace library produces, CTF, as well as a +basic text output that can be grep'ed. +The babeltrace command is part of the Open Source Babeltrace project. + +2. ``Trace Compass`` is a graphical user interface for viewing and analyzing +any type of logs or traces, including DPDK traces. + +Use the babeltrace command-line tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The simplest way to list all the recorded events of a trace is to pass its path +to babeltrace with no options:: + + babeltrace + +``babeltrace`` finds all traces recursively within the given path and prints +all their events, merging them in chronological order. + +You can pipe the output of the babeltrace into a tool like grep(1) for further +filtering. Below example grep the events for ``ethdev`` only:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev + +You can pipe the output of babeltrace into a tool like wc(1) to count the +recorded events. Below example count the number of ``ethdev`` events:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev | wc --lines + +Use the tracecompass GUI tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``Tracecompass`` is another tool to view/analyze the DPDK traces which gives +a graphical view of events. Like ``babeltrace``, tracecompass also provides +an interface to search for a particular event. +To use ``tracecompass``, following are the minimum required steps: + +- Install ``tracecompass`` to the localhost. Variants are available for Linux, + Windows, and OS-X. +- Launch ``tracecompass`` which will open a graphical window with trace + management interfaces. +- Open a trace using ``File->Open Trace`` option and select metadata file which + is to be viewed/analyzed. + +For more details, refer +`Trace Compass `_. + +Quick start +----------- + +This section steps you through the details of generating trace and viewing it. + +- Start the dpdk-test:: + + echo "quit" | ./build/app/test/dpdk-test --no-huge --trace=.* + +- View the traces with babeltrace viewer:: + + babeltrace $HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/ + +Implementation details +---------------------- + +As DPDK trace library is designed to generate traces that uses ``Common Trace +Format (CTF)``. ``CTF`` specification consists of the following units to create +a trace. + +- ``Stream`` Sequence of packets. +- ``Packet`` Header and one or more events. +- ``Event`` Header and payload. + +For detailed information, refer to +`Common Trace Format `_. + +The implementation details broadly divided into the following areas: + +Trace metadata creation +~~~~~~~~~~~~~~~~~~~~~~~ + +Based on the ``CTF`` specification, one of a CTF trace's streams is mandatory: +the metadata stream. It contains exactly what you would expect: data about the +trace itself. The metadata stream contains a textual description of the binary +layouts of all the other streams. + +This description is written using the Trace Stream Description Language (TSDL), +a declarative language that exists only in the realm of CTF. +The purpose of the metadata stream is to make CTF readers know how to parse a +trace's binary streams of events without CTF specifying any fixed layout. +The only stream layout known in advance is, in fact, the metadata stream's one. + +The internal ``trace_metadata_create()`` function generates the metadata. + +Trace memory +~~~~~~~~~~~~ + +The trace memory will be allocated through an internal function +``__rte_trace_mem_per_thread_alloc()``. The trace memory will be allocated +per thread to enable lock less trace-emit function. +The memory for the trace memory for DPDK lcores will be allocated on +``rte_eal_init()`` if the trace is enabled through a EAL option. +For non DPDK threads, on the first trace emission, the memory will be +allocated. + +Trace memory layout +~~~~~~~~~~~~~~~~~~~ + +.. _table_trace_mem_layout: + +.. table:: Trace memory layout. + + +-------------------+ + | packet.header | + +-------------------+ + | packet.context | + +-------------------+ + | trace 0 header | + +-------------------+ + | trace 0 payload | + +-------------------+ + | trace 1 header | + +-------------------+ + | trace 1 payload | + +-------------------+ + | trace N header | + +-------------------+ + | trace N payload | + +-------------------+ + +packet.header +^^^^^^^^^^^^^ + +.. _table_packet_header: + +.. table:: Packet header layout. + + +-------------------+ + | uint32_t magic | + +-------------------+ + | rte_uuid_t uuid | + +-------------------+ + +packet.context +^^^^^^^^^^^^^^ + +.. _table_packet_context: + +.. table:: Packet context layout. + + +----------------------+ + | uint32_t thread_id | + +----------------------+ + | char thread_name[32] | + +----------------------+ + +trace.header +^^^^^^^^^^^^ + +.. _table_trace_header: + +.. table:: Trace header layout. + + +----------------------+ + | event_id [63:48] | + +----------------------+ + | timestamp [47:0] | + +----------------------+ + +The trace header is 64 bits, it consists of 48 bits of timestamp and 16 bits +event ID. + +The ``packet.header`` and ``packet.context`` will be written in the slow path +at the time of trace memory creation. The ``trace.header`` and trace payload +will be emitted when the tracepoint function is invoked. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 7f2049a0fd..89a94a7464 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -56,6 +56,16 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **Added Trace Library and Tracepoints** + + A native implementation of ``common trace format(CTF)`` based trace library + has been added to provide the ability to add tracepoints in + application/library to get runtime trace/debug information for control and + fast APIs with minimum impact on fast path performance. + Typical trace overhead is ~20 cycles and instrumentation overhead is 1 cycle. + Added tracepoints in ``EAL``, ``ethdev``, ``cryptodev``, ``eventdev`` and + ``mempool`` libraries for important functions. + * **Added new API for rte_ring.** * New synchronization modes for rte_ring.