RTEMS Trace Tool

RTEMS Trace Tool - trace your application in RTEMS

Chris Johns is the primary contact on this. Please check Tracing for updated information.

The RTEMS trace project develops the RTEMS Trace framework. The framework provides a consistent user interface to instrumenting an application and tracing it. The framework has multiple parts.

  • Instrumenting the application using the RTEMS Trace Linker (rtems-tld).
  • Target execution and capture of data.
  • Control of event triggering and logging.
  • Integration to 3rd party trace toolkits.

There are a number of uses cases for tracing and these effect the requirements for the trace framework. They are:

  • Audit an RTEMS application for conformance of input/output parameters and/or timing
  • Debug an application.
  • Realtime visualization.


  • Chris Johns is the father of this effort and has mentored two Google Summer of Code students.
  • Cocan Lucian Marius worked on this project for Google Summer of Code 2009
  • Zeng (Alana) Reng worked on this project for Google Summer of Code 2008.


The following are the open tasks for the Trace project:

  1. RTEMS Trace linker (rtems-tld) Symbol Signatures
  2. Target Integration of the Capture Engine with rtems-tld
  3. CTF integration.
  4. Tracing tools improvements.

Note: The rtems-tld tool is written in C++ and to work on this tool you need good C++ coding skills and some knowledge of using the STL. The tools has a tight coding standard that is not documented but plenty of code exists you can see and follow. Help is available so please ask if you need help and are willing to learn.

RTEMS Trace linker (rtems-tld) Symbol Signatures

Update the elftools component and add libdwarf support to rtems-tools. Provide a C++ framework in a similar manner to the ELF framework to wrap libdwarf. The current use case for libdwarf is to get a function signature of a library symbol from the debug info and with this create a suitable function signature for the trace linker. Currently this is manually done by adding the signatures to the generator INI files. The manual approach is a stop gap measure and is not maintainable.

This task requires good C++ skills.

Target Integration of the Capture Engine with rtems-tld

The Capture Engine is a module in RTEMS for the target that provides low level services for buffers, triggers and captures for the target. This task integrates the Capture Engine to the RTEMS Trace Linker to allow rtems-tld linked application to use the Capture Engine for the capture, buffering and transport of the trace data. The task also includes looking at adding other transports to the Capture Engine other than a command line and console output. A role of the Capture Engine is to provide an API users can access to allow custom transports. This is important with custom hardware with restricted external interfaces.

CTF integration

CTF is really neat and brings to RTEMS a range of features we really need. The format is good and applicable to RTEMS and the post processing and real-time monitoring tools are valuable.

The CTF has a range of tools and parts and this task is to bring those together and into the RTEMS Trace Linker. The goal is to encapsulate what is needed to use CTF and what it provides with the functionality of the RTEMS Trace Linker to build an application for a user that uses CTF. The rtems-tld tool knows about RTEMS and function signatures (see task 1) and it can be taught to internally invoke the CTF tools to create a required configuration files and target code to integrate with the CTF post processing tools. Being able to use the Capture Engine is considered an advantage.

A key part is barectf. This accepts a YAML configuration and generates suitable C code for integration into the application. This tool can be wrapped inside the rtems-tld and used transparently to the user. The rtems-tld can build any barectf generated code for a target because it already knows the compiler and flags required by the target. The user's code is wrapped by the GNU ld and that creates the dependencies to the rtems-tld generated code and in turn to the barectf generated code. No user code is touched at the source or even the object file level and this is an important requirement.

What follow is mostly guess work on my part and needs to be clarified, corrected and cleaned up as we learn more. I base what is written on the current wrapping of trace calls is controlled by rtems-tld and so it can generate the barectf configuration file. The barectf generated code takes in the trace data and returns CTF format records suitable for buffering. The Capture Engine is used because it provides safe concurrent buffering of data. The project is bring all this together.

The task can be divided up into:

  1. Investigate how barectf works and what code is generated.
  2. Determine how the trace wrapped calls created by rtems-tld integrate to the barectf trace support. I suspect this is a matter of assigning event ids in the barectf configuration file to match the rtems-tld generated ids.
  3. Create an rtems-tld generator to generate C code to bind to the barectf calls.
  4. Integrate the barectf output support to the Capture Engine so it performs the buffering. This assumes the barectf code generated is sequential and event processing is sequential and contains no code to update global variables. The concurrent activity is the buffering and this is handled by the Capture Engine.
  5. Determine how to add extra special features such as CPU number.

The final part of this task is transport to the host machine. How this happens has not been determined yet and is open to suggestions and innovation.

Tracing Improvements

This is an opened ended task and not one to be initially focused on but is here for completeness.

The rtems-tld uses the binutils symbol mapping feature in GNU ld and I have observed some interesting behaviour with it. It would be good to characterise this behaviour and if possible improve the support in GNU ld. I however feel there are limitations, eg tracing static C functions internally optimised by the compiler, C++ and templates, and more. Long term there maybe the need to add passes to gcc to allow even better trace control. I can see us using the GCC Python Plug-In tool.

For this to happen I would like to see a separate GCC Plug-In project within RTEMS for RTEMS started and trace support being just one part of it. This would allow other interesting but yet to be defined things requiring GCC plug-in passes to be collected in one place. A separate project for special gcc passes would allow auditing and controls to be added so those users in our community who need to know everything about the code running in their target can control what happens.

Last modified on Mar 9, 2016 at 10:34:21 PM Last modified on Mar 9, 2016, 10:34:21 PM