Changes between Version 80 and Version 81 of Developer/Projects/Open/TraceTool


Ignore:
Timestamp:
Mar 7, 2016, 5:03:38 AM (4 years ago)
Author:
Chris Johns
Comment:

update for GSoC 2016.

Legend:

Unmodified
Added
Removed
Modified
  • Developer/Projects/Open/TraceTool

    v80 v81  
    99Chris Johns is the primary contact on this. Please check [wiki:Developer/Tracing Tracing] for updated information.
    1010
    11 Instrument RTEMS so that a timeline visualization tool can be used to "place a scope" on a running RTEMS system.  This project has multiple parts.
     11The RTEMS trace project develops the RTEMS Trace framework. The framework provides a consistent user interface to instrumenting an application and tracing it. The framework has multiple parts.
    1212
    13  *  Addition of trace points to generate log entries
    14  *  Transmission of logged data to host
    15  *  Receipt of logged data on host
    16  *  Decoding of logged data on host
    17  *  GUI Timeline Visualization tool
     13 * Instrumenting the application using the RTEMS Trace Linker (rtems-tld).
     14 * Target execution and capture of data.
     15 * Control of event triggering and logging.
     16 * Integration to 3rd party trace toolkits.
    1817
    19 The requirements of the Run-Time tracing is to provide a common means audit an RTEMS application for conformance of input/output parameters and/or timing, or to debug an application. The auditing process typically uses a static configuration the user manages. The debugging process usually targets a standard set of calls such as the RTEMS Classic API and uses dynamic filtering to control what is looked at as the debugging session unfolds.
     18There are a number of uses cases for tracing and these effect the requirements for the trace framework. They are:
     19
     20 * Audit an RTEMS application for conformance of input/output parameters and/or timing
     21 * Debug an application.
     22 * Realtime visualization.
    2023
    2124'''Acknowledgements'''
     
    2427 *  Cocan Lucian Marius worked on this project for Google Summer of Code 2009
    2528 *  Zeng (Alana) Reng worked on this project for Google Summer of Code 2008.
     29
    2630= Introduction =
    2731
     32The following are the open tasks for the Trace project:
    2833
    29 The RTEMS trace tool is a tool that allows users to integrate tracing into their applications. The following diagram shows the flow and status of the project as of the end of Google Summer of Code 2009.
     34 1. RTEMS Trace linker (rtems-tld) Symbol Signatures
     35 1. Target Integration of the Capture Engine with rtems-tld
     36 1. CTF integration.
     37 1. Tracing tools improvements.
    3038
    31 [[Image(wiki:GSoC/2009/Wrapup:RTEMS_Trace.png, 800px)]]
     39== RTEMS Trace linker (rtems-tld) Symbol Signatures ==
    3240
    33 = Command line interface =
     41Update the elftools component and add libdwarf support to rtems-tools. Provide a C++ framework in a similar manner to the ELF framework to wrap libdwarf. The current use case for libdwarf is to get a function signature of a library symbol from the debug info and with this create a suitable function signature for the trace linker. Currently this is manually done by adding the signatures to the generator INI files. The manual approach is a stop gap measure and is not maintainable.
    3442
    35 The RTEMS Trace Tool can currently work in 4 modes:
     43This task requires good C++ skills.
    3644
    37  *  Generate mode
    38  *  Compile mode
    39  *  Link mode
    40  *  Decode mode
    41 = Generate mode =
     45== Target Integration of the Capture Engine with rtems-tld ==
    4246
     47The Capture Engine is a module in RTEMS for the target that provides low level services for buffers, triggers and captures for the target. This task integrates the Capture Engine to the RTEMS Trace Linker to allow rtems-tld linked application to use the Capture Engine for the capture, buffering and transport of the trace data. The task also includes looking at adding other transports to the Capture Engine other than a command line and console output. A role of the Capture Engine is to provide an API users can access to allow custom transports. This is important with custom hardware with restricted external interfaces.
    4348
    44   Usage: ./rtems_trace.py '''-m''' ''generate''
    45                           '''-g''' ''path to where the wrapping code should be saved''
    46                           '''-c''' ''path to the configuration file''
    47          ./rtems_trace.py '''--mode'''=''generate''
    48                           '''--genpath'''=''path to where the wrapping code should be saved''
    49                           '''--config'''=''path to the configuration file''
     49== CTF integration ==
    5050
    51 Using this mode the user will generate the wrapping code, which is a set of files corresponding to the functions that he specified in the configuration file. If the --mode option it's not specified then the default mode chosen by the tool will be ''generate''. If the -c (--config) option it's not specified, then the tool will always look for the default configuration file ''rtems-trace.cfg'' from your current working directory. So if the user wants to generate and save the wrapping code in a directory named ''wrappers'' from the current working directory (cwd) which is the default path, then he can simply do this:
     51> Isaac, can this entry please be updated? Thanks Chris.
    5252
    53    ./rtems_trace.py
    54 = Compile mode =
     53CTF is really neat and brings to RTEMS a range of features we really need. The format is good and applicable to RTEMS and the post processing and real-time monitoring tools are valuable.
    5554
    56  
    57    Usage: ./rtems_trace.py '''-m''' ''compile''
    58                            '''-i''' ''name of the compiler''
    59                            '''-g''' ''path to the wrapping code''
    60                            '''-o''' ''path to where the compiled wrapping code should be saved''
    61                            '''-l''' ''path to where the RTEMS libraries were installed''
    62           ./rtems_trace.py '''--mode'''=''compile''
    63                            '''--compiler'''=''name of the compiler''
    64                            '''--genpath'''=''path to the wrapping code''
    65                            '''--objpath'''=''path to where the compiled wrapping code should be saved''
    66                            '''--lib'''=''path to where the RTEMS libraries were installed''
     55The CTF has a range of tools and parts and this task is to bring those together and into the RTEMS Trace Linker. The goal is to encapsulate what is needed to use CTF and what it provides with the functionality of the RTEMS Trace Linker to build an application for a user that uses CTF. The rtems-tld tool knows about RTEMS and function signatures (see task 1) and it can be taught to internally invoke the CTF tools to create a required meta-data files and target code to integrate with the CTF post processing tools. Being able to use the Capture Engine is considered an advantage.
    6756
    68 Using this mode the user can compile all the .c files from a directory (usually the .c files from the wrapping code directory) and save the resulted files (.o files) in the directory specified by the -o option. If the -g and -o options are not specified then the rtems_trace tool will look in the ''wrappers'' directory located in cwd and save the .o files in a directory named ''objects'' also from cwd. Also, if the -l option it's not specified than the tool will choose this path by default: ''/opt/rtems-4.10/i386-rtems4.10/pc386/lib/''. <BR>
    69 Example:
     57This task is not as defined as the other ones and requires some research and analysis following on from what Isaac has already done. Isaac has discussed the remote protocol and this is an important part and worth plenty of attention.
    7058
    71    ./rtems_trace.py --mode=compile --compiler=i386-rtems4.10-gcc
     59== Tracing Improvements ==
    7260
    73 will compile all the .c files from the cwd/wrappers using the gcc compiler for the i386 BSP and the 4.10 rtems version. The .o files will be saved in cwd/objects.
    74 = Link mode =
     61This is an opened ended task and not one to be initially focused on but is here for completeness.
    7562
     63The rtems-tld uses the binutils symbol mapping feature in GNU ld and I have observed some interesting behaviour with it. It would be good to characterise this behaviour and if possible improve the support in GNU ld. I however feel there are limitations, eg tracing static C functions internally optimised by the compiler, C++ and templates, and more. Long term there maybe the need to add passes to gcc to allow even better trace control. I can see us using the [[https://fedorahosted.org/gcc-python-plugin/ GCC Python Plug-In]] tool.
    7664
    77    Usage: ./rtems_trace.py '''-m''' ''link''
    78                            '''-i''' ''name of the compiler''
    79                            '''-b''' ''RTEMS build path''
    80                            '''-o''' ''path to the compiled code (.o files)''
    81                            '''-'''  ''list of comma separated object files that should be included in the final executable''
    82                            '''-l''' ''path to where the RTEMS libraries were installed''
    83           ./rtems_trace.py '''--mode'''=''link''
    84                            '''--compiler'''=''name of the compiler''
    85                            '''--build'''=''RTEMS build path''
    86                            '''--objpath'''=''path to the compiled code(.o files)''
    87                            '''--trace-o'''=''list of comma separated object files that should be included in the final executable''
    88                            '''--lib'''=''path to where the RTEMS libraries were installed''
    89 
    90 In this mode, the RTEMS trace tool will link all the object files from the specified directory (or from the default one if no directory is specified using the -o or --objpath option) if the --trace-o option it's not used. The user can specify only a subset of object files to be included in the final executable using the --trace-o option. The final executable file is placed into the same directory where the object files are found, under the ''wrapper.exe'' name. <BR>
    91 Example:
    92 
    93    ./rtems_trace.py --mode=link --compiler=i386-rtems4.10-gcc -build=/home/cocan/Faculta/RTEMS/build-i386-qemu-rtems
    94 
    95 will link all the object files from ''cwd/objects'', generating the final executable file ''cwd/objects/wrapper.exe''
    96 = Decode mode =
    97 
    98 
    99    Usage: ./rtems_trace.py '''-m''' ''decode''
    100                            '''-t''' ''path to the trace file''
    101                            '''-d''' ''where to save the final decoded data''
    102                            '''-c''' ''path to the configuration file''
    103           ./rtems_trace.py '''--mode'''=''decode''
    104                            '''--trace-file'''=''path to the trace file''
    105                            '''--dec-file'''=''where to save the final decoded data''
    106                            '''--config'''=''path to the configuration file''
    107 
    108 In this mode, the RTEMS trace tool will read the input file, specified using the -t or --trace-file option and decode it, generating the final decoded data in the file specified by the -d or --dec-file option. If these options are not specified then the file ''cwd/log.txt'' will be used as input file and ''cwd/decoded.txt'' will be used as output file.
    109 = Configuration files =
    110 
    111 
    112 There can be two types of configuration files: the main configuration file, where the user specifies the functions that he wants to trace and the API configuration file. Both of these two types of configuration files are presented in the next subsections.
    113 = Main configuration file – default: rtems-trace.cfg =
    114 
    115 
    116 In order for the rtems-trace tool to function correctly the user must provide a configuration file which should have the following format and include the following sections:
    117 
    118    1. include "''path_to_an_API_config_file''"
    119 This section specifies paths to other configuration files (called API configuration files) which contain information that is needed for the encoding of data and generation of ID's (functions ID, manager ID, API id etc). The section  “API configuration files” explains what these configuration files should contain. Check this for an example.
    120 
    121    2. rtems-functions: ''list of comma separated function names''
    122 This section should contain a list of comma separated function names that the user wants to trace. Check this for an example.
    123 
    124    3. rtems-managers: ''list of comma separated manager names''
    125 This section should contain a list of comma separated manager names. The manager names should be specified like this: API_name-manager_name. This is to help the user to specify more easily hole sets of functions.
    126 Example: If the user wants to trace all the functions from the task manager from the Classic API then he could do this:
    127 
    128    rtems-managers: classic-task
    129 
    130 Rather than doing this:
    131 
    132    rtems-functions: rtems_task_create, rtems_task_ident
    133                     rtems_task_self, rtems_task_start
    134                     rtems_task_restart, rtems_task_delete
    135                     rtems_task_suspend, rtems_task_resume
    136                     rtems_task_is_suspended, rtems_task_set_priority
    137                     rtems_task_mode, rtems_task_get_note
    138                     rtems_task_set_note, rtems_task_wake_after
    139                     rtems_task_wake_when, rtems_iterate_over_all_threads
    140                     rtems_task_variable_add, rtems_task_variable_get
    141                     rtems_task_variable_delete
    142 
    143    4. rtems-api: ''list of comma separated API names''
    144 This section should contain a list of comma separated API names. This is also to help the user to specify more easily hole sets of functions.
    145 Example: If the user wants to trace all the functions from the Classic API then he could do this:
    146 
    147     rtems-api: classic
    148 
    149 Rather than doing this:
    150 
    151     rtems-managers: classic-task, classic-interrupt, classic-clock, classic-timer,
    152                     classic-semaphore, classic-message, classic-event, classic-signal,
    153                     classic-io, classic-init
    154 
    155    5. rtems-libraries: ''list of comma separated library names''
    156 This section should contain a list of comma separated library names. Check this for an example.
    157 
    158    6. rtems-symbols-path: ''path to the folder containing the symbols files''
    159 This section should contain a valid path to a directory containing the symbols files. The symbols files (functions.sym, structs.sym, typedefs.sym, unions.sym) contain information, as their name suggest, about functions signature, struct, typedef and union definition. These files are specific to a library and should be found in a directory that has the same name as the library from which the files resulted.
    160 
    161         Below is a sample configuration file.
    162 
    163    # This is a comment.
    164    # Sample configuration file
    165    # Include section
    166    '''include''' "rtems_trace_classic.cfg"
    167    '''include''' "rtems_trace_posix.cfg" <BR>
    168    # The user can specify just some functions specific to an API to trace
    169    # or a hole set of functions specific to a manager from an API.
    170    # Specify some functions.
    171    '''rtems-functions''': rtems_task_create, rtems_task_delete <BR>
    172    # Specify a manager from an API.
    173    # API_name-manager_name
    174    '''rtems-managers''': classic-task, classic-interrupt, classic-clock, classic-timer, classic-semaphore,
    175                    classic-message, classic-event, classic-signal, classic-io, classic-init,
    176                    posix-mutex, posix-semaphore, posix-clock, posix-timer, posix-condvar,
    177                    posix-scheduler, posix-message <BR>
    178    # Specify a hole API.
    179    '''rtems-api''': classic, posix <BR>
    180    # Specify the libraries.
    181    '''rtems-libraries''': librtems.a, librtemscpu.a, libposix.a <BR>
    182    # The path to the symbols files.
    183    '''rtems-symbols-path''': /home/cocan/Faculta/RTEMS/gsoc_2009/branch-rtems-trace/rtems-trace/bin/rtems-trace/symbols/librtems/
    184 = API configuration file =
    185 
    186 
    187 An API configuration file should contain the following sections:
    188 
    189    1. API: ''name = name_of_API, id = numeric value''
    190    2. manager_name-id: ''numeric value''
    191    3. manager_name-functions-id: ''name = name_of_function, class = class_type, id = numeric_value''
    192    4. manager_name-libraries: ''set of libraries''
    193 
    194 All the API configuration files should include this information using the format specified.
    195 
    196 The following is an example of an API configuration file, for the classic API:
    197 
    198    '''API''': name = classic, id = 1 <BR>
    199    # Task manager
    200    '''task-id''': 1
    201    '''task-functions-id''': name = rtems_task_create, class = API, id = 1       
    202                       name = rtems_task_ident, class = API, id = 2
    203                       name = rtems_task_self, class = API, id = 3
    204                       name = rtems_task_start, class = API, id = 4
    205                       name = rtems_task_restart, class = API, id = 5
    206                       name = rtems_task_delete, class = API, id = 6
    207                       name = rtems_task_suspend, class = API, id = 7
    208                       name = rtems_task_resume, class = API, id = 8
    209                       name = rtems_task_is_suspended, class = API, id = 9
    210                       name = rtems_task_set_priority, class = API, id = 10
    211                       name = rtems_task_mode, class = API, id = 11
    212                       name = rtems_task_get_note, class = API, id = 12
    213                       name = rtems_task_set_note, class = API, id = 13
    214                       name = rtems_task_wake_after, class = API, id = 14
    215                       name = rtems_task_wake_when, class = API, id = 15
    216                       name = rtems_iterate_over_all_threads, class = API, id = 16
    217                       name = rtems_task_variable_add, class = API, id = 17
    218                       name = rtems_task_variable_get, class = API, id = 18
    219                       name = rtems_task_variable_delete, class = API, id = 19
    220    '''task-libraries''': librtems.a <BR>
    221    # Initialization manager
    222    '''init-id''': 2
    223    '''init-functions-id''': name = rtems_initialize_data_structures, class = API, id = 1
    224                       name = rtems_initialize_before_drivers, class = API, id = 2
    225                       name = rtems_initialize_device_drivers, class = API, id = 3
    226                       name = rtems_initialize_start_multitasking, class = API, id = 4
    227                       name = rtems_shutdown_executive, class = API, id = 5
    228    '''init-libraries''': librtemscpu.a <BR>
    229    # Interrrupt manager
    230    '''interrupt-id''': 3
    231    '''interrupt-functions-id''': name = rtems_interrupt_catch, class = API, id = 1
    232                            name = rtems_interrupt_disable, class = API, id = 2
    233                            name = rtems_interrupt_enable, class = API, id = 3
    234                            name = rtems_interrupt_flash, class = API, id = 4
    235                            name = rtems_interrupt_is_in_progress class = API, id = 5
    236    '''interrupt-libraries''': librtemscpu.a
    237    # And so on...
    238 = Tutorial =
    239 
    240 The following section will try to illustrate how to properly use the RTEMS trace tool in order to get the decoded file containing the information needed. The BSP used in this example is '''i386''' and the RTEMS version is '''4.10'''.
    241 
    242  *  Set up the configuration files as explained [http://www.rtems.org/wiki/index.php/RTEMS_Trace_Tool#Configuration_files here].
    243  *  Generate the wrapping code:
    244                          
    245    ./rtems_trace.py
    246 
    247  *  Compile the wrapping and the application code (copy your application code in the ''cwd/wrappers'' directory).
    248 
    249    ./rtems_trace.py --mode=compile --compiler=i386-rtems4.10-gcc
    250 
    251  *  Do the linking.
    252 
    253    ./rtems_trace.py --mode=link --compiler=i386-rtems4.10-gcc -build=/home/cocan/Faculta/RTEMS/build-i386-qemu-rtems
    254 
    255  *  Run the final executable in QEMU in order to get the trace file needed for decoding.
    256 TBD [wiki:TBR/UserManual/QemuTutorial QemuTutorial]
    257  *  Decode the trace file (copy the trace file in ''cwd'' and name it ''log.txt'' if it has a different name).
    258 
    259    ./rtems_trace.py --mode=decode
    260 
    261 This is it. All the tracing information it's now saved in the ''cwd/decoded.txt'' file.
    262 = Open Projects =
    263 
    264 
    265 This has been worked on during the 2008 and 2009 Google Summer of Code. 
    266 
    267 Chris Johns needs to merge this code and update this status.  He should be contacted for detailed information on the next logical step.
    268 
    269 See [wiki:Projects/LoggingTools LoggingTools] for ideas on visualization
     65For this to happen I would like to see a separate GCC Plug-In project within RTEMS for RTEMS started and trace support being just one part of it. This would allow other interesting but yet to be defined things requiring GCC plug-in passes to be collected in one place. A separate project for special gcc passes would allow auditing and controls to be added so those users in our community who need to know everything about the code running in their target can control what happens.