Changes between Version 22 and Version 23 of TBR/Review/Debugging/Start


Ignore:
Timestamp:
Nov 19, 2018, 8:22:35 PM (7 months ago)
Author:
Joel Sherrill
Comment:

Removed obviously out of date material that has current content elsewhere

Legend:

Unmodified
Added
Removed
Modified
  • TBR/Review/Debugging/Start

    v22 v23  
    11= Debugging =
    22= **WARNING** THE CONTENT MAY BE OUT OF DATE =
    3 = Symbolic Debug Information =
    43
    5 
    6 RTEMS and the RTEMS tool chains support symbolic debugging. The application, RTEMS and any libraries you wish to debug need to be compiled with the '''gcc''' option '''-g'''. This option causes the compiler to emit debugging information, while the code generated should not change. The '''GNU gcc''' debugging options can be found here -
    7 
    8 <center>
    9 http://gcc.gnu.org/onlinedocs/gcc-3.4.3/gcc/Debugging-Options.html#Debugging-Options
    10 </center>
    11 
    12 Using the '''-g''' option results in larger object and executable file sizes, and for C++ this can be quite large. For example a M68K C++ application can have a ELF executable file size of 19M bytes yet the code size is only 1.6M bytes. The actual memory foot print can be seen by using the '''size''' tool on an object file or the final executable -
    13 
    14 {{{
    15 $ m68k-rtems-size myfile.o
    16 }}}
    17 
    18 For RTEMS you typically do not need to strip the executable. The loading of the executable into the target memory or conversion to S records, Intel Hex, for binary format will automatically strip the debug information. Keeping the excutable with the debugging information is recommended. If a problem appears with code in the field and you recieve a some sort of dump or trace you can use the '''objdump''' tool to help locate the problem -
    19 
    20 {{{
    21 $ m68k-rtems-objdump -D --line-numbers --source --demangle  myapp.elf | less
    22 }}}
    234= Hardware Assisted Debugging =
    245
     
    5738
    5839 * Use the [wiki:TBR/UserManual/Capture_Engine Capture Engine] to aid the debugging and verfication of your real-time design.
    59 = GDB and RTEMS =
    6040
    6141
    62 Currently GDB is not RTEMS aware. GDB scripts exist that can help by providing presenting kernel structures in a user friendly manner. These can be found here: [wiki:Debugging/GDBScripts GdbScripts].
    63 
    64 The issue with making GDB aware of RTEMS is where the knowledge of the kernel structures is located. A version of GDB that contains a specific kernel structure layout will break if RTEMS changes. The ideal would be for GDB to find the structure elements using the applications debug information.
    65 = Eclipse Plug-in =
    66 
    67 Eclipse can be used as GUI for GDB. General Eclipse related information can be found at [wiki:Developer/Eclipse/Information RTEMS Eclipse Information]. An more specific example of how serial port is utilized for remote debugging can be found at [wiki:Developer/Eclipse/Plugin RTEMS Eclipse Plug-in].
    68 = How much memory is left in the C Program Heap? =
    69 
    70 
    71 The C heap is a region so this should work:
    72 
    73 {{{
    74   (gdb) p ((Region_Control *)_Region_Information->local_table[1])->Memory->first
    75   $9 {back_flag=1, front_flag=8058280, next=0x7ea5b4, previous=0x7ea5b0}
    76 }}}
    77 
    78 Let's look at that gdb command in more detail.
    79 
    80  * ''_Region_Information'' contains the information used to manage Region objects.
    81  * ''_Region_Information->local_table'' is the object pointer table for Regions.  It is
    82   indexed by the ''object index'' portion of the object ID.
    83  * ''Region_Information->local_table'' points to the first Region object.  It is of type ''(Region_Control *)''.
    84  * ''((Region_Control *)_Region_Information->local_table[1])->Memory'' points to the Heap control portion of this Region's control block.
    85  * ''((Region_Control *)_Region_Information->local_table[1])->Memory->first'' references the contents of the first ''heap block'' on this Heap.
    86 
    87 Notice that the ''front_flag'' is displayed as ''8058280''.  This is in decimal since we used ''p'' not ''p/x'' to gdb.  Since this number is even, we know the in use bit is 0 and the block is free.  Thus the first block on the heap is 8,058,280 bytes and there are at least that many bytes left.
    88 
    89 '''NOTE:''' This is really a crude estimate.
    90 
    91 If you have compiled RTEMS libraries with -DRTEMS_DEBUG, malloc will maintain statistics. From an RTEMS application:
    92 
    93 {{{
    94 #include <libcsupport.h>
    95     .....
    96   malloc_dump ();
    97 }}}
    98 
    99 will print malloc's heap statistics to ''stdout''. Example usage can be found in <tt>c/src/tests/frye/libcleak</tt> and <tt>c/src/tests/libtests/malloctest</tt>.
    100 
    101 The following GDB macro may also be of use. If provided with an argument of '''0''' for a summary, or '''1''' to explicitly list the regions.
    102 
    103 {{{
    104  define rtems-mallocheap-walk
    105    printf "walking the heap:\n"
    106    set $heapstart = ((Region_Control *)_Region_Information->local_table[RTEMS_Malloc_Heap&0xffff])->Memory->start
    107    set $currentblock = $heapstart
    108    set $used = 0
    109    set $numused = 0
    110    set $free = 0
    111    set $numfree = 0
    112    while $currentblock->front_flag != 1
    113      if $currentblock->front_flag & 1
    114        if $arg0 != 0
    115         printf "USED: %d\n", $currentblock->front_flag & ~1
    116        else
    117          printf "*"
    118        end
    119        set $used = $used + $currentblock->front_flag & ~1
    120        set $numused = $numused + 1
    121      else
    122        if $arg0 != 0
    123         printf "FREE: %d\n", $currentblock->front_flag & ~1
    124        else
    125          printf "."
    126        end
    127        set $free = $free + $currentblock->front_flag & ~1
    128        set $numfree = $numfree + 1
    129      end
    130      set $currentblock = (Heap_Block *)((char *)$currentblock + ($currentblock->front_flag&~1))
    131    end
    132    if $arg0 == 0
    133      printf "\n"
    134    end
    135    printf "TOTAL: %d (%d)\tUSED: %d (%d) \tFREE: %d (%d)\n", \
    136      $used + $free, $numused + $numfree, \
    137      $used, $numused, \
    138      $free, $numfree
    139  end
    140 }}}
    141 = How much memory is left in the RTEMS Workspace? =
    142 
    143 
    144 An RTEMS workspace overage can be fairly easily spotted with a debugger. Look at ''Workspace''Area. If first == last, then there is only one free block of memory in the workspace (very likely if no task or message queue deletions). Then do this:
    145 
    146 {{{
    147   (gdb) p (Heap_Block *) Workspace_Area->first
    148   $3 = {back_flag=1, front_flag=68552, next=0x1e260, previous=0x1e25c}
    149 }}}
    150 
    151  * ''Workspace_Area'' is the variable name of the RTEMS Workspace Heap control block.
    152  * ''(Heap_Block *) _Workspace_Area->first'' is the contents of the first heap block information.
    153 
    154 Just as with the C Program Heap, the number was even indicating it is free.  In this case, I had 68552 bytes left in the workspace.
    155 
    156 '''NOTE:''' This is really a crude estimate.  GDB 5.0 and newer support a macro language that provides the features necessary to write a function which would walk a heap structure and print out accurate statistics.  If you write this, submit it. :)
    15742= BSP rtems_initialize_executive_late call dies =
    15843
     
    17156
    17257The initialization task is a real task in RTEMS. The ''rtems_initialize_executive_late'' creates it and switches context to it. This means your BSP environment is another context that RTEMS will switch back to when RTEMS is shut down. This allows your BSP to take control again, then perform any specific functions it needs. A typical operational thing to do is to reboot as embedded targets should not stop.
     58
    17359= GDB Cannot Find My Source Files =
    17460
    17561
    176 If you find the source code paths in your executable as seen by GDB are missing, you may find using an absolute path to invoke the RTEMS configure script may help. When RTEMS libraries get built, nested ''Makefiles'' are executed that '''walk''' through the build directory structure. Therefore, each file is compiled from a certain point in the build directory structure that lies in paralllel to the source directory structure.
     62If you find the source code paths in your executable as seen by GDB are missing, you may find using an absolute path to invoke the RTEMS configure script may help. When RTEMS libraries get built, nested ''Makefiles'' are executed that '''walk''' through the build directory structure. Therefore, each file is compiled from a certain point in the build directory structure that lies in parallel to the source directory structure.
    17763
    178 As a consequece of this, you get a varing number of "dots", depending on how deep the corresponding directory is inside the build tree. There is no common location, from which the source file pointers in the debug info is correct for all object files.
     64As a consequence of this, you get a varying number of "dots", depending on how deep the corresponding directory is inside the build tree. There is no common location, from which the source file pointers in the debug info is correct for all object files.
    17965
    18066If you call the initial <tt>configure</tt> in an absolute way such as :
     
    19177
    19278GDB should find the source.
     79
    19380= Starting With Hello World =
    19481
     
    21198
    21299As you add to your program, you may have to increase the number of objects configured as well.
     100
    213101= Standard IO and File Issues =
    214102
     
    217105
    218106The stdio functions in newlib depend on both initialised and uninitialised data.  If you find they are returning -1, ensure your .bss and .data sections are correctly setup. Check your linkcmds file is creating the correct memory map and that your bsp boot process is copying/zeroing all appropriate sections in ram.  It's also worth double checking that your ram and other hardware is working correctly!
     107
    219108= open, fopen, and socket creation fail =
    220109
     
    227116
    228117Note: Most of what follows has been gleaned from the associated README files; I haven't added much original content, partly due to sloth, but more justifiably because I haven't yet used these tools extensively enough to be able to do any better!  My primary purpose, at this point, in including these is so others, especially "nuBees" are aware of them.
    229 =  RTEMS Monitor  =
    230118
    231119
    232 The RTEMS Monitor is run as a high-priority task, and provides a useful window into the operation of your system.  From the README:
    233 
    234 {{{
    235 monitor task
    236 
    237 The monitor task is an optional task that knows about RTEMS
    238 data structures and can print out information about them.
    239 It is a work-in-progress and needs many more commands, but
    240 is useful now.
    241 
    242 The monitor works best when it is the highest priority task,
    243 so all your other tasks should ideally be at some priority
    244 greater than 1.
    245 
    246 To use the monitor:
    247 -------------------
    248 
    249     #include <rtems/monitor.h>
    250 
    251     ...
    252 
    253     rtems_monitor_init(0);
    254 
    255     The parameter to rtems_monitor_init() tells the monitor whether
    256     to suspend itself on startup.  A value of 0 causes the monitor
    257     to immediately enter command mode; a non-zero value causes the
    258     monitor to suspend itself after creation and wait for explicit
    259     wakeup.
    260 
    261 
    262     rtems_monitor_wakeup();
    263    
    264     wakes up a suspended monitor and causes it to reenter command mode.
    265 
    266 Monitor commands
    267 ----------------
    268 
    269     The monitor prompt is 'rtems> '.
    270     Can abbreviate commands to "uniquity"
    271     There is a 'help' command.  Here is the output from various
    272     help commands:
    273 
    274         Commands (may be abbreviated)
    275 
    276           help      -- get this message or command specific help
    277           task      -- show task information
    278           queue     -- show message queue information
    279           symbol    -- show entries from symbol table
    280           pause     -- pause monitor for a specified number of ticks
    281           fatal     -- invoke a fatal RTEMS error
    282 
    283         task [id [id ...] ]
    284           display information about the specified tasks.
    285           Default is to display information about all tasks on this node
    286 
    287         queue [id [id ... ] ]
    288           display information about the specified message queues
    289           Default is to display information about all queues on this node
    290 
    291         symbol [ symbolname [symbolname ... ] ]
    292           display value associated with specified symbol.
    293           Defaults to displaying all known symbols.
    294 
    295         pause [ticks]
    296           monitor goes to "sleep" for specified ticks (default is 1)
    297           monitor will resume at end of period or if explicitly awakened
    298 
    299         fatal [status]
    300           Invoke 'rtems_fatal_error_occurred' with 'status'
    301           (default is RTEMS_INTERNAL_ERROR)
    302 
    303         continue
    304           put the monitor to sleep waiting for an explicit wakeup from the
    305           program running.
    306 
    307 
    308 Sample output from 'task' command
    309 ---------------------------------
    310 
    311     rtems> task
    312       ID       NAME   PRIO   STAT   MODES  EVENTS   WAITID  WAITARG  NOTES
    313     ------------------------------------------------------------------------
    314     00010001   UI1     2    READY    P:T:nA    NONE15: 0x40606348
    315     00010002   RMON    1    READY    nP    NONE15: 0x40604110
    316 
    317     'RMON' is the monitor itself, so we have 1 "user" task.
    318     Its modes are P:T:nA which translate to:
    319 
    320         preemptable
    321         timesliced
    322         no ASRS
    323 
    324     It has no events.
    325     It has a notepad value for notepad 15 which is 0x40606348
    326     (this is the libc thread state)
    327 }}}
    328 
    329 Note that this README is quite dated - it hasn't been changed in 11 '''years'''!  In fact, it provides much more information; here is the output from the help command on a recent (July 2007 HEAD) session:
    330 
    331 {{{
    332 rtems $ help
    333 config     - Show the system configuration.
    334 itask      - List init tasks for the system
    335 mpci       - Show the MPCI system configuration, if configured.
    336 pause      - Monitor goes to "sleep" for specified ticks (default is 1).
    337              Monitor will resume at end of period or if explicitly
    338              awakened
    339               pause [ticks]
    340 continue   - Put the monitor to sleep waiting for an explicit wakeup from
    341              the program running.
    342 go         - Alias for 'continue'
    343 node       - Specify default node number for commands that take id's.
    344               node [ node number ]
    345 symbol     - Display value associated with specified symbol. Defaults to
    346              displaying all known symbols.
    347               symbol [ symbolname [symbolname ... ] ]
    348 extension  - Display information about specified extensions. Default is to
    349              display information about all extensions on this node.
    350               extension [id [id ...] ]
    351 task       - Display information about the specified tasks. Default is to
    352              display information about all tasks on this node.
    353               task [id [id ...] ]
    354 queue      - Display information about the specified message queues.
    355              Default is to display information about all queues on this
    356              node.
    357               queue [id [id ... ] ]
    358 object     - Display information about specified RTEMS objects. Object id's
    359              must include 'type' information. (which may normally be
    360              defaulted)
    361               object [id [id ...] ]
    362 driver     - Display the RTEMS device driver table.
    363               driver [ major [ major ... ] ]
    364 dname      - Displays information about named drivers.
    365 exit       - Invoke 'rtems_fatal_error_occurred' with 'status' (default is
    366              RTEMS_SUCCESSFUL)
    367               exit [status]
    368 fatal      - 'exit' with fatal error; default error is RTEMS_TASK_EXITTED
    369               fatal [status]
    370 quit       - Alias for 'exit'
    371 help       - Provide information about commands. Default is show basic
    372              command summary.
    373             help [ command [ command ] ]
    374            - 'i"a
    375 rtems $
    376 }}}
    377 =  Capture Engine  =
    378 
    379 
    380 The [wiki:TBR/UserManual/Capture_Engine Capture Engine] is another neat tool.  Unlike the RTEMS Monitor, this one is already documented on the Wiki, though you have to know to search for it.  Check it out!
    381 =  CPU Usage Monitoring  =
    382 
    383 
    384 There is a CPU usage monitoring facility available in cpukit/libmisc/cpuuse.  Again, from the README:
    385 
    386 This directory contains code to report and reset per-task CPU usage.
    387 If the BSP supports nanosecond timestamp granularity, this this information
    388 is very accurate.  Otherwise, it is dependendent on the tick granularity.
    389 
    390 It provides two primary features:
    391 
    392  * Generate a CPU Usage Report
    393  * Reset CPU Usage Information
    394 ==  NOTES  ==
    395 
    396 
    397 #If configured for tick granularity, CPU usage is "docked" by a clock tick at each context switch.
    398 #If configured for nanosecond granularity, no work is done at each clock tick.  All bookkeeping is done as part of a context switch.
    399 =  Stack Checker  =
    400 
    401 ==  Introduction  ==
    402 
    403 
    404 This directory contains a stack bounds checker.  It provides two
    405 primary features:
    406 
    407 #check for stack overflow at each context switch
    408 #provides an educated guess at each task's stack usage
    409 ==  Enabling  ==
    410 
    411 
    412 Add the stack checker extension to the initial user extension set.
    413 If using confdefs.h to build your configuration table, this is
    414 as simple as adding -DSTACK_CHECK_ON to the gcc command line which
    415 compiles the file defining the configuration table.  In the RTEMS
    416 test suites and samples, this is always init.c.  Another way to
    417 enable it is to include the following prior to including confdefs.h:
    418 
    419 {{{
    420 #define STACK_CHECKER_ON
    421 }}}
    422 
    423 Once you've enabled the stack checker when building your application, it the stack checker
    424 runs automatically as part of a context switch.  Additionally, you can call
    425 
    426 {{{
    427 boolean rtems_stack_checker_is_blown(void);
    428 }}}
    429 
    430 at any time to check yourself; it returns FALSE if the stack appears okay,
    431 or TRUE if the stack pointer is out of range or the pattern marker has
    432 been corrupted.
    433 ==  Background  ==
    434 
    435 
    436 The stack overflow check at context switch works by looking for
    437 a 16 byte pattern at the logical end of the stack to be corrupted.
    438 The "guesser" assumes that the entire stack was prefilled with a known
    439 pattern and assumes that the pattern is still in place if the memory
    440 has not been used as a stack.
    441 
    442 Both of these can be fooled by pushing large holes onto the stack
    443 and not writing to them... or (much more unlikely) writing the
    444 magic patterns into memory.
    445 
    446 This code is provided as a tool for RTEMS users to catch the most common mistake in multitasking
    447 systems ... too little stack space.  Suggestions and comments are appreciated.
    448 ==  Optional Compile-time Selections  ==
    449 
    450 If someone ever gets VERY VERY desperate, Joel recently added some conditionals which can turn on walking
    451 the heap and checking the stack EVERY time you enter an RTEMS dispatching disabled critical section.
    452 You have to recompile but since this is such a heavy handed thing to have on, that seemed a fair trade off.
    453 The "heavy stack check" feature is enabled by defining RTEMS_HEAVY_STACK_DEBUG, and the "heavy malloc check"
    454 by defining RTEMS_HEAVY_MALLOC_DEBUG; in both cases, do this before you build and install RTEMS.
    455 ==  NOTES  ==
    456 
    457 
    458 #Stack usage information is questionable on CPUs which push large holes on stack.
    459 #Prior to 4.7.99.2, the stack checker used printf() instead of printk().  This often resulted in a fault when trying to print the helpful diagnostic message.  If using the older printf() stack checker and it comes out, congratulations. If not, then the variable Stack_check_Blown_task contains a pointer to the TCB of the offending task.  This is usually enough to go on.  Now that it is using printk(), it should be able to get messages out.
    460120=  = FUTURE ====
    461121