= Debugging =
= Symbolic Debug Information =
RTEMS and the RTEMS tool chains support symbolic debugging. The application, RTEMS and any libraries you wish to debug need to be compiled with the '''gcc''' option '''-g'''. This option causes the compiler to emit debugging information, while the code generated should not change. The '''GNU gcc''' debugging options can be found here -
http://gcc.gnu.org/onlinedocs/gcc-3.4.3/gcc/Debugging-Options.html#Debugging-Options
Using the '''-g''' option results in larger object and executable file sizes, and for C++ this can be quite large. For example a M68K C++ application can have a ELF executable file size of 19M bytes yet the code size is only 1.6M bytes. The actual memory foot print can be seen by using the '''size''' tool on an object file or the final executable -
{{{
$ m68k-rtems-size myfile.o
}}}
For RTEMS you typically do not need to strip the executable. The loading of the executable into the target memory or conversion to S records, Intel Hex, for binary format will automatically strip the debug information. Keeping the excutable with the debugging information is recommended. If a problem appears with code in the field and you recieve a some sort of dump or trace you can use the '''objdump''' tool to help locate the problem -
{{{
$ m68k-rtems-objdump -D --line-numbers --source --demangle myapp.elf | less
}}}
= Debugging a BSP =
To test a BSP you need an application. The samples and tests provide a proven set of applications that allow you to test your BSP. Select a sample application such as the '''Hello World''' to try first, then move to an application that uses more interrupt sources.
The following is a list of debugging tools and setups that we recommended. Time spent on this will be rewarded further into the project:
* Try to create a working debugger environment for the target hardware. The GNU debugger, GDB, supports a number of different ways to connect to a target. The interface can be a simple serial port using a [wiki:Debugging/GdbSerialMonitor GdbSerialMonitor], or an expensive TCP/IP type hardware probe.
* If your target processor supports a simulator it is recommended you learn how to use it. It will give you a stable debugging enviroment.
* Implement a ''printk'' interface in your BSP. For target hardware that does not have a video interface the ''printk'' outputs to a serial port. The driver is normally a simple polled UART driver.
* Use the [wiki:TBR/UserManual/Capture_Engine Capture Engine] to aid the debugging and verfication of your real-time design.
= GDB and RTEMS =
Currently GDB is not RTEMS aware. GDB scripts exist that can help by providing presenting kernel structures in a user friendly manner. These can be found here: [wiki:Debugging/GDBScripts GdbScripts].
The issue with making GDB aware of RTEMS is where the knowledge of the kernel structures is located. A version of GDB that contains a specific kernel structure layout will break if RTEMS changes. The ideal would be for GDB to find the structure elements using the applications debug information.
= How much memory is left in the C Program Heap? =
The C heap is a region so this should work:
{{{
(gdb) p ((Region_Control *)_Region_Information->local_table[1])->Memory->first
$9 {back_flag=1, front_flag=8058280, next=0x7ea5b4, previous=0x7ea5b0}
}}}
Let's look at that gdb command in more detail.
* ''_Region_Information'' contains the information used to manage Region objects.
* ''_Region_Information->local_table'' is the object pointer table for Regions. It is
indexed by the ''object index'' portion of the object ID.
* ''Region_Information->local_table'' points to the first Region object. It is of type ''(Region_Control *)''.
* ''((Region_Control *)_Region_Information->local_table[1])->Memory'' points to the Heap control portion of this Region's control block.
* ''((Region_Control *)_Region_Information->local_table[1])->Memory->first'' references the contents of the first ''heap block'' on this Heap.
Notice that the ''front_flag'' is displayed as ''8058280''. This is in decimal since we used ''p'' not ''p/x'' to gdb. Since this number is even, we know the in use bit is 0 and the block is free. Thus the first block on the heap is 8,058,280 bytes and there are at least that many bytes left.
'''NOTE:''' This is really a crude estimate.
If you have compiled RTEMS libraries with -DRTEMS_DEBUG, malloc will maintain statistics. From an RTEMS application:
{{{
#include
.....
malloc_dump ();
}}}
will print malloc's heap statistics to ''stdout''. Example usage can be found in c/src/tests/frye/libcleak and c/src/tests/libtests/malloctest.
The following GDB macro may also be of use. If provided with an argument of '''0''' for a summary, or '''1''' to explicitly list the regions.
{{{
define rtems-mallocheap-walk
printf "walking the heap:\n"
set $heapstart = ((Region_Control *)_Region_Information->local_table[RTEMS_Malloc_Heap&0xffff])->Memory->start
set $currentblock = $heapstart
set $used = 0
set $numused = 0
set $free = 0
set $numfree = 0
while $currentblock->front_flag != 1
if $currentblock->front_flag & 1
if $arg0 != 0
printf "USED: %d\n", $currentblock->front_flag & ~1
else
printf "*"
end
set $used = $used + $currentblock->front_flag & ~1
set $numused = $numused + 1
else
if $arg0 != 0
printf "FREE: %d\n", $currentblock->front_flag & ~1
else
printf "."
end
set $free = $free + $currentblock->front_flag & ~1
set $numfree = $numfree + 1
end
set $currentblock = (Heap_Block *)((char *)$currentblock + ($currentblock->front_flag&~1))
end
if $arg0 == 0
printf "\n"
end
printf "TOTAL: %d (%d)\tUSED: %d (%d) \tFREE: %d (%d)\n", \
$used + $free, $numused + $numfree, \
$used, $numused, \
$free, $numfree
end
}}}
= How much memory is left in the RTEMS Workspace? =
An RTEMS workspace overage can be fairly easily spotted with a debugger. Look at ''Workspace''Area. If first == last, then there is only one free block of memory in the workspace (very likely if no task or message queue deletions). Then do this:
{{{
(gdb) p (Heap_Block *) Workspace_Area->first
$3 = {back_flag=1, front_flag=68552, next=0x1e260, previous=0x1e25c}
}}}
* ''Workspace_Area'' is the variable name of the RTEMS Workspace Heap control block.
* ''(Heap_Block *) _Workspace_Area->first'' is the contents of the first heap block information.
Just as with the C Program Heap, the number was even indicating it is free. In this case, I had 68552 bytes left in the workspace.
'''NOTE:''' This is really a crude estimate. GDB 5.0 and newer support a macro language that provides the features necessary to write a function which would walk a heap structure and print out accurate statistics. If you write this, submit it. :)
= BSP rtems_initialize_executive_late call dies =
Your target is booting and the BSP is initialising but a call to ''rtems_initialize_executive_late'' results in an exception, or the target locking up. This can be due to a few reasons that you will need to work through.
* The memory map is not correct. RTEMS use the [wiki:WorkSpace WorkSpace] to create the initialization task, it's stack, the interrupt stack, plus more. If the [wiki:WorkSpace WorkSpace] is wrong the target will die. The exact way depends on the specific processor and mapping error. If you have implemented ''printk'' it may be a good idea to add some code to ''bsp_pretasking_hook'' to show you the base and size:
{{{
printk ("Heap : %5d KB @ 0x%08x\n ", heap_size >> 10, heap_start);
}}}
* The initialization process encountered an error. The set of these is documented in the '''Classic API C User's Guide'''. For RTEMS 4.6.1, this was documented in the [http://www.rtems.com/onlinedocs/releases/rtemsdocs-4.6.1/share/rtems/html/c_user/c_user00034.html Initialization Manager Failure] section.
* Interrupts are pending and no interrupt handler is present, or a bug exists in the interrupt handler, or vector table layout. RTEMS will enable interrupts when it switches to the initialization task ''rtems_initialize_executive_late'' creates. This problem can be found by getting to the context switch call, ''_CPU_Context_Switch'', then the crash or lockup. The fix is to make sure the BSP has masked all sources of interrupts in the hardware around the processor. This allows RTEMS and the application an ordered and controlled initialization of driver and therefore interrupts.
The initialization task is a real task in RTEMS. The ''rtems_initialize_executive_late'' creates it and switches context to it. This means your BSP environment is another context that RTEMS will switch back to when RTEMS is shut down. This allows your BSP to take control again, then perform any specific functions it needs. A typical operational thing to do is to reboot as embedded targets should not stop.
= GDB Cannot Find My Source Files =
If you find the source code paths in your executable as seen by GDB are missing, you may find using an absolute path to invoke the RTEMS configure script may help. When RTEMS libraries get built, nested ''Makefiles'' are executed that '''walk''' through the build directory structure. Therefore, each file is compiled from a certain point in the build directory structure that lies in paralllel to the source directory structure.
As a consequece of this, you get a varing number of "dots", depending on how deep the corresponding directory is inside the build tree. There is no common location, from which the source file pointers in the debug info is correct for all object files.
If you call the initial configure in an absolute way such as :
/usr/local/src/rtems/tools/rtems-4.6.0/configure
rather that a relative way :
../rtems-4.6.0/configure
GDB should find the source.
= Newlib's Stdio Functions return -1/EOF =
The stdio functions in newlib depend on both initialised and uninitialised data. If you find they are returning -1, ensure your .bss and .data sections are correctly setup. Check your linkcmds file is creating the correct memory map and that your bsp boot process is copying/zeroing all appropriate sections in ram. It's also worth double checking that your ram and other hardware is working correctly!