Changes between Version 100 and Version 101 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
12/31/15 20:53:44 (8 years ago)
Author:
Ralph Holmes
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • TBR/UserManual/RTEMS_Coverage_Analysis

    v100 v101  
    1616
    1717
    18 In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term [http://en.wikipedia.org/wiki/Code_coverage Code Coverage] is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.  It should be noted that Code Coverage does not prove correctness only that all code has been tested.  For some background information on Code Coverage Analysis, see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory].
     18In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term [http://en.wikipedia.org/wiki/Code_coverage Code Coverage] is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.  It should be noted that Code Coverage does not prove correctness, only that all code has been tested.  For some background information on Code Coverage Analysis, see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory].
    1919
    2020Traditionally, Code Coverage Analysis has been performed by instrumenting the source code or object code or by using special hardware to monitor the instructions executed.  The guidelines for the RTEMS code coverage effort were to use existing tools and to avoid altering the code to be analyzed.  This was accomplished by using a processor simulator that provides coverage analysis information.  The information was processed to determine which instructions are executed.  We called this Object Code Coverage and we defined 100% tested to be 100% Object Code Coverage.
    … …  
    5555When you run one, be sure about the following:
    5656
    57  *  The RTEMS toolset must be at the head of your PATH just like a normal build
    58  *  Add ''rtems-testing/bin'' to your PATH
     57 *  The RTEMS toolset must be at the head of your PATH just like a normal build.
     58 *  Add ''rtems-testing/bin'' to your PATH.
    5959
    6060The output produced by ''covoar'' is actually a set of HTML and simple ASCII files that give a developer the necessary information to quickly determine the current status of the Code Coverage and enough information to determine the location of the uncovered code. See
    … …  
    174174The compilation level and POSIX configuration options are passed as command line arguments to the RTEMS Coverage Scripts.  [wiki:Developer/Coverage/HowTo RTEMS Code Coverage How To] provides details concerning how to run the RTEMS Coverage Scripts.  When we started the RTEMS Code Coverage effort, the code analyzed was compiled with optimization level -Os.  This optimizes for size without making the object code too difficult to follow.  Following the object code is important when trying to determime how to resolve the uncovered code.  Once the analyzed code approaches 100% covered, it is desirable to change the optimization level to -O2.  This is the most often used optimization level.
    175175
    176 Enabling or disabling POSIX allows us to analyze the RTEMS code in its two most commonly use threading configurations.  When POSIX is enabled, RTEMS is configured to use POSIX threads and the POSIX tests are built and executed as part of the test suite.  When POSIX is disabled, RTEMS is configured to use Classic RTEMS threads and the POSIX tests are not included in the test suite.
     176Enabling or disabling POSIX allows us to analyze the RTEMS code in its two most commonly used threading configurations.  When POSIX is enabled, RTEMS is configured to use POSIX threads and the POSIX tests are built and executed as part of the test suite.  When POSIX is disabled, RTEMS is configured to use Classic RTEMS threads and the POSIX tests are not included in the test suite.
    177177= Internal Compilation and Configuration Options =
    178178