Notice: We have migrated to GitLab launching 2024-05-01 see here: https://gitlab.rtems.org/

Changes between Version 103 and Version 104 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
11/13/18 19:13:29 (6 years ago)
Author:
Marçal Comajoan Cara
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • TBR/UserManual/RTEMS_Coverage_Analysis

    v103 v104  
    1414
    1515
    16 In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term [http://en.wikipedia.org/wiki/Code_coverage Code Coverage] is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.  It should be noted that Code Coverage does not prove correctness, only that all code has been tested.  For some background information on Code Coverage Analysis, see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory].
     16In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term [http://en.wikipedia.org/wiki/Code_coverage Code Coverage] is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.  It should be noted that Code Coverage does not prove correctness, only that all code has been tested.  For some background information on Code Coverage Analysis, see [wiki:Developer/Coverage/Theory Coverage Analysis Theory].
    1717
    1818Traditionally, Code Coverage Analysis has been performed by instrumenting the source code or object code or by using special hardware to monitor the instructions executed.  The guidelines for the RTEMS code coverage effort were to use existing tools and to avoid altering the code to be analyzed.  This was accomplished by using a processor simulator that provides coverage analysis information.  The information was processed to determine which instructions are executed.  We called this Object Code Coverage and we defined 100% tested to be 100% Object Code Coverage.
    … …  
    7676|| sizes.txt || Provides a list identifying the file name and source line number of each uncovered range along with its size in bytes ||
    7777|| report.txt || Details the settings for the coverage run ||
    78 || Explanations.txt.NotFound || Contains the Explanations that were not found for this coverage run (see [wiki:Developer/Coverage/HowTo RTEMS Code Coverage How To] for more information about how and why to use Explanations) ||
     78|| Explanations.txt.!NotFound || Contains the Explanations that were not found for this coverage run (see [wiki:Developer/Coverage/HowTo RTEMS Code Coverage How To] for more information about how and why to use Explanations) ||
    7979|| annotated.dmp || Provides the disassembled listing of hello.exe with indications of the object code that was not executed ||
    8080|| hello.num || The symbol table of hello.exe ||
    … …  
    104104 *  A new test case is needed.
    105105
    106  *  Code unreachable in selected RTEMS configuration.  For example, the SuperCore could have a feature only exercised by a POSIX API object.  It should be disabled when POSIX is not configured.
     106 *  Code unreachable in selected RTEMS configuration.  For example, the !SuperCore could have a feature only exercised by a POSIX API object.  It should be disabled when POSIX is not configured.
    107107
    108108 *  Debug or sanity checking code which should be placed inside an RTEMS_DEBUG conditional.
    … …  
    166166
    167167
    168 At this point, the RTEMS Code Coverage effort has been focused on Object Code Coverage.  But we would like to go beyond Object Code Coverage and address other traditional coverage criteria (see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory]).  We would also like to remain true to our original guidelines of using existing tools and performing the analysis without modifying the code to analyze.
     168At this point, the RTEMS Code Coverage effort has been focused on Object Code Coverage.  But we would like to go beyond Object Code Coverage and address other traditional coverage criteria (see [wiki:Developer/Coverage/Theory Coverage Analysis Theory]).  We would also like to remain true to our original guidelines of using existing tools and performing the analysis without modifying the code to analyze.
    169169= Achieving Statement Coverage =
    170170
    … …  
    177177
    178178Adding a statement coverage report to ''covoar'' is an open project.
    179 = Achieving Condition/Decision Coverage =
    180 
    181 
    182 Achieving Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  Currently QEMU and tsim can be used to gather this information.
     179= Achieving !Condition/Decision Coverage =
     180
     181
     182Achieving !Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  Currently QEMU and tsim can be used to gather this information.
    183183
    184184tsim produces bitmaps indicating instruction executed, branch taken, and branch not taken.