Changes between Version 22 and Version 23 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
Sep 8, 2009, 11:18:10 PM (10 years ago)
Author:
GlennHumphrey
Comment:

Added more intro material

Legend:

Unmodified
Added
Removed
Modified
  • TBR/UserManual/RTEMS_Coverage_Analysis

    v22 v23  
    1313
    1414Be active and help us meet your application domain requirements while improving the product for all!
    15 = Achieving the Goal =
    16 
    17 
    18 In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term Code Coverage is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.
    1915= Applying Coverage Analysis to RTEMS =
    2016
    2117
    22 For some background information on Coverage Analysis, see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory].
     18In order to achieve the 100% tested goal, it is important to define what constitutes 100% tested.  A lot of information exists about how to completely test a software application.  In general, the term [http://en.wikipedia.org/wiki/Code_coverage Code Coverage] is used to refer to the analysis that is performed to determine what portions of the software are tested by the test suite and what portions are not tested.  For some background information on Code Coverage Analysis, see [wiki:Developer/Coverage/CoverageAnalysisTheory Coverage Analysis Theory].
     19
     20Traditional, Code Coverage Analysis has been performed by instrumenting the source code or object code or by using special hardware to monitor the instructions executed.  An objective of the RTEMS code coverage effort is to use existing tools and to avoid altering the code to be analyzed.  This can be accomplished by using a processor simulator that provides coverage analysis information.  The information can be processed to determine which instructions are executed.  We call this object code coverage.  Initially, we set out to achieve 100% object code coverage of the RTEMS executive.
     21= How it was Done =
     22
    2323
    2424We perform automated coverage testing using a processor simulator in conjunction with a set of RTEMS specific support scripts.
     25= What was Discovered =
     26
    2527
    2628There are multiple ways to measure progress on this task. We primarily use two metrics.  The first is the reduction in the number of uncovered binary code ranges from that identified initially.  The second is the percent of untested binary object code as a percentage of the total code size under analysis.  Together the metrics provide useful information.  Some uncovered ranges may be a single instruction so eliminating that case improves the first metric more than the second.
     29= Beyond Object Code Coverage =
     30
    2731= Statement Coverage =
    2832
     
    129133
    130134 *  pc386
    131 = SPARC =
     135= =SPARC ==
    132136
    133137
     
    137141 *  LEON2
    138142 *  LEON3
    139 = References =
     143=  =References==
    140144
    141 =  ==General Coverage Testing===
     145
     146===General Coverage Testing===
    142147
    143148