Changes between Version 57 and Version 58 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
Sep 18, 2009, 1:16:25 AM (10 years ago)
Author:
GlennHumphrey
Comment:

Added data to show code coverage progress

Legend:

Unmodified
Added
Removed
Modified
  • TBR/UserManual/RTEMS_Coverage_Analysis

    v57 v58  
    9090As mentioned above, the ''covmerge'' script produces reports that contain several metrics that can be used to measure progress.  The first is the number of uncovered object code ranges.  The second is the percent of untested object code as a percentage of the total object code size under analysis.  Together the metrics provide useful information about the status or progress of the Object Code Coverage.
    9191
    92 When we started the RTEMS Code Coverage effort, we did not immediately capture results to measure progress.  This actually ended up being the correct thing to do since the covmerge tool was in development and often produced results that were not directly comparable.  Now that the development of covmerge has largely settled, we performed coverage runs on several RTEMS releases and the current development head.  The results are shown below.
    93 
    94 INSERT TABLE
     92When we started the RTEMS Code Coverage effort, we did not immediately capture results to measure progress.  This actually ended up being the correct thing to do since the ''covmerge'' tool was in development and often produced results that were not directly comparable.  Now that the development of ''covmerge'' has largely settled, we can perform coverage runs on several RTEMS release points and see the progress of the coverage effort.  The results shown below were of the POSIX Enabled Profile run on the SPARC/ERC32.
     93
     94{| border="1" style="margin: 1em auto 1em auto;text-align: left;"
     95|+
     96|-
     97|'''Release''' || '''Covered %''' || '''Uncovered Ranges''' || '''Uncovered Bytes''' || '''Total Bytes'''
     98|-
     99| 4.7 || 77.51 || 454 || 17508 || 77840
     100|-
     101| 4.8 || 76.37 || 538 || 21772 || 92140
     102|-
     103| 4.9 || 96.41 || 167 || 2532 || 70564
     104|-
     105| 4.10 (head 09/09/2009) || 100 || 0 || 0 || 70480
     106|}
     107
     108Several interesting facts can be seen from the data in the table.  There was no organized effort to perform coverage analysis prior to the 4.8 release.  This is evident in that there was no measurable improvement in coverage between 4.7 and 4.8.  The unassisted developer is just not going to recognize the need for more test cases in the test suite.  The coverage analysis began prior to the 4.9 release.  Not surprising, the progress was significant between 4.8 and 4.9.  At that time we addressed large uncovered ranges by doing simple things like adding test cases to the test suite and disabling code that was not used by the chosen configuration.
    95109= Coverage Profiles =
    96110