Changes between Version 60 and Version 61 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
Sep 18, 2009, 7:24:09 PM (10 years ago)
Author:
GlennHumphrey
Comment:

/* Measuring Progress */ Added a reference to the periodic coverage run results

Legend:

Unmodified
Added
Removed
Modified
  • TBR/UserManual/RTEMS_Coverage_Analysis

    v60 v61  
    8888
    8989
    90 As mentioned above, the ''covmerge'' script produces reports that contain several metrics that can be used to measure progress.  The first is the number of uncovered object code ranges.  The second is the percent of untested object code as a percentage of the total object code size under analysis.  Together the metrics provide useful information about the status or progress of the Object Code Coverage.
     90As mentioned above, the ''covmerge'' script produces reports that contain several metrics that can be used to measure progress.  The first is the number of uncovered object code ranges.  The second is the percent of untested object code as a percentage of the total object code size under analysis.  Together these metrics provide useful information about the status or progress of the Object Code Coverage.
    9191
    9292When we started the RTEMS Code Coverage effort, we did not immediately capture results to measure progress.  This actually ended up being the correct thing to do since the ''covmerge'' tool was in development and often produced results that were not directly comparable.  Now that the development of ''covmerge'' has largely settled, we can perform coverage runs on several RTEMS release points and see the progress of the coverage effort.  The results shown below were of the POSIX Enabled Profile run on the SPARC/ERC32.
     
    106106|}
    107107
    108 Several interesting facts can be seen from the data in the table.  There was no organized effort to perform coverage analysis prior to the 4.8 release.  This is evident in that there was no measurable improvement in coverage between 4.7 and 4.8.  The unassisted developer is just not going to recognize the need for more test cases in the test suite.  The coverage analysis began prior to the 4.9 release.  Not surprising, the progress was significant between 4.8 and 4.9.  At that time we addressed large uncovered ranges by doing simple things like adding test cases to the test suite and disabling code that was not used by the chosen configuration.
     108Several interesting facts can be seen from the data in the table.  There was no organized effort to perform coverage analysis prior to the 4.8 release.  This is evident in that there was no measurable improvement in coverage between 4.7 and 4.8.  The unassisted developer is just not going to recognize the need for more test cases in the test suite.  The coverage analysis began prior to the 4.9 release.  Not surprising, the progress was significant between 4.8 and 4.9.  At that time we addressed large uncovered ranges by doing simple things like adding test cases to the test suite and disabling code that was not used by the chosen configuration.  The last 3.5% of uncovered code was much harder to address, but the development head has now achieved 100% coverage.
     109
     110Now that we have achieved 100% Code Coverage using the POSIX Enabled Profile, we would like to keep it 100% covered.  We have setup a periodic run of the coverage analysis against the development head.  The results are captured (http://rtems/ftp/pub/rtems/people/joel/coverage/) and can be monitored to ensure that future modifications to the analyzed code base do not produce uncovered code.
    109111= Coverage Profiles =
    110112