Changes between Version 75 and Version 76 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
Sep 21, 2009, 11:54:04 PM (10 years ago)
Author:
GlennHumphrey
Comment:

/* Coverage Profiles */ Added more information about options

Legend:

Unmodified
Added
Removed
Modified
  • TBR/UserManual/RTEMS_Coverage_Analysis

    v75 v76  
    116116Initially, the Baseline group included source code from the cpukit.  Specifically the following cpukit directories were included:  score, sapi, rtems and posix.  This group represents a full tasking and synchronization feature set.  What was not in the Baseline group was placed in the Developmental group.  The Developmental group included: libcsupport, libfs/imfs, libmisc/stackchk, libmisc/cpuuse, libmisc/bspcmdline, libmisc/dmpbuf and libmisc/devnull.
    117117
    118 From the two groups, we recognized the need to use different compiler optimization levels and to analyze each group with POSIX enabled and POSIX disabled.  Applying these options produced eight sub-groups that we called profiles.  The eight profiles are:
     118From the two groups, we recognized the need to use different compiler optimization levels and to analyze each group with POSIX threads enabled and POSIX threads disabled.  Applying these options produced eight sub-groups that we called profiles.  The eight profiles are:
    119119
    120120 *  Baseline (-Os, POSIX Enabled)
     
    127127 *  Developmental (-O2, POSIX Disabled)
    128128
    129 Initially, the code to be analyzed should be compiled with optimization level -Os.  This optimizes for size without making the object code too difficult to follow.  Following the object code can be important when trying to determime how to resolve the uncovered code.  Once the analyzed code approaches 100% covered, it is desirable to change the optimization level to -O2.  This is the most often used optimization level.
    130 
    131 The POSIX interface represents a significant difference in functionality within RTEMS.  When POSIX is enabled ***ADD MORE HERE***.  When POSIX is disabled, RTEMS is configured for the classic API only.
    132 
    133129Over time it is desirable to migrate code from the Developmental group to the Baseline.  As support libraries in cpukit become nearly 100% covered, they will be move from the Developmental group to the Baseline group.  Eventually, the Baseline group should contain all of the RTEMS code and the Developmental group should contain nothing.
    134130= Compilation and Configuration Options =
    135131
    136132
    137 Discuss impact of -O2 versus -Os with example from code.
     133The compilation and configuration options that effect covmerge are set in the configuration.txt file.  More information about the configuration.txt file can be found in XXX.  When we started the RTEMS Code Coverage effort, the code analyzed was compiled with optimization level -Os.  This optimizes for size without making the object code too difficult to follow.  Following the object code is important when trying to determime how to resolve the uncovered code.  Once the analyzed code approaches 100% covered, it is desirable to change the optimization level to -O2.  This is the most often used optimization level.
     134
     135Enabling or disabling POSIX allows us to analyze the RTEMS code in its two most commonly use threading configurations.  When POSIX is enabled, RTEMS is configured to use POSIX threads and the POSIX tests are built and executed as part of the test suite.  When POSIX is disabled, RTEMS is configured to use Classic RTEMS threads and the POSIX tests are not included in the test suite.
     136= Internal Compilation and Configuration Options =
     137
     138
     139There are several compilation and configuration options that are built into the covmerge script.
     140We want the coverage build to match the default build for RTEMS.  Current we're using the following configuration variables which alter the build
    138141
    139142Inlining _Thread_Dispatch_enable, etc.
    140 = Classic API Only (POSIX Disabled) =
    141 
    142 
    143 In this profile, we disable POSIX and focus on the contents of the score, sapi, and rtems directories in the cpukit directory.  The POSIX API and tests are disabled.  In this profile, we expect to identify:
    144 
    145  *  features in score only exercised by POSIX
    146  *  features in score available via Classic API but only tested via POSIX
    147  *  POSIX features like sleep() which are enabled when POSIX threads are disabled.
    148 
    149 The first case will allow us to disable score features in this configuration and reduce the code size.
    150 
    151 The second case allows us to approach 100% coverage in every RTEMS configuration.
    152 
    153 The third case is similar to the second and indicates the need for tests in this configuration for features that are technically part of the POSIX API support.
    154 = Developmental =
    155 
    156 
    157 This is an experimental/developmental coverage configuration and adds almost all of the CPUKit contents that are non-networked.  It nearly doubles the size of the code being covered.  We are aiming for the entire contents of libcsupport, libmisc, and various filesystems.  This is a large body of code and components like Termios and the file systems will require creativity to get automated coverage near 100%.
    158 
    159 We have done initial tests on this profile. There is work to be done improving the test coverage.  As components are covered 100%, they will be moved from experimental/developmental status to be included in the official coverage run.
    160 
    161 We welcome your contributions.
     143NDEBUG=1 - disable asserts, always keep
     144RTEMS_DO_NOT_INLINE_THREAD_ENABLE_DISPATCH=1 - resulted in branch explosion, 200+ new test cases needed to eliminate
     145RTEMS_DO_NOT_INLINE_CORE_MUTEX_SEIZE=1 - resulted in very difficult code to analyze, should be easy to eliminate
     146RTEMS_DO_NOT_UNROLL_THREADQ_ENQUEUE_PRIORITY=1 - unrolling loop results in multiple interrupt critical section, should be solvable
    162147= Beyond Object Code Coverage =
    163148