Changes between Version 94 and Version 95 of TBR/UserManual/RTEMS_Coverage_Analysis


Ignore:
Timestamp:
Jul 4, 2011, 8:34:19 PM (8 years ago)
Author:
JoelSherrill
Comment:

/* Achieving Condition/Decision? Coverage */ Augment

Legend:

Unmodified
Added
Removed
Modified
  • TBR/UserManual/RTEMS_Coverage_Analysis

    v94 v95  
    201201
    202202
    203 Achieving Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  The group responsible for the QEMU processor simulator is actively working to support coverage efforts.  QEMU produces a trace of the instructions executed when an executable is run.  The trace information could be analyzed to identify branch instructions and to determine whether the branch was taken and/or not taken.
    204 
    205 This is another open project at this point.
     203Achieving Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  Currently QEMU and tsim can be used to gather this information.
     204
     205tsim produces bitmaps indicating instruction executed, branch taken, and branch not taken.
     206
     207All versions of QEMU produce a debug log of the instructions executed when an executable is run.  The trace information is analysed to identify branch instructions and to determine whether the branch was taken and/or not taken.  Some versions of QEMU may also be able to produce a trace log which is denser but contains the same information.
     208
     209skyeye does not produce branch taken/not taken information.
     210
     211''covoar'' produces reports on which branch instructions are taken and not taken.  Our goal is to ensure that each branch instruction is taken and not taken. 
     212
     213GCC does not include debug information which indicates that a sequence of compare and branch instructions are part of a single logical condition.  This hinders our ability to augment ''covoar'' to make direct claims regarding Decision Coverage (DC) and Modified condition/decision coverage (MC/DC).
     214
     215We believe that for single condition ''if'' statements such as ''if (cond) action'' or ''if (cond) action1 else action2'', that we are achieving full DC and MC/DC coverage because all logical paths are exercised.
     216
     217Similarly given a dual OR condition ''if'' statement (in C) such as one the following:
     218
     219{{{
     220Case OR1: if (cond1 or cond2)
     221            action
     222Case OE2: if (cond1 or cond2)
     223            action1
     224          else
     225            action2
     226}}}
     227
     228We aim for the following cases given our branch coverage requirements:
     229
     230 *  cond1 branch taken, cond2 short-circuited
     231 *  cond1 branch not taken, cond2 taken
     232 *  cond1 branch not taken, cond2 not taken
     233
     234As the above set of test cases represent the entire set of possible execution paths, we have achieved DC and MC/DC level coverage.
     235
     236{{{
     237Case AND1: if (cond1 and cond2)
     238             action
     239Case AND2: if (cond1 and cond2)
     240             action1
     241           else
     242             action2
     243}}}
     244
     245We aim for the following cases given our branch coverage requirements:
     246
     247 *  cond1 branch taken, cond2 taken
     248 *  cond1 branch taken, cond2 not taken
     249 *  cond1 branch not taken, cond2 short-circuited
     250
     251Again, the above set of test cases represent the entire set of possible execution paths, we have achieved DC and MC/DC level coverage.
     252
     253Open projects in this area include:
     254
     255 *  proving our branch coverage testing policy meets decision coverage (DC) requirements in a more general sense.
     256 *  extending GCC to provide the debug information required to let covoar evaluate DC and MC/DC in C programs.
     257  *  IDEA: If GCC reliably reports that all conditions within a single ''if condition'' have the same line number, then we can use that information as the basis for the analysis.  Did we execute the proper set of cases for all branch instructions associated with a single debug line number.
    206258= Current Status =
    207259