Changes between Version 94 and Version 95 of TBR/UserManual/RTEMS_Coverage_Analysis

Timestamp:

07/04/11 20:34:19 (13 years ago)

Author:

JoelSherrill

Comment:

/* Achieving Condition/Decision? Coverage */ Augment

TBR/UserManual/RTEMS_Coverage_Analysis

@@ -201 +201 @@
 
 
-Achieving Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  The group responsible for the QEMU processor simulator is actively working to support coverage efforts.  QEMU produces a trace of the instructions executed when an executable is run.  The trace information could be analyzed to identify branch instructions and to determine whether the branch was taken and/or not taken.
-
-This is another open project at this point.
+Achieving Condition/Decision Coverage requires knowing whether each branch has been both taken and not taken.  Currently QEMU and tsim can be used to gather this information.
+
+tsim produces bitmaps indicating instruction executed, branch taken, and branch not taken.
+
+All versions of QEMU produce a debug log of the instructions executed when an executable is run.  The trace information is analysed to identify branch instructions and to determine whether the branch was taken and/or not taken.  Some versions of QEMU may also be able to produce a trace log which is denser but contains the same information.
+
+skyeye does not produce branch taken/not taken information.
+
+''covoar'' produces reports on which branch instructions are taken and not taken.  Our goal is to ensure that each branch instruction is taken and not taken.  
+
+GCC does not include debug information which indicates that a sequence of compare and branch instructions are part of a single logical condition.  This hinders our ability to augment ''covoar'' to make direct claims regarding Decision Coverage (DC) and Modified condition/decision coverage (MC/DC).
+
+We believe that for single condition ''if'' statements such as ''if (cond) action'' or ''if (cond) action1 else action2'', that we are achieving full DC and MC/DC coverage because all logical paths are exercised.
+
+Similarly given a dual OR condition ''if'' statement (in C) such as one the following:
+
+{{{
+Case OR1: if (cond1 or cond2)
+            action
+Case OE2: if (cond1 or cond2)
+            action1
+          else
+            action2
+}}}
+
+We aim for the following cases given our branch coverage requirements:
+
+ *  cond1 branch taken, cond2 short-circuited
+ *  cond1 branch not taken, cond2 taken
+ *  cond1 branch not taken, cond2 not taken
+
+As the above set of test cases represent the entire set of possible execution paths, we have achieved DC and MC/DC level coverage.
+
+{{{
+Case AND1: if (cond1 and cond2)
+             action
+Case AND2: if (cond1 and cond2)
+             action1
+           else
+             action2
+}}}
+
+We aim for the following cases given our branch coverage requirements:
+
+ *  cond1 branch taken, cond2 taken
+ *  cond1 branch taken, cond2 not taken
+ *  cond1 branch not taken, cond2 short-circuited
+
+Again, the above set of test cases represent the entire set of possible execution paths, we have achieved DC and MC/DC level coverage.
+
+Open projects in this area include:
+
+ *  proving our branch coverage testing policy meets decision coverage (DC) requirements in a more general sense.
+ *  extending GCC to provide the debug information required to let covoar evaluate DC and MC/DC in C programs.
+  *  IDEA: If GCC reliably reports that all conditions within a single ''if condition'' have the same line number, then we can use that information as the basis for the analysis.  Did we execute the proper set of cases for all branch instructions associated with a single debug line number.
 = Current Status =