wiki:Developer/Coverage/Theory

Version 11 (modified by GlennHumphrey, on Sep 11, 2009 at 8:30:49 PM) (diff)

Continued with explanations

Coverage Analysis Theory

Table of Contents

    Error: Page Developer/Coverage/CoverageAnalysisTheory does not exist

The subject of Code Coverage Analysis is broad and has been written about many times over. This background material is not intented to summerize or rehash what can be read elsewhere. Instead, the focus here will be on the aspects of Code Coverage Analysis as they pertain to the RTEMS Coverage Analysis effort.

The ultimate goal of Code Coverage Analysis is to ensure that a test suite adequately tests a particular body of code. In order to achieve this goal, several different coverage criteria may have to be examined. Let's consider the following criteria:

  • Statement Coverage - Has each line of the source code been executed?
  • Decision Coverage (also known as Branch coverage) - Has each control structure (such as an if statement) evaluated both to true and false?
  • Condition Coverage - Has each boolean sub-expression evaluated both to true and false (this does not necessarily imply decision coverage)?
  • Object Coverage - Has each line of generated assembly been executed?

Statement Coverage

Statement Coverage requires that each line of source code must be executed. This is often considered the simpliest criteria. The problem is that it only identifies the lines that were executed, and does not consider the logic flow of the code. It can be useful for indentifying "chunks" of code (i.e. new functionality) that are not covered by the test suite, but not much else.

Decision Coverage

Decision Coverage requires that each control structure evaluate to both TRUE and FALSE. This is a pretty good criteria because it generally ensures that both the TRUE and FALSE paths of an expression are covered. However, expressions without an else alternative can be a problem because

Condition Coverage

TBD

Object Coverage

Object Coverage requires that each line of generated assembly been executed.

An Example

In order to illustrate what is covered by each of the different criteria, consider the following example showing the source code for a simple if statement along with its generated pseudo-code instructions.

{| border="1" style="margin: 1em auto 1em auto;text-align: left;" |+

|Block | A
Source Code Block Object Pseudo-code
if (x OR y) A1 cmp x, 0

branch if FALSE to do something

|
A2 cmp y, 0

branch if TRUE around do something

| B
do something B do something instructions

|}

Statement Coverage

Statement Coverage requires that each line of source code must be executed. In our example, a single test case that allows the if statement to evaluate to TRUE will achieve 100% Statement Coverage.

Decision Coverage

Decision Coverage requires that each control structure evaluate to both TRUE and FALSE. In our example, two test cases are required to achieve 100% Decision Coverage. One case must allow the if statement to evaluate to TRUE and the other case must allow the if statement to evaluate to FALSE.

Condition Coverage

TBD

Object Coverage

Object Coverage requires that each line of generated assembly been executed. In our example, only a test case where x is FALSE and y is TRUE will achieve 100% Object Coverage.

Criteria Relationships

Imported from old wiki.]]

Each of these criteria can be used independently to analyze the code in question. Application of any one criteria will likely improve the test suite to some degree albeit at some cost. Examination of the criteria collectively, shows that there are clear relationships between the different criteria as shown in the picture. The completness of the test suite increases as it satifies first Statement Coverage and then Decision Coverage and finally MC/DC. If the test suite satisfies Statement Coverage, it will partially satisfy Decision Coverage and MC/DC. If the test suite satisfies Decision Coverage, it will completely satisfy Statement Coverage and partially satisfy MC/DC. There is also a complexity relationship where Statement Coverage is the least complex to satisfy and MC/DC is the most complex to satisfy.

<br style="clear: both" />

Attachments (1)

Download all attachments as: .zip