Changes between Version 6 and Version 7 of GCI/Documentation/CoverageAnalysis/Coverage


Ignore:
Timestamp:
Dec 8, 2018, 3:42:54 PM (7 months ago)
Author:
shashvat jain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GCI/Documentation/CoverageAnalysis/Coverage

    v6 v7  
    33[[TOC(GCI/Documentation/CoverageAnalysis/Coverage , depth=2)]]
    44
    5 = Coverage Analysis Theory =
     5== Coverage Analysis Theory ==
    66
    77The subject of Code Coverage Analysis is broad and has been written about many times over.  This background material is not intended to summarise or rehash what can be read elsewhere.  Instead, the focus here will be on the aspects of Code Coverage Analysis as they pertain to the [wiki:TBR/UserManual/RTEMS_Coverage_Analysis RTEMS Coverage Analysis] effort.
     
    1414 *  '''Object Coverage''' - Has each line of generated assembly been executed?
    1515
    16 == Statement Coverage ==
     16=== Statement Coverage ===
    1717
    1818
     
    2323
    2424Decision Coverage requires that each control structure evaluate to both TRUE and FALSE.  This is a pretty good criteria because it generally ensures that both the TRUE and FALSE paths of an expression are covered.  However, short-circuit operators will prevent some portions of a complex expression from being evaluated.
    25 == Condition Coverage ==
     25=== Condition Coverage ===
    2626
    2727
     
    2929Condition Coverage requires that each boolean sub-expression evaluate to both TRUE and FALSE.  This criteria goes a little further than Decision Coverage by ensuring that the component parts of a compound expression each evaluate to TRUE and FALSE.  But it should be noted that Condition Coverage by itself does not necessarily imply decision coverage.  Because of this fact, it is best to apply Decision Coverage and Condition Coverage together.
    3030
    31 == Object Coverage ==
     31=== Object Coverage ===
    3232
    3333
     
    4141Each of these criteria can be used independently to analyze the code in question.  Application of any one criteria will likely improve the test suite to some degree albeit at the cost of increasing the complexity of the test suite.  Examination of the criteria collectively, shows that there are clear relationships between the different criteria as shown in the picture.  The completeness and complexity of the test suite increases as it satisfies first Statement Coverage and then Decision Coverage and finally !Condition/Decision Coverage.  If the test suite satisfies Statement Coverage, it will partially satisfy Decision Coverage and !Condition/Decision Coverage.  If the test suite satisfies Decision Coverage, it will completely satisfy Statement Coverage and partially satisfy !Condition/Decision Coverage.  Note the fact that Object Coverage satisfies part of all of the other criteria.  There is also a complexity relationship where Statement Coverage is the least complex to satisfy and !Condition/Decision Coverage is the most complex to satisfy.
    4242
    43 = An Example =
     43== An Example ==
    4444
    4545
     
    5151|| B || do something || B || do something instructions ||
    5252
    53 == Statement Coverage ==
     53=== Statement Coverage ===
    5454
    5555
    5656A single test case that allows the if statement to evaluate to TRUE will execute blocks A and B.  This will achieve 100% Statement Coverage.
    5757
    58 == Decision Coverage ==
     58=== Decision Coverage ===
    5959
    6060
    6161A minimum of two test cases are required to achieve 100% Decision Coverage.  One case must force the if statement to evaluate to TRUE and the other case must force the if statement to evaluate to FALSE.  A test case that forces a TRUE outcome will either execute blocks A1 and B or A1, A2 and B.  A test case that forces a FALSE outcome will execute blocks A1 and A2.
    6262
    63 == !Condition/Decision Coverage ==
     63=== !Condition/Decision Coverage ===
    6464
    6565
    6666A minimum of two test cases are required to achieve 100% !Condition/Decision Coverage.  In the first case, x and y must be TRUE.  In the second case, x and y must be FALSE.  The test case that forces a TRUE outcome will execute blocks A1 and B.  The test case that forces a FALSE outcome will execute blocks A1 and A2.
    6767
    68 == Object Coverage ==
     68=== Object Coverage ===
    6969
    7070
     
    8787
    8888
    89 == Running covoar To generate coverage reports ==
     89=== Running covoar To generate coverage reports ===
    9090
    9191To run covoar we need to run covoar from the RTEMS-KERNEL BUILD directory.
     
    9999}}}
    100100
    101 === `covoar` usage : ===
     101==== `covoar` usage : ====
    102102{{{
    103103Usage: covoar [-v] -T TARGET -f FORMAT [-E EXPLANATIONS] -e EXE_EXTENSION -c COVERAGEFILE_EXTENSION EXECUTABLE1 ... EXECUTABLE2
     
    119119}}}
    120120
    121 == Running RTEMS-TESTER for Coverage analysis ==
     121=== Running RTEMS-TESTER for Coverage analysis ===
    122122
    123123RTEMS-TESTER when run with `--coverage` option, generates an html coverage analysis report (report.html)
     
    141141Please visit [https://thelunatic.github.io/rtems_gsoc18/blog/coverage-report/ my development blog] to see examples of coverage report
    142142
    143 == Resources ==
     143=== Resources ===
    144144
    145145TBA
    146146
    147 = References =
     147== References ==
    148148 *  TBD