wiki:Projects/GSoC/TestingReview/Test-dept

Version 6 (modified by Xiaochen Pan, on Jul 8, 2012 at 4:20:13 AM) (diff)

TestingReview/Test?-dept

General Comparison

TestDept? is unit testing framework providing stubing capabilities.

  • TestDept? support the xUnit architecture with stubbing functionality.
  • Stub functions are similar but different concept than mocking functions. It is supported by the replace_function(functions_to_be_replace, stub_function) interface. Mocking functions can deal with the argument expectations and mocked return values for any number of times you want. While stubing functions can only provide a uni-interface. For example, you can decide that a mocking function mock_function() to return 1 at the first time of being callled and return 2 at the second time of being called. When stub technique is used, you cannot expect the same function to return different values unless you provide the inside logic basing on inputs of arguments, which is time-consuming and inconvenient for test automation. And also, because of the uni-interface of the stubbing function, it cannot specify the different expected value for certain argument each time the stubbing function is called.
  • The automatic generator for the test is convenient in certain way, but also increase the chance of fail to get the correct executive binary for cross-compilation.
  • The configure tool provided by test-dept does not have cross-compilation options. In this way, this software is not made for test applications in cross-compilation environment. I failed to compile it correctly and produce the right binary executive. Further effort is needed.
  • In conclusion, stubing functions are not as flexible and useful as mocking functions.

RTEMS based Test Result

Currently, haven't been able to get the tests results. It needs further integration of test-dept framework generator to produce the binary executive correctly under RTEMS. I can compile the tests successfully under RTEMS. But I can't get the correct result of the tests.

Host based Test Results

File:Dept host.png?

The above screenshot displays an example test output for a host-based tests. One of the 17 total tests failed. A dot denotes one test case pass. And a big F denotes one test case failed. The order of tests does not follow the order in the test written though. In the following, it shows which test case has not pass. The failed pass is in test suite assertion_test and the test case is test_function_with_assertions().