wiki:GSoC/2013/Testing

Version 6 (modified by Chris Johns, on 12/02/14 at 10:41:05) (diff)

--

RTEMSTestingSOCIS2013

Project Outline

This document outlines the process of integrating a unit testing tool called Unity with a new RTEMS test suite currently under development. This project was carried out during the ESA Summer of Code 2013.

Unity

  • unity.c: implementation
  • unity.h: function and macro definitions
  • unity internals.h: internal definitions

Integration

Unity has been modified to work with the new RTEMS testsuite. All developed code can be found here. It will soon be merged to Unity.

JSON Test Output

An option has been added to output the test results in JSON format.

Test result with normal output:

main.c:39:test_rtems_tick_announce:IGNORE: TODO main.c:44:test_rtems_clock_set:PASS main.c:57:test_rtems_clock_get:PASS main.c:113:test_rtems_clock_maintain:IGNORE: TODO ----------------------- 4 Tests 0 Failures 2 Ignored OK

Test result with JSON output:

{ "tests": [

{

"test_rtems_tick_announce": {

"linenumber": 39, "file": "main.c", "result": "IGNORE", "message": "TODO"

}

}, {

"test_rtems_clock_set": {

"linenumber": 44, "file": "main.c", "result": "PASS"

}

}, {

"test_rtems_clock_get": {

"linenumber": 57, "file": "main.c", "result": "PASS"

}

}, {

"test_rtems_clock_maintain": {

"linenumber": 113, "file": "main.c", "result": "IGNORE", "message": "TODO"

}

}, {}

], "summary": {

"tests": 4, "failed": 0, "ignored": 2 }

}

Source

  • unity.c: testsuite/librtemstest/unity.c
  • unity.h: testsuite/include/unity.h
  • unity_internals.h: testsuite/include/unity_internals.h

Testsuite Organisation

Unity

  • include - Headers
  • librtemstest - Unity and utilities

Test Directories

  • general - General Areas
  • deviceio - Device IO
  • filesystems - File Systems
  • memory - Memory Management
  • network - Network
  • tasks - Tasks Management
  • timing - Timing Tests

Example Test

/*

  • COPYRIGHT (c) 1989-2011.
  • On-Line Applications Research Corporation (OAR). *
  • The license and distribution terms for this file may be
  • found in the file LICENSE in this distribution or at
  • http://www.rtems.com/license/LICENSE. */

/* RTEMS Clock Tests

  • based on sp1, spclockget */

#ifdef HAVE_CONFIG_H #include "config.h" #endif

#include <rtems/testsuite.h> #include <rtems/tmacros.h> #include <rtems/pritime.h>

void setUp(void) { }

void tearDown(void) { }

/* Announce a tick */ void test_rtems_tick_announce(void) {

TEST_IGNORE_MESSAGE( "TODO" );

}

/* Set clock */ void test_rtems_clock_set(){

rtems_status_code status; rtems_time_of_day time; rtems_id id;

build_time( &time, 12, 31, 1988, 9, 0, 0, 0 ); status = rtems_clock_set( &time ); TEST_ASSERT_EQUAL( status, RTEMS_SUCCESSFUL );

}

/* Get clock */ void test_rtems_clock_get() {

rtems_status_code status; rtems_time_of_day time; rtems_interval interval; struct timeval timev;

/* NULL parameter */ status = rtems_clock_get( RTEMS_CLOCK_GET_TICKS_SINCE_BOOT, NULL ); TEST_ASSERT_EQUAL_MESSAGE( status, RTEMS_INVALID_ADDRESS, "null pointer" );

/* ... */

}

/* Maintain clock */ void test_rtems_clock_maintain(void) {

TEST_IGNORE_MESSAGE( "TODO" );

}

/* Main test runner */ rtems_task Init(rtems_task_argument argument) {

/* The lines below should be generated automatically instead */ Unity.TestFile? = "main.c"; UnityBegin?(); RUN_TEST( test_rtems_tick_announce, 36 ); RUN_TEST( test_rtems_clock_set, 44 ); RUN_TEST( test_rtems_clock_get, 57 ); RUN_TEST( test_rtems_clock_maintain, 110 ); UnityEnd?();

exit( 0 );

}

/* Configuration */ #define CONFIGURE_APPLICATION_NEEDS_CONSOLE_DRIVER #define CONFIGURE_APPLICATION_NEEDS_CLOCK_DRIVER

#define CONFIGURE_RTEMS_INIT_TASKS_TABLE #define CONFIGURE_MAXIMUM_TASKS 1

#define CONFIGURE_INIT #include <rtems/confdefs.h>

Automated Test Runner

An option was added to automatically generate code for the Init task which executes all the tests. An existing Ruby script in Unity for this task (generate_test_runner.rb) has been modified to work with RTEMS.

An example runner.c generated by the script:

/* AUTOGENERATED FILE. DO NOT EDIT. */

=======External Functions This Runner Calls=====

extern void setUp(void); extern void tearDown(void); extern void test_rtems_tick_announce(void); extern void test_rtems_clock_set(); extern void test_rtems_clock_get(); extern void test_rtems_clock_maintain(void);

=======Test Reset Option=====

void resetTest() {

tearDown(); setUp();

}

=======MAIN=====

rtems_task Init(rtems_task_argument argument) {

!Unity.TestFile? = "general/clock/main.c"; UnityBegin(); RUN_TEST(test_rtems_tick_announce, 37); RUN_TEST(test_rtems_clock_set, 45); RUN_TEST(test_rtems_clock_get, 58); RUN_TEST(test_rtems_clock_maintain, 111);

UnityEnd();

}

Once compiled, it is linked with the test binary and can be executed. This fully automates testing and allows the developer to concentrate on writing unit tests.