Changeset 5a801c2 in rtems-tools


Ignore:
Timestamp:
07/22/15 11:42:13 (9 years ago)
Author:
Ric Claus <claus@…>
Branches:
4.10, 5, master
Children:
711f715
Parents:
92935ed
git-author:
Ric Claus <claus@…> (07/22/15 11:42:13)
git-committer:
Chris Johns <chrisj@…> (09/26/15 07:16:44)
Message:

rtems-tester.txt, options.py: Improved clarity and spelling.

Files:
2 edited

Legend:

Unmodified
Added
Removed
  • doc/rtems-tester.txt

    r92935ed r5a801c2  
    1616
    1717The RTEMS Tester is a test framework. It includes a command line interface to
    18 run tests on supported targets. The framework provides backend support for
     18run tests on supported targets. The framework provides back-end support for
    1919common simulators and debuggers. The board support package (BSP) configurations
    2020for RTEMS are provided and can be used to run all the tests provided with
     
    2626from open source simulators, commercial simulators, debuggers with simulators,
    2727to debuggers with hardware specific pods and devices. Testing RTEMS requires
    28 the cross-compiled test executable is transfered to the target hardware,
    29 executed and the output returned to the host where it is analyised to determine
     28the cross-compiled test executable is transferred to the target hardware,
     29executed and the output returned to the host where it is analyzed to determine
    3030the test result. The RTEMS Tester provides a framework to do this.
    3131
    3232Running all the RTEMS tests on your target is very important. It provides you
    33 with a tracable record your RTEMS version and its tools and working at the
     33with a traceable record your RTEMS version and its tools are working at the
    3434level the RTEMS development team expect when releasing RTEMS. Being able to
    35 easly run the tests and verify the results is critical in maintiaining a high
     35easily run the tests and verify the results is critical in maintaining a high
    3636standard.
    3737
     
    4040* Command line tool (+rtems-test+)
    4141* BSP Configuration scripts
    42 * Backend Configuration scripts
    43 * Backend Python classes
     42* Back-end Configuration scripts
     43* Back-end Python classes
    4444* Python based framework
    4545
     
    5555The RTEMS Tester is part of the RTEMS Tools Project. The code is released under
    5656the OSI approved The BSD 2-Clause License. It is free to use and we encourage
    57 this including operating systems other than RTEMS.
     57this, including on operating systems other than RTEMS.
    5858
    5959The code and command line tools must retain the same names and always reference
     
    6565The quick start will show you how to run the test suite for a BSP. It will
    6666explain how to get the RTEMS Tester, set it up and run the tests for the SIS
    67 BSP. It assumes you have a valid SPARC tool chain and built SIS BSP version of
    68 RTEMS. 4.11.
     67BSP. It assumes you have a valid SPARC tool chain and have built the SIS BSP
     68version of RTEMS. 4.11.
    6969
    7070Setup
    7171~~~~~
    7272
    73 Setup a development work space:
     73Set up a development work space:
    7474
    7575-------------------------------------------------------------
     
    8282
    8383-------------------------------------------------------------
    84 $ git git://git.rtems.org/rtems-tools.git rtems-tools.git
     84$ git clone git://git.rtems.org/rtems-tools.git rtems-tools.git
    8585$ cd rtems-tools.git/tester
    8686-------------------------------------------------------------
    8787
    88 Available BSPs
    89 ~~~~~~~~~~~~~~
    90 
    91 You can list the available BSP's with:
     88Available BSP testers
     89~~~~~~~~~~~~~~~~~~~~~
     90
     91You can list the available BSP testers with:
    9292
    9393-------------------------------------------------------------
     
    118118
    119119Some of the BSPs may appear more than once in the list. These are aliased BSP
    120 configuration's that may use a different backend. An example is the SPARC
    121 Instruction Simulator (SIS) BSP. There is the 'sis' BSP which uses the GDB
    122 backend and the 'sis-run' which uses the command line version of the SIS
    123 simulator. We will show how to use +rtems-test+ conmand with the SIS BSP
     120configurations that may use a different back-end. An example is the SPARC
     121Instruction Simulator (SIS) BSP. There is the 'sis' tester which uses the GDB
     122back-end and the 'sis-run' tester which uses the command line version of the SIS
     123simulator. We will show how to use +rtems-test+ command with the SIS BSP
    124124because it is easy to build an use.
    125125
     
    131131[NOTE]
    132132=============================================================
    133 The following assumes a Unix type host and the tools have been built with
     133The following assumes a Unix-type host and that the tools have been built with
    134134a prefix of +$HOME/development/rtems/4.11+.
    135135=============================================================
     
    154154
    155155Building all the tests takes time and it uses more disk so be patient. When
    156 finished all the tests will be built ready to run. Before running all the tests
     156finished all the tests will be built and ready to run. Before running all the tests
    157157it is a good idea to run the +hello+ test. The +hello+ test is an RTEMS version
    158158of the classic "Hello World" example and running it shows you have a working
     
    219219executable. You can pass more than one executable on the command line. If the
    220220executable is a path to a directory the directories under that path are
    221 searched for any file with a +.exe+ extension. This is the detault extension
     221searched for any file with a +.exe+ extension. This is the default extension
    222222for RTEMS executables built within RTEMS.
    223223
     
    271271<6> The output has been shortened so it fits nicely here.
    272272<7> The test results. It shows passes, fails, timeouts, and invalid results. In
    273 this run 495 tests passed and 5 tests timedout. The timeouts are probability
    274 due the tests not having enough execute time to complete. The default timeout
     273this run 495 tests passed and 5 tests timed out. The timeouts are probably due
     274to the tests not having enough execute time to complete. The default timeout
    275275is 180 seconds and some of the interrupt tests need longer. The amount of time
    276276depends on the performance of your host CPU running the simulations.
     
    280280+sparc-rtems4.11-gdb+ command that is part of the RTEMS tools. Not every BSP
    281281will require this option so you will need to check the specifics of the BSP
    282 configration to determine if it is needed.
    283 
    284 The output you see is each test starting to run. The +rtems-test+ command can
    285 run multiple SIS GDB simulations in parallel so you will see a number start
    286 quickly and then tests start as others finish. The output shown here is from an
     282configuration to determine if it is needed.
     283
     284The output you see is each test starting to run. The +rtems-test+ command by
     285default runs multiple tests in parallel so you will see a number start quickly
     286and then new tests start as others finish. The output shown here is from an
    2872878 core processor so the first 8 are started in parallel and the status shows
    288 the order they actually started which is not 1 to 8.
     288the order in which they actually started, which is not 1 to 8.
    289289
    290290The test start line shows the current status of the tests. The status reported
    291291is when the test starts and not the result of that test. A fail, timeout or
    292292invalid count changing means a test running before this test started failed,
    293 not the starting test. The status here has 495 tests pass and no failures and 5
     293not the starting test. The status here has 495 tests passed, no failures and 5
    294294timeouts.:
    295295
     
    299299<1> The test number, in this case test 295 of 500 tests.
    300300<2> Passed test count.
    301 <3> Failied test count.
     301<3> Failed test count.
    302302<4> Timeout test count.
    303303<5> Invalid test count.
     
    306306
    307307The test log records all the tests and results. The reporting mode by default
    308 only provides the output history if a test fails, timeouts, or is invalid. The
     308only provides the output history if a test fails, times out, or is invalid. The
    309309time taken by each test is also recorded.
    310310
     
    315315specifics of the host machine being used. A test per core is the most stable
    316316method even though more tests can be run than available cores. If your machine
    317 needs longer or you are using a VM you may need to lengthen the time out.
     317needs longer or you are using a VM you may need to lengthen the timeout.
    318318
    319319Test Status
     
    340340.Timeout
    341341If the test does not complete within the timeout setting the test is marked as
    342 timed out.
     342having timed out.
    343343
    344344.Invalid
    345345If no start marker is seen the test is marked as invalid. If you are testing on
    346346real target hardware things can sometimes go wrong and the target may not
    347 initialise or respond to the debugger in an expected way.
     347initialize or respond to the debugger in an expected way.
    348348
    349349Reporting
     
    442442
    443443The RTEMS Tester supports parallel execution of tests by default. This only
    444 makes sense if the test backend can run in parallel without resulting in
    445 resource contention. Simulators are an example of backends that can run in
    446 parallel. A hardware debug tool like a BDM or JTAG pod can only a single test
    447 at once to the tests need to be run one at a time.
     444makes sense if the test back-end can run in parallel without resulting in
     445resource contention. Simulators are an example of back-ends that can run in
     446parallel. A hardware debug tool like a BDM or JTAG pod can manage only a
     447single test at once so the tests need to be run one at a time.
    448448
    449449The test framework manages the test jobs and orders the output in the report
     
    467467--keep-going                 : Do not stop on an error.
    468468--list-bsps                  : List the supported BSPs
    469 --log file                   : Log file where all build out is written too
     469--log file                   : Log file where all build output is written to
    470470--macros file[,file]         : Macro format files to load after the defaults
    471471--no-clean                   : Do not clean up the build tree
     
    480480-------------------------------------------------------------
    481481
    482 Developement
     482Development
    483483------------
    484484
    485485The RTEMS Tester framework and command line tool is under active
    486 development. This are changing, being fixed, broken and generally improved. If
    487 you want to help please see the Wiki page for open itmes.
     486development. These are changing, being fixed, broken and generally improved. If
     487you want to help please see the Wiki page for open items.
    488488
    489489
     
    494494The RTEMS Tester is based on a refactored base of Python code used in the RTEMS
    495495Source Builder. This code provided a working tested base that has been extended
    496 and expanded to meet the needs of the RTEMS Tester. The tester uses the
     496and expanded to meet the requirements for the RTEMS Tester. The tester uses the
    497497specifics found in the various scripts and configurations in the
    498498rtems-testing.git repo that has been accumulated over many years. The shell
    499 script implementation is restricted in what can it do and per BSP script is a
    500 maintenance burden, for example the command lines and options vary between each
     499script implementation is restricted in what it can do and, per BSP script, is a
     500maintenance burden. For example the command lines and options vary between each
    501501script.
  • rtemstoolkit/options.py

    r92935ed r5a801c2  
    105105            '--jobs=[0..n,none,half,full]': 'Run with specified number of jobs, default: num CPUs.',
    106106            '--macros file[,file]':         'Macro format files to load after the defaults',
    107             '--log file':                   'Log file where all build out is written too',
     107            '--log file':                   'Log file where all build output is written to',
    108108        }
    109109        self.opts = { 'params' : [] }
Note: See TracChangeset for help on using the changeset viewer.