#2962 assigned defect

Set test configurations to reflect test results.

Reported by: Chris Johns Owned by:
Priority: normal Milestone: 5.1
Component: unspecified Version: 5
Severity: blocker Keywords: testsuite testing GCI
Cc: Blocked By:
Blocking:

Description

Tests can be set to the result we expect such as expected-fail, user-input and indeterminate as well as excluded so we can maintain accurate results for the testsuite.

This ticket covers setting of the correct state for all tests for tier 1 BSPs. Please add an "Updates #" for this ticket to any related commits and once done we can close this ticket.

Note: I have add a top level test configuration file called testsuites/rtems.tcfg that lets us specify a test configuration for all BSPs.

Change History (8)

comment:1 Changed on Oct 10, 2017 at 6:46:55 AM by Sebastian Huber

Component: testingunspecified

comment:2 Changed on Oct 12, 2017 at 2:47:16 AM by Joel Sherrill

Owner: changed from joel.sherrill@… to Chris Johns
Status: newassigned

comment:3 Changed on Nov 9, 2017 at 6:27:14 AM by Sebastian Huber

Milestone: 4.12.05.1

Milestone renamed

comment:4 Changed on Oct 14, 2018 at 1:10:08 AM by Joel Sherrill

Keywords: GCI added

Get Chris to write GCI instructions.

comment:5 Changed on Oct 14, 2018 at 10:11:26 PM by Chris Johns

  1. Run the test suite for a bsp.
  2. Get the list of failures.
  3. Update or create a bsps/<arch>/<bsp>/config/<bsp>-testsuite.tcfg adding a line for each failure we expect to get with:
    expected: <test-name>
    

comment:6 Changed on Oct 19, 2018 at 12:26:06 AM by Chris Johns

Owner: Chris Johns deleted

comment:7 Changed on Nov 15, 2018 at 10:58:12 PM by Chris Johns

Severity: normalblocker

We need to document in the tests which tests are known to fail for specific archs or BSPs. It is important for us to be able to tell a user this is what we have baselined in a release.

comment:8 Changed on Nov 22, 2018 at 2:09:42 AM by Chris Johns

The loopback test needs to be set to USER-INPUT to avoid a timeout when testing the samples with a network build.

Note: See TracTickets for help on using tickets.