#3151 new defect

Should Tests Be Added to .tcfg file Which Only Fail to Link with RTEMS Debug is Enabled?

Reported by: Joel Sherrill Owned by: joel.sherrill@…
Priority: normal Milestone:
Component: unspecified Version: 5
Severity: normal Keywords:
Cc: Blocked By:
Blocking:

Description

A recent build sweep with RTEMS debug enabled shows a number of BSPs which fail to build all tests which do when RTEMS debug is disabled. The stance by the RTEMS Project to these failures needs to be formally decided. I see a few options:

  • Do nothing
  • Add them to .tcfg
  • Add a special mode for "fails in debug mode" to .tcfg

I hate to see us with BSPs not building. This is just a bad thing to do.

I hate to add them to .tcfg when they do link in normal build mode since this would reduce testing.

The third option is the best but it would require adding another keyword to the .tcfg syntax and require modifying the processing of those files. But it seems like the best option.

Comments welcomed.

Change History (4)

comment:1 Changed on Sep 21, 2017 at 11:35:56 PM by Joel Sherrill

Summary: Shuold Tests Be Added to .tcfg file Which Only Fail to Link with RTEMS Debug is Enabled?Should Tests Be Added to .tcfg file Which Only Fail to Link with RTEMS Debug is Enabled?

comment:2 Changed on Sep 22, 2017 at 12:15:53 AM by Chris Johns

It would mean some attribute, tag or grouping would need to be added to the .tcfg files. The .tcfg processing is now in Python so this makes that side of things easier to implement. I suggest we add groupings. For example:

[default]
 exclude: fileio
 exclude: cdtest
 exclude: iostream
 exclude: flashdisk01

[debug]
 exclude: jffs2_fspermission
 exclude: jffs2_fsrdwr

[smp]
 exclude: pppd

[smp-debug]
 exclude: utf8proc01

There are some issues we need to decide on or solve:

  1. Which build options do we create variants for? We should formalize these.
  2. How would combinations of variant options be handled? For example is a test in the debug group not built for any debug variant such as smp-debug?
  3. How do we perform the make level processing in testsuites/automake/subdirs.am to create a suitable list of variants?

comment:3 Changed on Sep 22, 2017 at 12:18:26 AM by Chris Johns

I should also point out the rtems-bsp-builder has a concept of variants here https://git.rtems.org/rtems-tools/tree/tester/rtems/rtems-bsps.ini#n138.

comment:4 Changed on Oct 10, 2017 at 6:46:55 AM by Sebastian Huber

Component: testingunspecified
Note: See TracTickets for help on using tickets.