Should Tests Be Added to .tcfg file Which Only Fail to Link with RTEMS Debug is Enabled?

[Joel Sherrill]

A recent build sweep with RTEMS debug enabled shows a number of BSPs which fail to build all tests which do when RTEMS debug is disabled. The stance by the RTEMS Project to these failures needs to be formally decided. I see a few options:

• Do nothing
• Add them to .tcfg
• Add a special mode for "fails in debug mode" to .tcfg

I hate to see us with BSPs not building. This is just a bad thing to do.

I hate to add them to .tcfg when they do link in normal build mode since this would reduce testing.

The third option is the best but it would require adding another keyword to the .tcfg syntax and require modifying the processing of those files. But it seems like the best option.

Comments welcomed.

[Chris Johns]

It would mean some attribute, tag or grouping would need to be added to the .tcfg files. The .tcfg processing is now in Python so this makes that side of things easier to implement. I suggest we add groupings. For example:

[default]
 exclude: fileio
 exclude: cdtest
 exclude: iostream
 exclude: flashdisk01

[debug]
 exclude: jffs2_fspermission
 exclude: jffs2_fsrdwr

[smp]
 exclude: pppd

[smp-debug]
 exclude: utf8proc01

There are some issues we need to decide on or solve:

1. Which build options do we create variants for? We should formalize these.
2. How would combinations of variant options be handled? For example is a test in the debug group not built for any debug variant such as smp-debug?
3. How do we perform the make level processing in ​testsuites/automake/subdirs.am to create a suitable list of variants?

[Chris Johns]

I should also point out the rtems-bsp-builder has a concept of variants here ​https://git.rtems.org/rtems-tools/tree/tester/rtems/rtems-bsps.ini#n138.