Notice: We have migrated to GitLab launching 2024-05-01 see here: https://gitlab.rtems.org/

Opened on 09/09/20 at 00:21:59

Last modified on 07/27/23 at 16:37:38

#4072 assigned defect

testsuite: Add ticket ids for expected-fail and indeterminate states, add annotations

Reported by: Chris Johns Owned by: Chris Johns
Priority: high Milestone: 6.1
Component: test Version: 6
Severity: blocker Keywords: test rtems-test
Cc: Blocked By:
Blocking:

Description

  • Require a ticket id be present for any expected-fail and indeterminate test states.
    1. Parse the ticket number from the .tcgf file and pass into the test.
    2. Have rtems-test obtain the ticket number from the test when run and provide the ticket number in the results.
  • Add annotations for tests. The list is to be provided by Joel.

Change History (6)

comment:1 Changed on 11/29/22 at 22:18:54 by Chris Johns

Priority: normal → high
Severity: normal → blocker

comment:2 Changed on 06/29/23 at 23:34:15 by Chris Johns

This is now updated in the spec YAML files.

comment:3 Changed on 06/29/23 at 23:40:04 by Kinsey Moore

I have some to handle for AArch64 and ARM and any BSPs that run specifically on QEMU.

comment:4 Changed on 07/10/23 at 13:49:21 by Kinsey Moore <kinsey.moore@…>

In [changeset:"f46c15fd76a4c1904760360cd220a061d74edad4/rtems" f46c15f/rtems]:

Error: Processor CommitTicketReference failed
/data/trac/repo/rtems.git does not appear to be a Git repository.

comment:5 Changed on 07/10/23 at 13:52:26 by Kinsey Moore

The above commit takes care of all the indeterminate test states that I'm aware of in the AArch64 realm.

comment:6 Changed on 07/27/23 at 16:37:38 by Kinsey Moore

The only indeterminate test state is now documented properly. These are the remaining files that have a test state of expected-fail:

spec/build/bsps/tst-xfail-intr.yml (validation-intr for powerpc/psim)
spec/build/bsps/i386/pc386/tst-xfail-val.yml (various validation tests for i386/pc386)
spec/build/bsps/tst-xfail-dl06.yml (dl06 for arm/imx7)
spec/build/bsps/tst.yml (psxfenv01 for all arches)

Note: See TracTickets for help on using tickets.