#3885 closed defect (fixed)

Context switch extension is broken in SMP configurations

Reported by: Sebastian Huber Owned by: Sebastian Huber
Priority: normal Milestone: 5.1
Component: score Version: 5
Severity: normal Keywords:
Cc: Blocked By:
Blocking:

Description (last modified by Sebastian Huber)

The context switch extensions are called during _Thread_Do_dispatch():

void _Thread_Do_dispatch( Per_CPU_Control *cpu_self, ISR_Level level )
{
  Thread_Control *executing;

  executing = cpu_self->executing;

  ...
  do {
    Thread_Control *heir;

    heir = _Thread_Get_heir_and_make_it_executing( cpu_self );
    ...
    _User_extensions_Thread_switch( executing, heir );
    ...
    _Context_Switch( &executing->Registers, &heir->Registers );
    ...
  } while ( cpu_self->dispatch_necessary );
  ...
}

In uniprocessor configurations, the context switch extensions are called for all thread switches except the very first thread switch to the initialization thread. However, in SMP configurations, the context switch may be invalidated and updated in the low-level _Context_Switch() routine. See:

https://docs.rtems.org/branches/master/c-user/symmetric_multiprocessing_services.html#thread-dispatch-details

In case such an update happens, a thread executes on the processor which was not visible to the context switch extensions. This can confuse for example event record consumers which use events generated by a context switch extension.

Fixing this is not straight forward. The context switch extensions call must move after the low-level context switch. The problem here is that we may end up in _Thread_Handler(). Adding the context switch extensions call to _Thread_Handler() covers now also the thread switch to the initialization thread. We also have to save the last executing thread of the processor. Registers or the stack cannot be used for this purpose. We have to add it to the per-processor information. Existing extensions may be affected, since now context switch extensions use the stack of the heir thread.

Calling the thread switch extensions in the low level context switch is difficult since at this point an intermediate stack is used which is only large enough to enable servicing of interrupts.

Change History (11)

comment:1 Changed on Feb 25, 2020 at 6:34:30 PM by Joel Sherrill

As I recall, doing this in _Thread_Handler is similar to having to restore FPU. Are there other actions with this characteristic?

comment:2 Changed on Feb 26, 2020 at 2:55:06 AM by Chris Johns

Is this an issue that needs to be fixed for RTEMS 5?

comment:3 in reply to:  2 Changed on Feb 26, 2020 at 8:47:44 AM by Sebastian Huber

Replying to Chris Johns:

Is this an issue that needs to be fixed for RTEMS 5?

Yes, I think that the thread switch extensions should work in SMP configurations.

comment:4 Changed on Feb 26, 2020 at 8:49:11 AM by Sebastian Huber

Moving the thread switch extension calls into the context of the heir thread breaks for example the stack checker. It checks that the current stack pointer is in the stack area of a thread.

comment:5 Changed on Feb 26, 2020 at 8:51:14 AM by Sebastian Huber

Description: modified (diff)

comment:6 in reply to:  4 ; Changed on Feb 27, 2020 at 12:23:43 AM by Chris Johns

Replying to Sebastian Huber:

Moving the thread switch extension calls into the context of the heir thread breaks for example the stack checker. It checks that the current stack pointer is in the stack area of a thread.

Is a patch to fix the stack checker being worked on?

I know of applications that have this running during development.

comment:7 in reply to:  6 Changed on Feb 27, 2020 at 6:58:32 AM by Sebastian Huber

Replying to Chris Johns:

Replying to Sebastian Huber:

Moving the thread switch extension calls into the context of the heir thread breaks for example the stack checker. It checks that the current stack pointer is in the stack area of a thread.

Is a patch to fix the stack checker being worked on?

I know of applications that have this running during development.

This was easy to fix:

https://lists.rtems.org/pipermail/devel/2020-February/057759.html

I just wanted to highlight that this change can easily break existing extensions.

comment:8 Changed on Feb 27, 2020 at 10:46:31 PM by Chris Johns

Thanks.

comment:9 Changed on Mar 2, 2020 at 6:52:07 AM by Sebastian Huber <sebastian.huber@…>

In fcb11510/rtems:

score: Fix context switch extensions (SMP)

In uniprocessor and SMP configurations, the context switch extensions
were called during _Thread_Do_dispatch():

void _Thread_Do_dispatch( Per_CPU_Control *cpu_self, ISR_Level level )
{

Thread_Control *executing;

executing = cpu_self->executing;

...
do {

Thread_Control *heir;

heir = _Thread_Get_heir_and_make_it_executing( cpu_self );
...
_User_extensions_Thread_switch( executing, heir );
...
_Context_Switch( &executing->Registers, &heir->Registers );
...

} while ( cpu_self->dispatch_necessary );
...

}

In uniprocessor configurations, this is fine and the context switch
extensions are called for all thread switches except the very first
thread switch to the initialization thread. However, in SMP
configurations, the context switch may be invalidated and updated in the
low-level _Context_Switch() routine. See:

https://docs.rtems.org/branches/master/c-user/symmetric_multiprocessing_services.html#thread-dispatch-details

In case such an update happens, a thread will execute on the processor
which was not seen in the previous call of the context switch
extensions. This can confuse for example event record consumers which
use events generated by a context switch extension.

Fixing this is not straight forward. The context switch extensions call
must move after the low-level context switch. The problem here is that
we may end up in _Thread_Handler(). Adding the context switch
extensions call to _Thread_Handler() covers now also the thread switch
to the initialization thread. We also have to save the last executing
thread (ancestor) of the processor. Registers or the stack cannot be
used for this purpose. We have to add it to the per-processor
information. Existing extensions may be affected, since now context
switch extensions use the stack of the heir thread. The stack checker
is affected by this.

Calling the thread switch extensions in the low-level context switch is
difficult since at this point an intermediate stack is used which is
only large enough to enable servicing of interrupts.

Update #3885.

comment:10 Changed on Mar 3, 2020 at 6:30:26 AM by Sebastian Huber <sebastian.huber@…>

In 198c07e5/rtems:

sptests/spextensions01: Add comment

Update #3885.

comment:11 Changed on Mar 3, 2020 at 12:13:32 PM by Sebastian Huber <sebastian.huber@…>

Resolution: fixed
Status: assignedclosed

In 8bd4e6a/rtems-docs:

c-user: Document thread switch extension changes

Close #3885.

Note: See TracTickets for help on using tickets.