Changeset 9154c3f9 in rtems


Ignore:
Timestamp:
07/17/15 08:53:22 (9 years ago)
Author:
Sebastian Huber <sebastian.huber@…>
Branches:
4.11, 5, master
Children:
3f923fd2
Parents:
3bb342ca
git-author:
Sebastian Huber <sebastian.huber@…> (07/17/15 08:53:22)
git-committer:
Sebastian Huber <sebastian.huber@…> (07/17/15 09:57:38)
Message:

doc: Add thread dispatch details for SMP

File:
1 edited

Legend:

Unmodified
Added
Removed
  • doc/user/smp.t

    r3bb342ca r9154c3f9  
    372372support.  So basically Ada is not available on RTEMS SMP.
    373373
     374@subsection Thread Dispatch Details
     375
     376This section gives background information to developers interested in the
     377interrupt latencies introduced by thread dispatching.  A thread dispatch
     378consists of all work which must be done to stop the currently executing thread
     379on a processor and hand over this processor to an heir thread.
     380
     381On SMP systems, scheduling decisions on one processor must be propagated to
     382other processors through inter-processor interrupts.  So, a thread dispatch
     383which must be carried out on another processor happens not instantaneous.  Thus
     384several thread dispatch requests might be in the air and it is possible that
     385some of them may be out of date before the corresponding processor has time to
     386deal with them.  The thread dispatch mechanism uses three per-processor
     387variables,
     388@itemize @bullet
     389@item the executing thread,
     390@item the heir thread, and
     391@item an boolean flag indicating if a thread dispatch is necessary or not.
     392@end itemize
     393Updates of the heir thread and the thread dispatch necessary indicator are
     394synchronized via explicit memory barriers without the use of locks.  A thread
     395can be an heir thread on at most one processor in the system.  The thread context
     396is protected by a TTAS lock embedded in the context to ensure that it is used
     397on at most one processor at a time.  The thread post-switch actions use a
     398per-processor lock.  This implementation turned out to be quite efficient and
     399no lock contention was observed in the test suite.
     400
     401The current implementation of thread dispatching has some implications with
     402respect to the interrupt latency.  It is crucial to preserve the system
     403invariant that a thread can execute on at most one processor in the system at a
     404time.  This is accomplished with a boolean indicator in the thread context.
     405The processor architecture specific context switch code will mark that a thread
     406context is no longer executing and waits that the heir context stopped
     407execution before it restores the heir context and resumes execution of the heir
     408thread (the boolean indicator is basically a TTAS lock).  So, there is one
     409point in time in which a processor is without a thread.  This is essential to
     410avoid cyclic dependencies in case multiple threads migrate at once.  Otherwise
     411some supervising entity is necessary to prevent deadlocks.  Such a global
     412supervisor would lead to scalability problems so this approach is not used.
     413Currently the context switch is performed with interrupts disabled.  Thus in
     414case the heir thread is currently executing on another processor, the time of
     415disabled interrupts is prolonged since one processor has to wait for another
     416processor to make progress.
     417
     418It is difficult to avoid this issue with the interrupt latency since interrupts
     419normally store the context of the interrupted thread on its stack.  In case a
     420thread is marked as not executing, we must not use its thread stack to store
     421such an interrupt context.  We cannot use the heir stack before it stopped
     422execution on another processor.  If we enable interrupts during this
     423transition, then we have to provide an alternative thread independent stack for
     424interrupts in this time frame.  This issue needs further investigation.
     425
     426The problematic situation occurs in case we have a thread which executes with
     427thread dispatching disabled and should execute on another processor (e.g. it is
     428an heir thread on another processor).  In this case the interrupts on this
     429other processor are disabled until the thread enables thread dispatching and
     430starts the thread dispatch sequence.  The scheduler (an exception is the
     431scheduler with thread processor affinity support) tries to avoid such a
     432situation and checks if a new scheduled thread already executes on a processor.
     433In case the assigned processor differs from the processor on which the thread
     434already executes and this processor is a member of the processor set managed by
     435this scheduler instance, it will reassign the processors to keep the already
     436executing thread in place.  Therefore normal scheduler requests will not lead
     437to such a situation.  Explicit thread migration requests, however, can lead to
     438this situation.  Explicit thread migrations may occur due to the scheduler
     439helping protocol or explicit scheduler instance changes.  The situation can
     440also be provoked by interrupts which suspend and resume threads multiple times
     441and produce stale asynchronous thread dispatch requests in the system.
     442
    374443@c
    375444@c
Note: See TracChangeset for help on using the changeset viewer.