Changeset 89e72a80 in rtems

Mar 10, 2014, 4:35:16 PM (6 years ago)
Joel Sherrill <joel.sherrill@…>
4.11, master
Joel Sherrill <joel.sherrill@…> (03/10/14 16:35:16)
Joel Sherrill <joel.sherrill@…> (03/12/14 00:03:53)

smp.t: Add Background and Operation Sections

1 edited


  • doc/user/smp.t

    rd46ab11b r89e72a80  
    22@c  COPYRIGHT (c) 2014.
    33@c  On-Line Applications Research Corporation (OAR).
    4 @c  All rights reserved. 
     4@c  All rights reserved.
    1111This chapter describes the services related to Symmetric Multiprocessing
    12 provided by RTEMS. 
     12provided by RTEMS.
    1414The application level services currently provided are:
    2020@end itemize
    2225@section Background
     27@subsection Uniprocessor versus SMP Parallelism
     29Uniprocessor systems have long been used in embedded systems. In this hardware
     30model, there are some system execution characteristics which have long been
     31taken for granted:
     33@itemize @bullet
     34@item one task executes at a time
     35@item hardware events result in interrupts
     36@end itemize
     38There is no true parallelism. Even when interrupts appear to occur
     39at the same time, they are processed in largely a serial fashion.
     40This is true even when the interupt service routines are allowed to
     41nest.  From a tasking viewpoint,  it is the responsibility of the real-time
     42operatimg system to simulate parallelism by switching between tasks.
     43These task switches occur in response to hardware interrupt events and explicit
     44application events such as blocking for a resource or delaying.
     46With symmetric multiprocessing, the presence of multiple processors
     47allows for true concurrency and provides for cost-effective performance
     48improvements. Uniprocessors tend to increase performance by increasing
     49clock speed and complexity. This tends to lead to hot, power hungry
     50microprocessors which are poorly suited for many embedded applications.
     52The true concurrency is in sharp contrast to the single task and
     53interrupt model of uniprocessor systems. This results in a fundamental
     54change to uniprocessor system characteristics listed above. Developers
     55are faced with a different set of characteristics which, in turn, break
     56some existing assumptions and result in new challenges. In an SMP system
     57with N processors, these are the new execution characteristics.
     59@itemize @bullet
     60@item N tasks execute in parallel
     61@item hardware events result in interrupts
     62@end itemize
     64There is true parallelism with a task executing on each processor and
     65the possibility of interrupts occurring on each processor. Thus in contrast
     66to their being one task and one interrupt to consider on a uniprocessor,
     67there are N tasks and potentially N simultaneous interrupts to consider
     68on an SMP system.
     70This increase in hardware complexity and presence of true parallelism
     71results in the application developer needing to be even more cautious
     72about mutual exclusion and shared data access than in a uniprocessor
     73embedded system. Race conditions that never or rarely happened when an
     74application executed on a uniprocessor system, become much more likely
     75due to multiple threads executing in parallel. On a uniprocessor system,
     76these race conditions would only happen when a task switch occurred at
     77just the wrong moment. Now there are N-1 tasks executing in parallel
     78all the time and this results in many more opportunities for small
     79windows in critical sections to be hit.
     81@subsection Task Affinity
     83@cindex task affinity
     84@cindex thread affinity
     86RTEMS provides services to manipulate the affinity of a task. Affinity
     87is used to specify the subset of processors in an SMP system on which
     88a particular task can execute.
     90By default, tasks have an affinity which allows them to execute on any
     91available processor.
     93Task affinity is a possible feature to be supported by SMP-aware
     94schedulers. However, only a subset of the available schedulers support
     95affinity. Although the behavior is scheduler specific, if the scheduler
     96does not support affinity, it is likely to ignore all attempts to set
     99@subsection Critical Section Techniques and SMP
     101As discussed earlier, SMP systems have opportunities for true parallelism
     102which was not possible on uniprocessor systems. Consequently, multiple
     103techniques that provided adequate critical sections on uniprocessor
     104systems are unsafe on SMP systems. In this section, some of these
     105unsafe techniques will be discussed.
     107In general, applications must use proper operating system provided mutual
     108exclusion mechanisms to ensure correct behavior. This primarily means
     109the use of binary semaphores or mutexes to implement critical sections.
     111@subsubsection Disable Interrupts
     113Again on a uniprocessor system, there is only a single processor which
     114logically executes a single task and takes interrupts. On an SMP system,
     115each processor may take an interrupt. When the application disables
     116interrupts, it generally does so by altering a processor register to
     117mask interrupts and later to re-enable them. On a uniprocessor system,
     118changing this in the single processor is sufficient. However, on an SMP
     119system, this register in @strong{ALL} processors must be changed. There
     120are no comparable capabilities in an SMP system to disable all interrupts
     121across all processors.
     123@subsubsection Highest Priority Task Assumption
     125On a uniprocessor system, it is safe to assume that when the highest
     126priority task in an application executes, it will execute without being
     127preempted until it voluntarily blocks. Interrupts may occur while it is
     128executing, but there will be no context switch to another task unless
     129the highest priority task voluntarily initiates it.
     131Given the assumption that no other tasks will have their execution
     132interleaved with the highest priority task, it is possible for this
     133task to be constructed such that it does not need to acquire a binary
     134semaphore or mutex for protected access to shared data.
     136In an SMP system, it cannot be assumed there will never be a single task
     137executing. It should be assumed that every processor is executing another
     138application task. Further, those tasks will be ones which would not have
     139been executed in a uniprocessor configuration and should be assumed to
     140have data synchronization conflicts with what was formerly the highest
     141priority task which executed without conflict.
     143@subsubsection Disable Preemption
     145On a uniprocessor system, disabling preemption in a task is very similar
     146to making the highest priority task assumption. While preemption is
     147disabled, no task context switches will occur unless the task initiates
     148them voluntarily. And, just as with the highest priority task assumption,
     149there are N-1 processors also running tasks. Thus the assumption that no
     150other tasks will run while the task has preemption disabled is violated.
     152@subsection Task Unique Data and SMP
     154Per task variables are a service commonly provided by real-time operating
     155systems for application use. They work by allowing the application
     156to specify a location in memory (typically a @code{void *}) which is
     157logically added to the context of a task. On each task switch, the
     158location in memory is stored and each task can have a unique value in
     159the same memory location. This memory location is directly accessed as a
     160variable in a program.
     162This works well in a uniprocessor environment because there is one task
     163executing and one memory location containing a task-specific value. But
     164it is fundamentally broken on an SMP system because there are always N
     165tasks executing. With only one location in memory, N-1 tasks will not
     166have the correct value.
     168This paradigm for providing task unique data values is fundamentally
     169broken on SMP systems.
     171@subsubsection Classic API Per Task Variables
     173The Classic API provides three directives to support per task variables. These are:
     175@itemize @bullet
     176@item @code{@value{DIRPREFIX}task_variable_add} - Associate per task variable
     177@item @code{@value{DIRPREFIX}task_variable_get} - Obtain value of a a per task variable
     178@item @code{@value{DIRPREFIX}task_variable_delete} - Remove per task variable
     179@end itemize
     181As task variables are unsafe for use on SMP systems, the use of these
     182services should be eliminated in all software that is to be used in
     183an SMP environment. It is recommended that the application developer
     184consider the use of POSIX Keys or Thread Local Storage (TLS). POSIX Keys
     185are not enabled in all RTEMS configurations.
     187@b{STATUS}: As of March 2014, some support services in the
     188@code{rtems/cpukit} use per task variables. When these uses are
     189eliminated, the per task variable directives will be disabled when
     190building RTEMS in SMP configuration.
    24195@section Operations
     197@subsection Setting Affinity to a Single Processor
     199In many embedded applications targeting SMP systems, it is common to lock individual tasks to specific cores. In this way, one can designate a core for I/O tasks, another for computation, etc.. The following illustrates the code sequence necessary to assign a task an affinity for processor zero (0).
     202rtems_status_code sc;
     203cpu_set_t         set;
     205CPU_EMPTY( &set );
     206CPU_SET( 0, &set );
     208sc = rtems_task_set_affinity(rtems_task_self(), sizeof(set), &set);
     209assert(sc == RTEMS_SUCCESSFUL);
     210@end example
     212It is important to note that the @code{cpu_set_t} is not validated until the
     213@code{@value{DIRPREFIX}task_set_affinity} call is made. At that point,
     214it is validated against the current system configuration.
    26219@section Directives
    77270  rtems_id             id,
    78271  size_t               cpusetsize,
    79   cpu_set_t           *cpuset 
     272  cpu_set_t           *cpuset
    81274@end example
    89282@table @b
    90283@item RTEMS_UNSATISFIED
    91 The cpuset pointer argument is invalid.
     284The @code{cpuset} pointer argument is invalid.
    93286@item RTEMS_UNSATISFIED
    125318  rtems_id             id,
    126319  size_t               cpusetsize,
    127   cpu_set_t           *cpuset 
     320  cpu_set_t           *cpuset
    129322@end example
    137330@table @b
    138331@item RTEMS_UNSATISFIED
    139 The cpuset pointer argument is invalid.
     332The @code{cpuset} pointer argument is invalid.
    141334@item RTEMS_UNSATISFIED
    158351The @code{rtems_task_set_affinity} routine is used to set the
    159352@code{affinity.set} field of the thread control object associated
    160 with @code{id}.  This value controls the cpuset that the task can
     353with @code{id}.  This value controls the @code{cpuset} that the task can
    161354execute on.
Note: See TracChangeset for help on using the changeset viewer.