1 | @c |
---|
2 | @c COPYRIGHT (c) 2014. |
---|
3 | @c On-Line Applications Research Corporation (OAR). |
---|
4 | @c All rights reserved. |
---|
5 | @c |
---|
6 | |
---|
7 | @chapter Symmetric Multiprocessing Services |
---|
8 | |
---|
9 | @section Introduction |
---|
10 | |
---|
11 | This chapter describes the services related to Symmetric Multiprocessing |
---|
12 | provided by RTEMS. |
---|
13 | |
---|
14 | The application level services currently provided are: |
---|
15 | |
---|
16 | @itemize @bullet |
---|
17 | @item @code{rtems_get_processor_count} - Get processor count |
---|
18 | @item @code{rtems_get_current_processor} - Get current processor index |
---|
19 | @item @code{rtems_scheduler_ident} - Get ID of a scheduler |
---|
20 | @item @code{rtems_scheduler_get_processor_set} - Get processor set of a scheduler |
---|
21 | @item @code{rtems_task_get_scheduler} - Get scheduler of a task |
---|
22 | @item @code{rtems_task_get_affinity} - Get task processor affinity |
---|
23 | @item @code{rtems_task_set_affinity} - Set task processor affinity |
---|
24 | @end itemize |
---|
25 | |
---|
26 | @c |
---|
27 | @c |
---|
28 | @c |
---|
29 | @section Background |
---|
30 | |
---|
31 | @subsection Uniprocessor versus SMP Parallelism |
---|
32 | |
---|
33 | Uniprocessor systems have long been used in embedded systems. In this hardware |
---|
34 | model, there are some system execution characteristics which have long been |
---|
35 | taken for granted: |
---|
36 | |
---|
37 | @itemize @bullet |
---|
38 | @item one task executes at a time |
---|
39 | @item hardware events result in interrupts |
---|
40 | @end itemize |
---|
41 | |
---|
42 | There is no true parallelism. Even when interrupts appear to occur |
---|
43 | at the same time, they are processed in largely a serial fashion. |
---|
44 | This is true even when the interupt service routines are allowed to |
---|
45 | nest. From a tasking viewpoint, it is the responsibility of the real-time |
---|
46 | operatimg system to simulate parallelism by switching between tasks. |
---|
47 | These task switches occur in response to hardware interrupt events and explicit |
---|
48 | application events such as blocking for a resource or delaying. |
---|
49 | |
---|
50 | With symmetric multiprocessing, the presence of multiple processors |
---|
51 | allows for true concurrency and provides for cost-effective performance |
---|
52 | improvements. Uniprocessors tend to increase performance by increasing |
---|
53 | clock speed and complexity. This tends to lead to hot, power hungry |
---|
54 | microprocessors which are poorly suited for many embedded applications. |
---|
55 | |
---|
56 | The true concurrency is in sharp contrast to the single task and |
---|
57 | interrupt model of uniprocessor systems. This results in a fundamental |
---|
58 | change to uniprocessor system characteristics listed above. Developers |
---|
59 | are faced with a different set of characteristics which, in turn, break |
---|
60 | some existing assumptions and result in new challenges. In an SMP system |
---|
61 | with N processors, these are the new execution characteristics. |
---|
62 | |
---|
63 | @itemize @bullet |
---|
64 | @item N tasks execute in parallel |
---|
65 | @item hardware events result in interrupts |
---|
66 | @end itemize |
---|
67 | |
---|
68 | There is true parallelism with a task executing on each processor and |
---|
69 | the possibility of interrupts occurring on each processor. Thus in contrast |
---|
70 | to their being one task and one interrupt to consider on a uniprocessor, |
---|
71 | there are N tasks and potentially N simultaneous interrupts to consider |
---|
72 | on an SMP system. |
---|
73 | |
---|
74 | This increase in hardware complexity and presence of true parallelism |
---|
75 | results in the application developer needing to be even more cautious |
---|
76 | about mutual exclusion and shared data access than in a uniprocessor |
---|
77 | embedded system. Race conditions that never or rarely happened when an |
---|
78 | application executed on a uniprocessor system, become much more likely |
---|
79 | due to multiple threads executing in parallel. On a uniprocessor system, |
---|
80 | these race conditions would only happen when a task switch occurred at |
---|
81 | just the wrong moment. Now there are N-1 tasks executing in parallel |
---|
82 | all the time and this results in many more opportunities for small |
---|
83 | windows in critical sections to be hit. |
---|
84 | |
---|
85 | @subsection Task Affinity |
---|
86 | |
---|
87 | @cindex task affinity |
---|
88 | @cindex thread affinity |
---|
89 | |
---|
90 | RTEMS provides services to manipulate the affinity of a task. Affinity |
---|
91 | is used to specify the subset of processors in an SMP system on which |
---|
92 | a particular task can execute. |
---|
93 | |
---|
94 | By default, tasks have an affinity which allows them to execute on any |
---|
95 | available processor. |
---|
96 | |
---|
97 | Task affinity is a possible feature to be supported by SMP-aware |
---|
98 | schedulers. However, only a subset of the available schedulers support |
---|
99 | affinity. Although the behavior is scheduler specific, if the scheduler |
---|
100 | does not support affinity, it is likely to ignore all attempts to set |
---|
101 | affinity. |
---|
102 | |
---|
103 | @subsection Critical Section Techniques and SMP |
---|
104 | |
---|
105 | As discussed earlier, SMP systems have opportunities for true parallelism |
---|
106 | which was not possible on uniprocessor systems. Consequently, multiple |
---|
107 | techniques that provided adequate critical sections on uniprocessor |
---|
108 | systems are unsafe on SMP systems. In this section, some of these |
---|
109 | unsafe techniques will be discussed. |
---|
110 | |
---|
111 | In general, applications must use proper operating system provided mutual |
---|
112 | exclusion mechanisms to ensure correct behavior. This primarily means |
---|
113 | the use of binary semaphores or mutexes to implement critical sections. |
---|
114 | |
---|
115 | @subsubsection Disable Interrupts |
---|
116 | |
---|
117 | Again on a uniprocessor system, there is only a single processor which |
---|
118 | logically executes a single task and takes interrupts. On an SMP system, |
---|
119 | each processor may take an interrupt. When the application disables |
---|
120 | interrupts, it generally does so by altering a processor register to |
---|
121 | mask interrupts and later to re-enable them. On a uniprocessor system, |
---|
122 | changing this in the single processor is sufficient. However, on an SMP |
---|
123 | system, this register in @strong{ALL} processors must be changed. There |
---|
124 | are no comparable capabilities in an SMP system to disable all interrupts |
---|
125 | across all processors. |
---|
126 | |
---|
127 | @subsubsection Highest Priority Task Assumption |
---|
128 | |
---|
129 | On a uniprocessor system, it is safe to assume that when the highest |
---|
130 | priority task in an application executes, it will execute without being |
---|
131 | preempted until it voluntarily blocks. Interrupts may occur while it is |
---|
132 | executing, but there will be no context switch to another task unless |
---|
133 | the highest priority task voluntarily initiates it. |
---|
134 | |
---|
135 | Given the assumption that no other tasks will have their execution |
---|
136 | interleaved with the highest priority task, it is possible for this |
---|
137 | task to be constructed such that it does not need to acquire a binary |
---|
138 | semaphore or mutex for protected access to shared data. |
---|
139 | |
---|
140 | In an SMP system, it cannot be assumed there will never be a single task |
---|
141 | executing. It should be assumed that every processor is executing another |
---|
142 | application task. Further, those tasks will be ones which would not have |
---|
143 | been executed in a uniprocessor configuration and should be assumed to |
---|
144 | have data synchronization conflicts with what was formerly the highest |
---|
145 | priority task which executed without conflict. |
---|
146 | |
---|
147 | @subsubsection Disable Preemption |
---|
148 | |
---|
149 | On a uniprocessor system, disabling preemption in a task is very similar |
---|
150 | to making the highest priority task assumption. While preemption is |
---|
151 | disabled, no task context switches will occur unless the task initiates |
---|
152 | them voluntarily. And, just as with the highest priority task assumption, |
---|
153 | there are N-1 processors also running tasks. Thus the assumption that no |
---|
154 | other tasks will run while the task has preemption disabled is violated. |
---|
155 | |
---|
156 | @subsection Task Unique Data and SMP |
---|
157 | |
---|
158 | Per task variables are a service commonly provided by real-time operating |
---|
159 | systems for application use. They work by allowing the application |
---|
160 | to specify a location in memory (typically a @code{void *}) which is |
---|
161 | logically added to the context of a task. On each task switch, the |
---|
162 | location in memory is stored and each task can have a unique value in |
---|
163 | the same memory location. This memory location is directly accessed as a |
---|
164 | variable in a program. |
---|
165 | |
---|
166 | This works well in a uniprocessor environment because there is one task |
---|
167 | executing and one memory location containing a task-specific value. But |
---|
168 | it is fundamentally broken on an SMP system because there are always N |
---|
169 | tasks executing. With only one location in memory, N-1 tasks will not |
---|
170 | have the correct value. |
---|
171 | |
---|
172 | This paradigm for providing task unique data values is fundamentally |
---|
173 | broken on SMP systems. |
---|
174 | |
---|
175 | @subsubsection Classic API Per Task Variables |
---|
176 | |
---|
177 | The Classic API provides three directives to support per task variables. These are: |
---|
178 | |
---|
179 | @itemize @bullet |
---|
180 | @item @code{@value{DIRPREFIX}task_variable_add} - Associate per task variable |
---|
181 | @item @code{@value{DIRPREFIX}task_variable_get} - Obtain value of a a per task variable |
---|
182 | @item @code{@value{DIRPREFIX}task_variable_delete} - Remove per task variable |
---|
183 | @end itemize |
---|
184 | |
---|
185 | As task variables are unsafe for use on SMP systems, the use of these |
---|
186 | services should be eliminated in all software that is to be used in |
---|
187 | an SMP environment. It is recommended that the application developer |
---|
188 | consider the use of POSIX Keys or Thread Local Storage (TLS). POSIX Keys |
---|
189 | are not enabled in all RTEMS configurations. |
---|
190 | |
---|
191 | @b{STATUS}: As of March 2014, some support services in the |
---|
192 | @code{rtems/cpukit} use per task variables. When these uses are |
---|
193 | eliminated, the per task variable directives will be disabled when |
---|
194 | building RTEMS in SMP configuration. |
---|
195 | |
---|
196 | @c |
---|
197 | @c |
---|
198 | @c |
---|
199 | @section Operations |
---|
200 | |
---|
201 | @subsection Setting Affinity to a Single Processor |
---|
202 | |
---|
203 | In many embedded applications targeting SMP systems, it is common to lock individual tasks to specific cores. In this way, one can designate a core for I/O tasks, another for computation, etc.. The following illustrates the code sequence necessary to assign a task an affinity for processor zero (0). |
---|
204 | |
---|
205 | @example |
---|
206 | rtems_status_code sc; |
---|
207 | cpu_set_t set; |
---|
208 | |
---|
209 | CPU_EMPTY( &set ); |
---|
210 | CPU_SET( 0, &set ); |
---|
211 | |
---|
212 | sc = rtems_task_set_affinity(rtems_task_self(), sizeof(set), &set); |
---|
213 | assert(sc == RTEMS_SUCCESSFUL); |
---|
214 | @end example |
---|
215 | |
---|
216 | It is important to note that the @code{cpu_set_t} is not validated until the |
---|
217 | @code{@value{DIRPREFIX}task_set_affinity} call is made. At that point, |
---|
218 | it is validated against the current system configuration. |
---|
219 | |
---|
220 | @c |
---|
221 | @c |
---|
222 | @c |
---|
223 | @section Directives |
---|
224 | |
---|
225 | This section details the symmetric multiprocessing services. A subsection |
---|
226 | is dedicated to each of these services and describes the calling sequence, |
---|
227 | related constants, usage, and status codes. |
---|
228 | |
---|
229 | @c |
---|
230 | @c rtems_get_processor_count |
---|
231 | @c |
---|
232 | @page |
---|
233 | @subsection GET_PROCESSOR_COUNT - Get processor count |
---|
234 | |
---|
235 | @subheading CALLING SEQUENCE: |
---|
236 | |
---|
237 | @ifset is-C |
---|
238 | @example |
---|
239 | uint32_t rtems_get_processor_count(void); |
---|
240 | @end example |
---|
241 | @end ifset |
---|
242 | |
---|
243 | @ifset is-Ada |
---|
244 | @end ifset |
---|
245 | |
---|
246 | @subheading DIRECTIVE STATUS CODES: |
---|
247 | |
---|
248 | The count of processors in the system. |
---|
249 | |
---|
250 | @subheading DESCRIPTION: |
---|
251 | |
---|
252 | On uni-processor configurations a value of one will be returned. |
---|
253 | |
---|
254 | On SMP configurations this returns the value of a global variable set during |
---|
255 | system initialization to indicate the count of utilized processors. The |
---|
256 | processor count depends on the physically or virtually available processors and |
---|
257 | application configuration. The value will always be less than or equal to the |
---|
258 | maximum count of application configured processors. |
---|
259 | |
---|
260 | @subheading NOTES: |
---|
261 | |
---|
262 | None. |
---|
263 | |
---|
264 | @c |
---|
265 | @c rtems_get_current_processor |
---|
266 | @c |
---|
267 | @page |
---|
268 | @subsection GET_CURRENT_PROCESSOR - Get current processor index |
---|
269 | |
---|
270 | @subheading CALLING SEQUENCE: |
---|
271 | |
---|
272 | @ifset is-C |
---|
273 | @example |
---|
274 | uint32_t rtems_get_current_processor(void); |
---|
275 | @end example |
---|
276 | @end ifset |
---|
277 | |
---|
278 | @ifset is-Ada |
---|
279 | @end ifset |
---|
280 | |
---|
281 | @subheading DIRECTIVE STATUS CODES: |
---|
282 | |
---|
283 | The index of the current processor. |
---|
284 | |
---|
285 | @subheading DESCRIPTION: |
---|
286 | |
---|
287 | On uni-processor configurations a value of zero will be returned. |
---|
288 | |
---|
289 | On SMP configurations an architecture specific method is used to obtain the |
---|
290 | index of the current processor in the system. The set of processor indices is |
---|
291 | the range of integers starting with zero up to the processor count minus one. |
---|
292 | |
---|
293 | Outside of sections with disabled thread dispatching the current processor |
---|
294 | index may change after every instruction since the thread may migrate from one |
---|
295 | processor to another. Sections with disabled interrupts are sections with |
---|
296 | thread dispatching disabled. |
---|
297 | |
---|
298 | @subheading NOTES: |
---|
299 | |
---|
300 | None. |
---|
301 | |
---|
302 | @c |
---|
303 | @c rtems_scheduler_ident |
---|
304 | @c |
---|
305 | @page |
---|
306 | @subsection SCHEDULER_IDENT - Get ID of a scheduler |
---|
307 | |
---|
308 | @subheading CALLING SEQUENCE: |
---|
309 | |
---|
310 | @ifset is-C |
---|
311 | @example |
---|
312 | rtems_status_code rtems_scheduler_ident( |
---|
313 | rtems_name name, |
---|
314 | rtems_id *id |
---|
315 | ); |
---|
316 | @end example |
---|
317 | @end ifset |
---|
318 | |
---|
319 | @ifset is-Ada |
---|
320 | @end ifset |
---|
321 | |
---|
322 | @subheading DIRECTIVE STATUS CODES: |
---|
323 | |
---|
324 | @code{@value{RPREFIX}SUCCESSFUL} - successful operation@* |
---|
325 | @code{@value{RPREFIX}INVALID_ADDRESS} - @code{id} is NULL@* |
---|
326 | @code{@value{RPREFIX}INVALID_NAME} - invalid scheduler name |
---|
327 | |
---|
328 | @subheading DESCRIPTION: |
---|
329 | |
---|
330 | Identifies a scheduler by its name. The scheduler name is determined by the |
---|
331 | scheduler configuration. @xref{Configuring a System Configuring |
---|
332 | Clustered/Partitioned Schedulers}. |
---|
333 | |
---|
334 | @subheading NOTES: |
---|
335 | |
---|
336 | None. |
---|
337 | |
---|
338 | @c |
---|
339 | @c rtems_scheduler_get_processor_set |
---|
340 | @c |
---|
341 | @page |
---|
342 | @subsection SCHEDULER_GET_PROCESSOR_SET - Get processor set of a scheduler |
---|
343 | |
---|
344 | @subheading CALLING SEQUENCE: |
---|
345 | |
---|
346 | @ifset is-C |
---|
347 | @example |
---|
348 | rtems_status_code rtems_scheduler_get_processor_set( |
---|
349 | rtems_id scheduler_id, |
---|
350 | size_t cpusetsize, |
---|
351 | cpu_set_t *cpuset |
---|
352 | ); |
---|
353 | @end example |
---|
354 | @end ifset |
---|
355 | |
---|
356 | @ifset is-Ada |
---|
357 | @end ifset |
---|
358 | |
---|
359 | @subheading DIRECTIVE STATUS CODES: |
---|
360 | |
---|
361 | @code{@value{RPREFIX}SUCCESSFUL} - successful operation@* |
---|
362 | @code{@value{RPREFIX}INVALID_ADDRESS} - @code{cpuset} is NULL@* |
---|
363 | @code{@value{RPREFIX}INVALID_ID} - invalid scheduler id@* |
---|
364 | @code{@value{RPREFIX}INVALID_NUMBER} - the affinity set buffer is too small for |
---|
365 | set of processors owned by the scheduler |
---|
366 | |
---|
367 | @subheading DESCRIPTION: |
---|
368 | |
---|
369 | Returns the processor set owned by the scheduler in @code{cpuset}. A set bit |
---|
370 | in the processor set means that this processor is owned by the scheduler and a |
---|
371 | cleared bit means the opposite. |
---|
372 | |
---|
373 | @subheading NOTES: |
---|
374 | |
---|
375 | None. |
---|
376 | |
---|
377 | @c |
---|
378 | @c rtems_task_get_scheduler |
---|
379 | @c |
---|
380 | @page |
---|
381 | @subsection TASK_GET_SCHEDULER - Get scheduler of a task |
---|
382 | |
---|
383 | @subheading CALLING SEQUENCE: |
---|
384 | |
---|
385 | @ifset is-C |
---|
386 | @example |
---|
387 | rtems_status_code rtems_task_get_scheduler( |
---|
388 | rtems_id id, |
---|
389 | rtems_id *scheduler_id |
---|
390 | ); |
---|
391 | @end example |
---|
392 | @end ifset |
---|
393 | |
---|
394 | @ifset is-Ada |
---|
395 | @end ifset |
---|
396 | |
---|
397 | @subheading DIRECTIVE STATUS CODES: |
---|
398 | |
---|
399 | @code{@value{RPREFIX}SUCCESSFUL} - successful operation@* |
---|
400 | @code{@value{RPREFIX}INVALID_ADDRESS} - @code{scheduler_id} is NULL@* |
---|
401 | @code{@value{RPREFIX}INVALID_ID} - invalid task id |
---|
402 | |
---|
403 | @subheading DESCRIPTION: |
---|
404 | |
---|
405 | Returns the scheduler identifier of a task in @code{scheduler_id}. |
---|
406 | |
---|
407 | @subheading NOTES: |
---|
408 | |
---|
409 | None. |
---|
410 | |
---|
411 | @c |
---|
412 | @c rtems_task_get_affinity |
---|
413 | @c |
---|
414 | @page |
---|
415 | @subsection TASK_GET_AFFINITY - Get task processor affinity |
---|
416 | |
---|
417 | @subheading CALLING SEQUENCE: |
---|
418 | |
---|
419 | @ifset is-C |
---|
420 | @example |
---|
421 | rtems_status_code rtems_task_get_affinity( |
---|
422 | rtems_id id, |
---|
423 | size_t cpusetsize, |
---|
424 | cpu_set_t *cpuset |
---|
425 | ); |
---|
426 | @end example |
---|
427 | @end ifset |
---|
428 | |
---|
429 | @ifset is-Ada |
---|
430 | @end ifset |
---|
431 | |
---|
432 | @subheading DIRECTIVE STATUS CODES: |
---|
433 | |
---|
434 | @code{@value{RPREFIX}SUCCESSFUL} - successful operation@* |
---|
435 | @code{@value{RPREFIX}INVALID_ADDRESS} - @code{cpuset} is NULL@* |
---|
436 | @code{@value{RPREFIX}INVALID_ID} - invalid task id@* |
---|
437 | @code{@value{RPREFIX}INVALID_NUMBER} - the affinity set buffer is too small for |
---|
438 | the current processor affinity set of the task |
---|
439 | |
---|
440 | @subheading DESCRIPTION: |
---|
441 | |
---|
442 | Returns the current processor affinity set of the task in @code{cpuset}. A set |
---|
443 | bit in the affinity set means that the task can execute on this processor and a |
---|
444 | cleared bit means the opposite. |
---|
445 | |
---|
446 | @subheading NOTES: |
---|
447 | |
---|
448 | None. |
---|
449 | |
---|
450 | @c |
---|
451 | @c rtems_task_set_affinity |
---|
452 | @c |
---|
453 | @page |
---|
454 | @subsection TASK_SET_AFFINITY - Set task processor affinity |
---|
455 | |
---|
456 | @subheading CALLING SEQUENCE: |
---|
457 | |
---|
458 | @ifset is-C |
---|
459 | @example |
---|
460 | rtems_status_code rtems_task_set_affinity( |
---|
461 | rtems_id id, |
---|
462 | size_t cpusetsize, |
---|
463 | const cpu_set_t *cpuset |
---|
464 | ); |
---|
465 | @end example |
---|
466 | @end ifset |
---|
467 | |
---|
468 | @ifset is-Ada |
---|
469 | @end ifset |
---|
470 | |
---|
471 | @subheading DIRECTIVE STATUS CODES: |
---|
472 | |
---|
473 | @code{@value{RPREFIX}SUCCESSFUL} - successful operation@* |
---|
474 | @code{@value{RPREFIX}INVALID_ADDRESS} - @code{cpuset} is NULL@* |
---|
475 | @code{@value{RPREFIX}INVALID_ID} - invalid task id@* |
---|
476 | @code{@value{RPREFIX}INVALID_NUMBER} - invalid processor affinity set |
---|
477 | |
---|
478 | @subheading DESCRIPTION: |
---|
479 | |
---|
480 | Sets the processor affinity set for the task specified by @code{cpuset}. A set |
---|
481 | bit in the affinity set means that the task can execute on this processor and a |
---|
482 | cleared bit means the opposite. |
---|
483 | |
---|
484 | @subheading NOTES: |
---|
485 | |
---|
486 | None. |
---|