Changes between Version 88 and Version 89 of Developer/SMP


Ignore:
Timestamp:
Feb 16, 2015, 2:59:00 PM (5 years ago)
Author:
Sebastian Huber
Comment:

Fix layout

Legend:

Unmodified
Added
Removed
Modified
  • Developer/SMP

    v88 v89  
    1 = SMP =
    2 
    3 
    4 
    5 [[TOC(Developer/SMP, depth=2)]]
     1[[TOC(Developer/SMP, depth=3)]]
    62
    73=  Status of Effort  =
     
    117113=  Design Issues  =
    118114
    119 =  Low-Level Start  =
    120 
    121 = Status =
     115==  Low-Level Start  ==
     116
     117=== Status ===
    122118
    123119
     
    130126
    131127
    132 = Future Directions =
     128=== Future Directions ===
    133129
    134130
    135131None.
    136 =  Processor Affinity  =
    137 
    138 =  Status  =
     132==  Processor Affinity  ==
     133
     134===  Status  ===
    139135
    140136
     
    142138
    143139The APIs for <sys/cpuset.h>, pthread affinity, and Classic API task affinity have been community reviewed and patches have been posted. The current work is "data complete" in that one can get and set using the APIs but no scheduler currently supports affinity.
    144 =  Background  =
     140===  Background  ===
    145141
    146142
     
    161157
    162158Work remains to be done on adding affinity to a new SMP Scheduler variant. The current plan calls for having the schedulers without affinity ignore input on a set affinity operation and return a cpu set with all CPUs set on all get affinity operations. A new SMP scheduler variant will be added to support affinity. Initially, this scheduler will only be sufficient to properly manage the thread's CPU set as a data element. Then an assignment algorithm will be added.
    163 =  Theory  =
     159===  Theory  ===
    164160
    165161
     
    253249[[Image(rtems-smp-affinity.png)]]
    254250
    255 =  Atomic Operations  =
    256 
    257 =  Status  =
     251==  Atomic Operations  ==
     252
     253===  Status  ===
    258254
    259255
     
    323319        .size   release, .-release
    324320        .ident  "GCC: (GNU) 4.9.0 20130917 (experimental)"
    325 =  Future Directions  =
     321=== Future Directions ===
    326322
    327323
    328324 *  Review and integrate the GSoC work.
    329325 *  Make use of atomic operations.
    330 =  SMP Locks  =
    331 
    332 =  Status  =
     326==  SMP Locks  ==
     327
     328===  Status  ===
    333329
    334330
    335331SMP locks are implemented as a ticket lock using CPU architecture specific atomic operations.  The SMP locks use a local context to be able to use scalable lock implementations like the Mellor-Crummey and Scotty (MCS) queue-based locks.
    336 =  Future Directions  =
     332=== Future Directions ===
    337333
    338334
    339335Introduce read-write locks.  Use phase-fair read-write lock implementation.  This can be used for example by the time management code.  The system time may be read frequently, but updates are infrequent.
    340 =  ISR Locks  =
    341 
    342 =  Status  =
     336==  ISR Locks  ==
     337
     338===  Status  ===
    343339
    344340
     
    349345on single processor configurations.  On SMP configurations they use an SMP lock
    350346to ensure mutual exclusion throughout the system.
    351 =  Future Directions  =
     347=== Future Directions ===
    352348
    353349
    354350 *  See [wiki:#SMP_Locks  SMP locks].
    355351 *  Ensure via a RTEMS assertion that normal interrupt disable/sequences are only used intentional outside of the Giant lock critical sections.  Review usage of ISR disable/enable sequences of the complete code base.
    356 =  Giant Lock vs. Fine Grained Locking  =
    357 
    358 =  Status  =
     352==  Giant Lock vs. Fine Grained Locking  ==
     353
     354===  Status  ===
    359355
    360356
     
    383379likely lead to undesirable high worst case latencies which linear increase with
    384380the processor count <ref name="Brandenburg2011">[https://www.cs.unc.edu/~anderson/diss/bbbdiss.pdf Björn B. Brandenburg, Scheduling and Locking in Multiprocessor Real-Time Operating Systems, 2011.]</ref>.
    385 =  Future Directions  =
     381=== Future Directions ===
    386382
    387383
     
    421417Thus the scheduler needs its own locks.  This will lead to nested locking and
    422418deadlocks must be prevented.
    423 =  Watchdog Handler  =
    424 
    425 =  Status  =
     419==  Watchdog Handler  ==
     420
     421===  Status  ===
    426422
    427423
     
    429425RTEMS.  It is a core service of the operating system and used by the high level APIs.  It is used for time of day maintenance and time events based on ticks or
    430426wall clock time for example.  The term ''watchdog'' has nothing to do with hardware watchdogs.
    431 =  Future Directions  =
     427=== Future Directions ===
    432428
    433429
     
    435431watchdog resource leads to scalability problems.  A SMP system is scalable if additional processors will increase the performance metrics.  If we add more processors, then in general the number of timers increases proportionally and thus leads to longer delta chains.  Also the chance for lock contention increases.  Thus the watchdog resource should
    436432move to the scheduler scope with per scheduler instances.  Additional processors can then use new scheduler instances.
    437 =  Per-CPU Control  =
    438 
    439 =  Status  =
     433==  Per-CPU Control  ==
     434
     435===  Status  ===
    440436
    441437
     
    445441code.  CPU ports can add custom fields to the per-CPU control.  This is used on
    446442SPARC for the ISR dispatch disable indication.
    447 =  Future Directions  =
     443=== Future Directions ===
    448444
    449445
    450446None.
    451 =  Interrupt Support  =
    452 
    453 =  Status  =
     447==  Interrupt Support  ==
     448
     449===  Status  ===
    454450
    455451
    456452The interrupt support is BSP specific in general.
    457 =  Future Directions  =
     453=== Future Directions ===
    458454
    459455
    460456 *  The SMP capable BSPs should implement the [http://www.rtems.org/onlinedocs/doxygen/cpukit/html/group__rtems__interrupt__extension.html Interrupt Manager Extension].
    461457 *  Add interrupt processor affinity API to ''Interrupt Manager Extension''.  This should use the affinity sets API used for thread processor affinity.
    462 =  Global Scheduler  =
    463 
    464 =  Status  =
     458==  Global Scheduler  ==
     459
     460===  Status  ===
    465461
    466462
     
    468464 *  Basic SMP support for schedulers like processor allocation and extended thread state information.
    469465 *  Two global fixed priority schedulers (G-FP) are available with SMP support.
    470 =  Future Directions  =
     466=== Future Directions ===
    471467
    472468
     
    477473 *  The new SMP schedulers lack test coverage.
    478474 *  Implement clustered (or partitioned) scheduling with a sophisticated resource sharing protocol (e.g. <ref name="BurnsWellings2013">A. Burns and A.J. Wellings, A Schedulability Compatible Multiprocessor Resource Sharing Protocol - MrsP, Proceedings of the 25th Euromicro Conference on Real-Time Systems (ECRTS 2013), July 2013. </ref>).  To benefit from this the Giant lock must be eliminated.
    479 =  Clustered Scheduling  =
    480 
    481 =  Status  =
     475==  Clustered Scheduling  ==
     476
     477===  Status  ===
    482478
    483479
    484480Clustered scheduling is not supported.
    485 =  Future Directions  =
     481=== Future Directions ===
    486482
    487483
     
    512508application-level threads in global resources - not the full task, only that
    513509''protected'' execution - in accord with <ref name="BurnsWellings2013"/>.
    514 =  Task Variables  =
    515 
    516 =  Status  =
     510==  Task Variables  ==
     511
     512===  Status  ===
    517513
    518514
     
    521517of the executing thread.  For Newlib the access to the re-entrancy structure is
    522518now performed via ''__getreent()'',  see also ''__DYNAMIC_REENT__'' in Newlib.  The POSIX keys and the POSIX once function are now available for all RTEMS configurations (they no longer depend on POSIX enabled).  Task variables have been replaced with POSIX keys for the RTEMS shell, the file system environment and the C++ support.
    523 =  Future Directions  =
     519=== Future Directions ===
    524520
    525521
    526522 *  Fix Ada self task reference.
    527 =  Non-Preempt Mode for Mutual Exclusion  =
    528 
    529 =  Status  =
     523==  Non-Preempt Mode for Mutual Exclusion  ==
     524
     525===  Status  ===
    530526
    531527
     
    534530a mode change request is issued.  The alternatives are mutexes and condition
    535531variables.
    536 =  Future Directions  =
     532=== Future Directions ===
    537533
    538534
     
    541537 *  Fix the block device cache ''bdbuf''.
    542538 *  Fix C++ support.
    543 =  Thread Restart  =
    544 
    545 =  Status  =
     539==  Thread Restart  ==
     540
     541===  Status  ===
    546542
    547543
    548544The restart of threads is implemented.
    549 =  Future Directions  =
     545=== Future Directions ===
    550546
    551547
     
    572568
    573569This scheme could be used for all architectures.
    574 =  Thread Delete  =
    575 
    576 =  Status  =
     570==  Thread Delete  ==
     571
     572=== Status ===
    577573
    578574
    579575Deletion of threads is implemented.
    580 =  Future Directions  =
     576=== Future Directions ===
    581577
    582578
    583579None.
    584 =  Semaphores and Mutexes  =
    585 
    586 =  Status  =
     580==  Semaphores and Mutexes  ==
     581
     582=== Status ===
    587583
    588584
     
    590586configurations this first acquires the Giant lock and then interrupts are
    591587disabled.
    592 =  Future Directions  =
     588=== Future Directions ===
    593589
    594590
     
    597593=  Implementations  =
    598594
    599 =  Tool Chain  =
    600 
    601 =  Binutils  =
     595==  Tool Chain  ==
     596
     597===  Binutils  ===
    602598
    603599
    604600A Binutils 2.24 or later release must be used due to the LEON3 support.
    605 =  GCC  =
     601===  GCC  ===
    606602
    607603
    608604A GCC 4.8 2013-11-25 (e.g. 4.8.3) or later must be used due to the LEON3 support.  The LEON3 support for GCC includes a proper C11 memory model definition for this processor and C11 atomic operations using the CAS instruction.  The backport of the LEON3 support was initiated by EB [http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02255.html].
    609 =  GDB  =
    610 
    611 
    612 Current GDB versions have problems with the debug format generated by GCC 4.8 and later [https://sourceware.org/bugzilla/show_bug.cgi?id=16215].
    613 =  Profiling  =
    614 
    615 =  Reason  =
     605===  GDB  ===
     606
     607A GDB 7.8.2 or later must be used since earlier versions have problems with the
     608debug format generated by GCC 4.8 and later
     609[https://sourceware.org/bugzilla/show_bug.cgi?id=16215].
     610
     611==  Profiling  ==
     612
     613=== Implementation ===
    616614
    617615
     
    619617observability requirements.  Vital timing data can be gathered on a per object
    620618basis through profiling.
    621 =  Status  =
     619=== Status ===
    622620
    623621
    624622CPU counter support is complete [http://git.rtems.org/rtems/commit/?id=24bf11eca11947d961cc9bb5f7d92dabff169e93].  The profiling support is complete [http://git.rtems.org/rtems/commit/?id=4dad4b84112d57cf6e77409f8e267706db446ec0].
    625 =  RTEMS API Changes  =
     623=== RTEMS API Changes ===
    626624
    627625
    628626None.
    629 =  High-Performance CPU Counters  =
     627===  High-Performance CPU Counters  ===
    630628
    631629
     
    656654  */
    657655 uint64_t _CPU_counter_To_nanoseconds( CPU_counter counter )
    658 =  SMP Lock Profiling  =
     656===  SMP Lock Profiling  ===
    659657
    660658
     
    780778  */
    781779 void _SMP_lock_Iterate( SMP_lock_Visitor visitor, void *arg );
    782 =  Interrupt and Thread Profiling  =
     780===  Interrupt and Thread Profiling  ===
    783781
    784782
     
    865863   Per_CPU_Stats Stats;
    866864 };
    867 =  Interrupt Support  =
    868 
    869 =  Reason  =
     865==  Interrupt Support  ==
     866
     867=== Implementation ===
    870868
    871869
     
    873871system.  In combination with partitioned/clustered scheduling this can reduce
    874872the amount of inter-processor synchronization and thread migrations.
    875 =  Status  =
     873=== Status ===
    876874
    877875
    878876This is TBD.
    879 =  RTEMS API Changes  =
     877=== RTEMS API Changes ===
    880878
    881879
     
    931929   cpu_set_t *affinity_set
    932930 );
    933 =  Clustered Scheduling  =
    934 
    935 =  Reason  =
     931==  Clustered Scheduling  ==
     932
     933=== Implementation ===
    936934
    937935
     
    942940scheduling it is possible to honor the cache topology of a system and thus
    943941avoid expensive cache synchronization traffic.
    944 =  Status  =
     942=== Status ===
    945943
    946944
    947945Support for clustered/partitioned scheduling is complete [http://git.rtems.org/rtems/commit/?id=c5831a3f9af11228dbdaabaf01f69d37e55684ef].
    948 =  RTEMS API Changes  =
     946=== RTEMS API Changes ===
    949947
    950948
     
    10751073   rtems_id *scheduler_id
    10761074 );
    1077 =  Scheduler Configuration  =
     1075===  Scheduler Configuration  ===
    10781076
    10791077
     
    11671165configuration approach enables an additional error source which is avoided by
    11681166the processor configuration table.
    1169 =  Scheduler Implementation  =
     1167=== Scheduler Implementation  ===
    11701168
    11711169
     
    12131211all dependencies explicit and easier to access (allows more efficient machine
    12141212code).
    1215 =  Multiprocessor Resource Sharing Protocol - MrsP  =
    1216 
    1217 =  Reason  =
     1213==  Multiprocessor Resource Sharing Protocol - MrsP  ==
     1214
     1215=== Implementation ===
    12181216
    12191217
     
    12421240Feature (a) prevents lower-priority threads from running in preference to the waiting higher-priority thread and stealing resources that it might want to use in the future as part of the current execution; should that stealing happen, the blocking penalty potentially suffered on access to global resources would skyrocket to untenable levels.
    12431241Feature (b), which brings in the sole welcome extent of migration in the proposed model, which is useful when higher-priority tasks running on the processor of the global resource prevent it from completing execution; in that case, the slack allowed for by local spinning on other processors where other threads are waiting, is used to speed up the completion of the execution in the global resource and therefore reduce blocking.
    1244 =  Status  =
     1242=== Status ===
    12451243
    12461244
    12471245Implementation is complete [http://git.rtems.org/rtems/commit/?id=5c3d2509593476869e791111cd3d93cc1e840b3a].
    1248 =  RTEMS API Changes  =
     1246=== RTEMS API Changes ===
    12491247
    12501248
     
    12921290   size_t priority_ceilings_count
    12931291 );
    1294 =  Implementation  =
     1292===  Implementation  ===
    12951293
    12961294
     
    13011299which is executing.  A resource release will remove this thread from the
    13021300temporary scheduler domain and move it back to the original scheduler domain.
    1303 =  Fine Grained Locking  =
    1304 
    1305 =  Reason  =
     1301==  Fine Grained Locking  ==
     1302
     1303=== Implementation ===
    13061304
    13071305
     
    13151313the operating system state is distributed allowing true parallelism of
    13161314independent components.
    1317 =  Status  =
     1315=== Status ===
    13181316
    13191317
    13201318This is TBD.
    1321 =  RTEMS API Changes  =
     1319=== RTEMS API Changes ===
    13221320
    13231321
    13241322None.
    1325 =  Locking Protocol Analysis  =
     1323===  Locking Protocol Analysis  ===
    13261324
    13271325
     
    14581456                                wait_queue.set_sync_status(TIMEOUT)
    14591457                                wait_control = executing.get_wait_control()
    1460                                 wait_control.set_status(TIMEOUT)
    1461                         <span style="color:red">ISR_enable(level)</span>
    1462                 else:
    1463                         <span style="color:red">ISR_enable(level)</span>
    1464                         <span style
     1458                                wai