wiki:Developer/Projects/SequencedInitialization

RTEMSSequencedInitialization

Table of Contents

    Status: Complete, see also #2408.

    Mentor: Chris Johns and Sebastian Huber

    Student: Wenjie Zhang

    Note: This project was called RTEMS Initialization By Constructor.

    Overview

    This project adds to RTEMS a Sequencer that calls user defined functions held in an unordered table in a specific order. An example of its use is the RTEMS Managers initialization. The initialisation of the RTEMS Managers is a specific ordered sequence of calls currently hard coded into RTEMS. The sequencing code will take a table of nodes that is un-order, determine the order and make calls to the user provided functions which in this case is the manager initialisation calls.

    The solution should be general enough to support RTEMS managers, drivers and BSD sysctl type nodes. The result of this work is expected to impact the TinyRTEMS effort as it would provide a central mechanism to automatically eliminate unused code. If you don't reference a part of the RTEMS API the initialization code would automatically drop out. The dropping of unused code is critical to both the TinyRTEMS effort and to those woh use RTEMS in safety-critical systems which must be validated. The automatic part here refers to the creation of the table of nodes.

    A prototype of the work can be found in PR1253. This work combines the call sequencing with the automatic generation of the table of nodes. The solution is not suitable as it uses the word "constructor" which we have since decided should not be used, as well as too much RAM. The ideal solution should use little if any RAM and all nodes and tables be held in ROM type memory. A typical small foot print CPU which would use a TinyRTEMS solution has much more ROM than RAM. Also this is a once off initialization procedure and so using RAM means it is gone from application use.

    The automatic table generation uses the same linker technique used to create C++ static object constructor and destructor tables. Consider the RTEMS Message API. In the Message Manager's initialization call's source file you create a {{{static const</code> structure with the specific fields needed by the sequencer code then reference this variable from the source file containing the {{{rtems_message_queue_create</code> code. The application must make a call to {{{rtems_message_queue_create</code> or all the other Message API calls will fail therefore this API needs only one reference from a single source file. The application call of {{{rtems_message_queue_create</code> will pull in its code and as this code references the node in the initialization call's file it is also pulled in along with the initialization code. We also create a variable that is a pointer to the sequencer node and this is placed in a special section using the GCC "attribute" modifier. The linker command file for each BSP must be modified to group all these pointers together in one location in ROM. This creates the table that is passed to the sequencer code.

    The sequencer code iterates over the table calling entries in the order specified. The order could be a number or it could be relative. The relative order design makes for a more robust system because you have moved away from specific numbers. The idea here is to allow high level ordering operators, for example "first", "last", "after", "before", "just after", "just before", etc. You can then say "message" is "after" "heap" and the order is determined at runtime. Most system parts are relative not absolute. It may even be possible to allow relative and absolute ordering to be mixed. Priorities are similar. At the end of the day we do not need 256 levels if we only use 4. Typically all we need is to say is this task is higher or lower than another task. Sequencing ca viewed the same way.

    Work Breakdown

    This project tasks are: # Develop the sequencer code and API with documentation and tests. # Cleanup the linker command files on a per architecture basis to use a general purpose base linker command file and a board specific linker command file. # Update all linker command files to handle the initialization automatic table entries. # Update the RTEMS Classic API, POSIX API and ITRON API initialization calls with the sequencer implementation using automatic table generation. This also includes removal of the hard coded call with a sequencer call. # Initial testing can all be done on SPARC/sis. The modification of other BSP linkcmds should be done with caution but mistakes will be caught by Test Builds of RTEMS by JoelSherrill which include all BSP configurations.

    Code and data size measurements should be made before and after the implementation is done on the sparc/sis and arm/rtl22xx_t BSPs. These BSPs are used for code size analysis across releases.

    Implementation Details

    Definitions

    ; Input Section : Collection of input items identified by a section name for the linker. ; Output Section : Output part of the linker. ; Linker Set : An input section that contains only identical items with respect to size and layout.

    Sequenced initialization requires support by the linker. The general approach is to put certain items identified by an input section into a special output section and mark the begin and end of this section.

    FreeBSD Approach

    FreeBSD uses a fixed linker set which contains arbitrarily ordered system initialization items. Each item has an order field which is used to sort the items during runtime. The initialization functions will be call with respect to this order.

    • [PRO] The order information is stored in the item and can evaluated on a per item basis.
    • [PRO] Flexible order granularity.
    • [CON] The order information requires space.
    • [CON] Sorting at runtime requires space and time.

    Linux Approach

    Linux uses several linker sets which contain arbitrarily ordered system initialization items. The linker sets have a hard coded order defined in the linker command file.

    • [PRO] No space overhead for the order information.
    • [PRO] No runtime and space overhead to sort the items.
    • [CON] Very limited order granularity.
    • [CON] Relative huge code section in the linker command file. Hard to maintain.
    • [CON] Order information must be derived from the place in the linker sets. It is impossible to derive the order value specified at compile time.

    RTEMS Approach

    RTEMS should use a fixed linker set which contains ordered system initialization items. The order will be defined by the input section name. The GNU linker is able to sort input sections by name.

    • [PRO] No space overhead for the order information.
    • [PRO] No runtime and space overhead to sort the items.
    • [PRO] Flexible order granularity.
    • [CON] Depends on GNU linker feature.
    • [CON] Order information must be derived from the place in the linker set. It is impossible to derive the order value specified at compile time.

    Linker command file example:

    .rtems : {

    . = ALIGN(8); /* The alignment value depends on the linker set items. */ rtems_sysinit_begin = .; *(SORT(.rtems.sysinit.*)) rtems_sysinit_end = .;

    }

    Header file example:

    #define RTEMS_SYSINIT_MAKE_STRING(str) #str

    #define RTEMS_SYSINIT(group, func) \

    static const uintptr_t \ rtems_sysinit_ ## func \ attribute((section(".rtems.sysinit." RTEMS_SYSINIT_MAKE_STRING(group)))) \ attribute((used)) \

    (uintptr_t) &func

    extern uintptr_t rtems_sysinit_begin [];

    extern uintptr_t rtems_sysinit_end [];

    Initialization and Termination

    The sequenced initialization can be used also for termination. We have two alternatives here:

    # Add a second linker set for termination items. # Use the initialization linker set and pass a state parameter to the initialization function. The initialization functions should be called in reversed initialization order for termination.

    We should use the second approach. This is more flexible and reduces the code inside the linker command files. Linker command files are hard to maintain.

    Proposed API:

    typedef enum {

    RTEMS_SYSINIT_STARTUP, RTEMS_SYSINIT_TERMINATE

    } rtems_sysinit_state;

    typedef void (*rtems_sysinit_function)(rtems_sysinit_state state);

    Error Handling

    • What should happen if an error occurs during initialization or termination?
    • Should we do the error handling on a per function basis?
    • Should we use a return status code and let the initialization sequencer decide what to do?

    Linker Set Construction

    What action will add an item to the linker set?

    # We have an explicit reference to an object which contains a linker set item. # We add an object which contains a linker set item to our binary.

    Since RTEMS is used as a library the first action is trivial.

    To trigger the second action may be difficult. Some objects may perform basic actions during system initialization but are not referenced by the application for example the board support package dependent startup. How can we ensure that these objects will be part of the application? Normal kernels like FreeBSD or Linux are a collection of objects and not a library. The build process determines the set of kernel objects.

    C++ Support

    It should be possible to initialize global references to C++ objects via the sequenced initialization mechanism.

    Example:

    #include <new>

    static char blob [sizeof(T)];

    T &obj = *reinterpret_cast<T *>(blob);

    static void init_obj(rtems_sysinit_state state) {

    switch (state) {

    case RTEMS_SYSINIT_STARTUP:

    new (blob) T(1234567890); break;

    case RTEMS_SYSINIT_TERMINATE:

    obj.~T(); break;

    default:

    break;

    }

    }

    RTEMS_SYSINIT(0, init_obj);

    Open Projects

    Summary of the irc meeting

    Last week Sebhub, Chris, Joel and me make a irc meeting to discuss the implement of Rtems Sequenced Initialization API. Here i make a summary for this meeting:

    First we talked about the disadvantage of run time sort apporach. This solution needs space to store the order information and the code to sort the items, so it is not suitable for the embedded system which has little RAM. Then the practicability of the link time sort approach was discussed. This solution exploits the GNU ld sort feature to order the Sequenced Initialization information in the specifical section. Because it is a lexicographical sort we can define domains of ordering to make a combination term. Such as "rtems_seq_0002" or "rtems_seq_0020", there are not limited in the number of domains. We can also add a group so we can manage the list a little better, for example in a group we define 0->20 for uses and 100-?? for others. By the way because of the diffcult of maintainance we discard the "Linux Approach". In the end there is also a problem waiting for a appropriate solution to fix it. The problem is how to reference the sysinit_core struct which contains the information of Sequenced Initialization. For example: the application use RTEMS Event and RTEMS Signal API there is no similar call like rtems_message_queue_creat which the application must call or it will fail. So we must consider another method to solve this problem.

    Components of the API

    We use a combination of three domains to define the order of initialization sequence, the two domains are (group, index). Every domain is an key word to sort.

    The fields of Sysinit_Entry may be extended in the future, now there is only one field filling in the structure. The field handler is the entry function of RTEMS sequenced initialization or termination.

    /

    • @brief Sysinit state. */

    typedef enum {

    SYSINIT_INITIALIZE, SYSINIT_TERMINATE

    } Sysinit_State;

    /

    • @brief Sysinit handler type. */

    typedef void ( *Sysinit_Handler )( Sysinit_State state );

    /

    • @brief Sysinit linker set entry. */

    typedef struct {

    Sysinit_Handler handler;

    } Sysinit_Entry;

    #define SYSINIT_MAKE_ORDER(group, index) \

    group "." index

    #define SYSINIT_MAKE_INDEX(i3, i2, i1, i0) \

    #i3 #i2 #i1 #i0

    #define SYSINIT_INDEX_FIRST SYSINIT_MAKE_INDEX(0, 0, 0, 0) #define SYSINIT_INDEX_MIDDLE SYSINIT_MAKE_INDEX(5, 0, 0, 0) #define SYSINIT_INDEX_LAST SYSINIT_MAKE_INDEX(9, 9, 9, 9)

    #define SYSINIT_GROUP_FIRST "0000" #define SYSINIT_GROUP_EVENT "0110" #define SYSINIT_GROUP_SEMAPHORE "0120" #define SYSINIT_GROUP_LAST "9999"

    #define SYSINIT_ENTRY_NAME(handler) \

    _Sysinit_Entry_ ## handler

    #define SYSINIT_REFERENCE_NAME(handler) \

    _Sysinit_Reference_ ## handler

    #define SYSINIT_DECLARE_ENTRY(handler) \

    extern const Sysinit_Entry SYSINIT_ENTRY_NAME(handler)

    #define SYSINIT_ENTRY_WITH_INDEX(handler, group, index) \

    SYSINIT_DECLARE_ENTRY(handler); \ const Sysinit_Entry SYSINIT_ENTRY_NAME(handler) \ attribute((section(".score.sysinit." SYSINIT_MAKE_ORDER(group, index)))) \

    { handler }

    #define SYSINIT_ENTRY(handler, group) \

    SYSINIT_ENTRY_WITH_INDEX(handler, group, SYSINIT_INDEX_MIDDLE)

    #define SYSINIT_REFERENCE(handler) \

    SYSINIT_DECLARE_ENTRY(handler); \ static const Sysinit_Entry * const SYSINIT_REFERENCE_NAME(handler) \ attribute((used)) \

    &SYSINIT_ENTRY_NAME(handler)

    /

    • @brief Iterates through all entries of the system initialization set. *
    • Calls each handler with the @a state parameter. */

    void _Sysinit_Iterate(Sysinit_State state);

    /

    • @brief Iterates through all entries of the system initialization set in
    • reversed order. *
    • Calls each handler with the @a state parameter. */

    void _Sysinit_Iterate_reversed(Sysinit_State state);

    The implement of referencing the sequenced initialization struct. In the every C source code file include initialization function we define a sequenced initialization struct which contain the init function entry. And then a reference function point will be defined in the another file include a function which will be called if this object is used by application. For example ,consider the RTEMS Message API. In the RTEMS Message API's initialization call's source file we create a structure Sysinit_Entry include the field of the RTEMS Message API's initialization call entry. Then we reference this variable from the source file containing the rtems_message_queue_create code. Because the application must make a call to rtems_message_queue_create or all the other Message API calls will fail therefore this API needs only one reference from a single source file. But there is also a exception that if the application use RTEMS Event and RTEMS Signal API there is no similar call like rtems_message_queue_creat which the application must call or it will fail. So we must consider another method to solve this problem. Now we donot have a better way to fix it.

    Consider the RTEMS Message API as a example, its initialization function is in file msg.c. And we also define the sequenced initialization struct in this file.

    void _Message_queue_Manager_initialization(void);

    SYSINIT_ENTRY(_Message_queue_Manager_initialization, SYSINIT_GROUP_MESSAGE);

    Then we reference the sequenced initialization struct in the file Msgcreat.c which contains rtems_message_queue_create function.

    SYSINIT_REFERENCE(_Message_queue_Manager_initialization);

    Summary of RTEMS Sequenced Initialization Error_Handling IRC meeting

    (attender DrJoel?, sebhub, gedare, zwj. all is IRC name)

    In june 16 there is a IRC meeting talking about RTEMS Sequenced Initialization Error_Handling. There is a error handler in the RTEMS code naming rtems_fatal_error_occured which maybe depends on some initialization steps. But DrJoel? pointed out that the Sequenced Initialization code is after the initialization of rtems_fatal_error_occured. So it will be ok to call the error handler. The RTEMS Sequenced Initialization Error_Handling is designed to invoke the fatal error handler themselves when something goes wrong and we want it to grow to later initialization points and user code such as initializing bdbuf via this mechanism or drivers. And initialization fatal errors are reserved for things that are truly horrible and reflect an inconsistent system configuration that you shouldn't be allowed to run the first task with Lab failures. RTEMS is not a desktop OS, every fatal error is dangerous. Then at the moment it is reasonable that we specifiy all errors during the sysinit invokation are fatal. So The entire set of these errors is in an enum, there is no need to check the return status for RTEMS purposes. Gedare suggested that we can configure whether an initialiation error is fatal or not. But this will make it more complicated and errors during initialization are software or configuration bugs, such applications are not worth to live long. So it is not worth doing that.

    Last modified on Feb 11, 2016 at 2:51:14 PM Last modified on Feb 11, 2016, 2:51:14 PM