Changes between Version 8 and Version 9 of Projects/libdl


Ignore:
Timestamp:
Apr 16, 2013, 2:28:58 PM (7 years ago)
Author:
ChrisJohns
Comment:

Bring up to date.

Legend:

Unmodified
Added
Removed
Modified
  • Projects/libdl

    v8 v9  
    66
    77
    8 '''Status:''' Android Bison libc dlfnc.h functionality ported to RTEMS.  Simple examples work but important use cases do not.  Needs to support more CPU architectures.
     8'''Status:''' RTEMS target loader is working and an RTEMS Linker is available. More CPU architectures need to be support.
     9
     10The project's documentation is generated by doxygen. You can view a current version here:
     11
     12http://www.rtems.org/ftp/pub/rtems/people/chrisj/rtl/rtems-linker/
    913
    1014Dynamic loading of code into a running RTEMS target has been a long term wish for many RTEMS users. Dynamic loading is not for all systems but systems with the resources to support it could make use of the advantages it offers.
     
    1317system when running to help locate a problem. This is particularly useful when a project is in an integration or testing phase and something has gone wrong.
    1418
    15 Dynamic loading for RTEMS does not provide some of the advantages seen with virtual memory operating system such as Linux. On these systems dynamic loading allows code to be shared between separate processes. RTEMS is a single process operating system so there is nothing to share code with. Dynamic loading uses more resources and takes longer to start the application. Memory is needed by the dynamic linker and space is needed on the target for the libraries if held locally. You need a working file system to read the code into memory, plus there is the management of symbols. A potential down side of dynamic linking if not handle efficiently is the possible loading of a complete library. Virtual memory operating systems such as Linux avoid this issue by code sharing and demand loading executable files.
    16 
    17 In recent years Till Straumann has provided a separate package call cexp to allow loading of modules of code in RTEMS. His excellent package provides a similar model used in some commercial real-time operating systems. It how ever does not follow any of the standard APIs that RTEMS currently follows. Being open source and following standards are two corner stones of RTEMS. The cexp provides custom solutions for some of the more complex issues that arise with dynamic loading of code. This code and his efforts provide an important base for this and future work related to dynamic loading in RTEMS. An important area he has solved is the management of targeted linking of libraries such as libc. How this work translates to the {{{<dlfcn.h></code> API is yet to be determined.
     19Dynamic loading for RTEMS does not provide some of the advantages seen with virtual memory operating system such as Unix. On these systems dynamic loading allows code to be shared between separate processes. RTEMS is a single process operating system so there is nothing to share code with. Dynamic loading uses more resources and takes longer to start the application. Memory is needed by the dynamic linker and space is needed on the target for the libraries if held locally. You need a working file system to read the code into memory, plus there is the management of symbols. A potential down side of dynamic linking if not handle efficiently is the possible loading of a complete library. Virtual memory operating systems such as Linux avoid this issue by code sharing and demand loading executable files.
    1820
    1921The IEEE Std 1003.1-2004 standard defines [http://www.opengroup.org/onlinepubs/009695399/functions/dlopen.html  <dlfcn.h>]. This is a small API that makes an executable object file available to the calling program. The API calls are:
     
    2931
    3032Some operating systems provide extra interfaces to help manage dynamically loaded object files. For example FreeBSD provides {{{dlinfo</code>. The RTEMS can provide these types of interfaces as implementation demands.
    31 
    32 The central component is the [http://www.freebsd.org/cgi/man.cgi?query=rtld&sektion=1 dynamic linker]. The dynamic linker
    33 provides run-time loading and link-editing of object files. The linker loads the object file code for all shared  libraries into the process's address space performing any relocation, then proceeds to resolve external references from both the main program and all object files loaded. The linker calls initialisation routines for each object file
    34 loaded giving a shared object an opportunity to perform any extra set-up before execution of the program proper beings. C++ libraries that contain static constructors require this type of initialisation. The dynamic linker is specific to the ELF file format. This means the RTEMS object file format is ELF for targets that require dynamic loading of object files.
    35 
    36 The main application can be viewed as an object file that is required to loaded, relocated and initialised before being started. It can be considered the root of a tree of dynamically referenced object files loaded at run-time.
    3733= Acknowledgements =
    3834
     
    4440
    4541
    46 The task for RTEMS can be split in two, the host side and the target side. The two sides need to agree on the specific interfaces, formats and services performed. An example of issues that need to be considered is libgcc and RTEMS itself. Are these object files also considered dynamic and loaded by the dynamic linker ?
     42The current status is two separate git repos exist:
    4743
    48 The issue of version numbers for the various components needs consideration.
     44http://git.rtems.org/chrisj/rtl.git/
     45http://git.rtems.org/chrisj/rtl-host.git/
    4946
    50 On the host side the aim is to leverage as much existing code as possible. Dynamic linking is well understand and stable on a number of operating system using the GNU Compiler Collection. RTEMS also uses this tool set so the use of dynamic linking features of these tools is to be investigated and reported. The task starts with attempting to link a simple application such as "Hello World" dynamically. This will require the current static RTEMS libraries be made to appear as
    51 dynamic libraries. For an evaluation how this is done is not important. The static libraries may need to be unpacked and repacked as dynamic libraries or rebuilt from source. Not all libraries need to be dyna during this phase of the task. One dynamic library may be enough to report suitable findings.
     47These are both waf based projects. The RTL Host project builds the RTEMS Linker (rtems-ld) and the RTL project builds an RTEMS application that runs on the i386 and SPARC processors.
    5248
    53 The target side needs to handle ELF files in an efficient manner. The license for the code must be compatible with RTEMS. This rules out the BFD library in the binutils package. It is GPL code and too generic to meet the specific needs of RTEMS target code. Efficient handling of symbols is also needed. Compression can be useful here as it provides for smaller libraries plus it provides a run-time check sum on the object file or library. Java handles its class libraries this way.
     49The RTL project uses a version of libsbports I am yet to make public (no time) where libbspport is a dynamic module loaded at runtime. It runs on a real PC, loads and detects a PCI based network card. This is a sizeable application with over 14,000 relocation fix ups.
    5450
    55 A complexity raised earlier that Till handles with a special host side tool is selective linking against a library. Consider the "Hello World" example where a call to 'printf' is made. The linking phase sees the printf symbol is held in the dynamic library libc and fills the ELF file with the required information to allow the dynamic linker to perform its task. Our target run-time linker sees the reference to printf and libc and opens and loads the whole library. It has to do this because libc has no smaller dynamic library component. Lets now consider we have a very clever run-time linker that loads only the object files from libc that are used. That is we dynamically link everything based on demand at the object file level. The loading and relocating component can be handled, how-ever the initialisation needs consideration. How do you handle adding a {{{dlopen</code> call to our example inside {{{main</code> to load an object file that also references libc but a part that is not currently loaded and needs loading ? How is this initialisation handled ? A possible solution is the breaking down of libraries into a smaller collection of dynamic libraries. RTEMS could be packaged this way with a certain effort how-ever libc, libm plus other libraries also need consideration.
     51The RTEMS Linker support the RTEMS Application Format (RAP) file format. This is a compressed post link processed file.
    5652= Open Projects =
    5753
    5854
    59 There is a port of the Android libc functionality to RTEMS.  It can be used to successfully load and execute a method in a simple dynamic library.  The following use cases must be satisfied before the code can be considered ready for inclusion in RTEMS.
    60 
    61  *  load simple dynamic library and execute method
    62  *  load simple dynamic library, execute method and it call a method in the base image (e.g. printf)
    63  *  load two dynamic libraries, have the second depend on a symbol in the first and have it work.
    64 
    65 [wiki:User:JoelSherrill  Joel Sherrill] has tests for the above.
    66 
    6755The RTEMS port supports i386 and SPARC.  We need more architectures.  Updating the source and remerging our extra targets will resolve some of these. 
    6856
    69 The Android source has a different structure for addressing per-target issues.  We should track their code, add our own targets and submit to them.  They have ARM, SuperH, and x86.  We could submit SPARC.  There is a MIPS Android port we could leverage and push to get merged into the main repository.
     57Developing more test cases.
    7058
    71 The current source is available via:
    72 
    73 {{{
    74 git clone git://github.com/absabs/objdl
    75 }}}
    76 
    77 The original Android source is available here.  This is NOT the version the above code is based upon. 
    78 
    79 {{{
    80 http://android.git.kernel.org/?p=platform/bionic.git;a=tree;f=linker;h=39b12066c60cfc2748fa2a3b0d7903966ac7ad5b;hb=HEAD
    81 }}}
     59Functional level unit testing of the RTEMS Linker. This is a complex C++ application and the RAP processing is complex and contains bugs.
    8260= References =
    8361