Changes between Version 59 and Version 60 of Projects/OpenLDAP


Ignore:
Timestamp:
Jul 2, 2012, 5:29:13 AM (7 years ago)
Author:
C Rempel
Comment:

/* Current / Future Directions */

Legend:

Unmodified
Added
Removed
Modified
  • Projects/OpenLDAP

    v59 v60  
    385385Automatically configure OpenLDAP to NOT fork
    386386     slapd -d
    387 =  Current / Future Directions  =
    388 
    389 
    390 Although I found that the filio sample test and multio git package are existing RTEMS applications that uses the RTEMS shell, I'm leaning towards using a network-demo, because I'm thinking SLAPD doesn't require a shell, and why put something on an RTEMS application that doesn't need to be there?
     387==  ==OC|right}}
     388
     389'''Status:''' Exploratory
     390Light Weight Directory Access Protocol (LDAP) is used to manage distributed directories over a network.
     391
     392
     393According to Wikipedia: ''Historically the OpenLDAP server (slapd, the Standalone LDAP Daemon) architecture was split between a frontend which handles network access and protocol processing, and a backend which deals strictly with data storage. The architecture is modular and many different backends are available for interfacing to other technologies, not just traditional databases.''
     394
     395
     396OpenLDAP has been identified by Dr. Sherrill as a "high-priority" for RTEMS.  According to Beyond Linux from Scratch ''The OpenLDAP package provides an open source implementation of the Lightweight Directory Access Protocol,''.
     397
     398
     399In addition to helping me learn valuable porting skills, this project is on of the first steps towards adding needed modern protocols to RTEMS, so that RTEMS can use LDAP.  One of the more major first steps towards modernizing the networking protocols in RTEMS is the [http://git.rtems.org/rtems-libbsd.git/ RTEMS BSD Porting project].
     400= =Goal ==
     401
     402
     403Concise statement of the overall goal of the project. Refine this initial statement to include: project deliverable (code, docs, testing), required/suggested methodology, standards of quality, possible goal extensions beyond the main objective.
     404
     405C Rempel's main goal for porting OpenLDAP to RTEMS is to develop a method of porting semi-complex existing open source packages to RTEMS that is efficient, elegant and complete.  Efficient meaning the person porting the software can do so in a matter of days or weeks.  Elegant meaning the essence of the original source package, and the RTEMS source are preserved: as few lines of code are modified as possible in both the original source package and the RTEMS source as possible, as well as libraries, programs, and executable are installed in the same locations as they would be if they were built in the RTEMS source tree.  Completeness meaning not only the libraries are ported, but the programs and tests as well.
     406==  Deliverables  ==
     407
     408Just some ideas from C Rempel:
     409== = RTEMS Webkit Port  ===
     410
     411# A Makefile.openldap that builds the SlapD libraries for RTEMS applications
     412# A Makefile that builds the programs.
     413# A program that will run the LDAP testing suite provided by OpenLDAP.  ''Should be automated, as additional tests will be used in the future.''==  Suggested Methodology  ==
     414
     415
     416Start by porting to RTEMS Addon Packages, then adding a "network-demo style test" using a quick start as a guide.
     417==  Standards of Quality  ==
     418
     419
     420The standards of quality will be a SLAPD server, with a simple webkit demonstration. 
     421
     422Eventually, there should be tests that demonstrate how the SLAPD implementation meets [tools.ietf.org/html/rfc4510 RFC 4510].
     423==  Possible Goal Extensions  ==
     424
     425
     426
     427Explore and (possibly port to RTEMS) the additional packages used by OpenLDAP:
     428# Integrate [http://www.rtems.org/wiki/index.php/RTEMS_DBKit Database Support]
     429# Authentication (such as Cyrus SASL) [http://cyrusimap.web.cmu.edu/mediawiki/index.php/Downloads#SASL_Library website][http://git.cyrusimap.org/cyrus-sasl/ git]
     430# Additional encryption (when configuring openLDAP just type ./configure --help)
     431
     432
     433Explore and (possibly port to RTEMS) the additional packages that use OpenLDAP:
     434# Apache (did this already, but could be ported more efficiently, elegantly, and completely)
     435# Others
     436
     437
     438Add an OpenLDAP test to the RTEMS Networking Demos
     439=  ==Requirements===
     440
     441==  Required Programming Languages  ==
     442
     443
     444C, Bash, Makefile
     445==  Specific Areas of RTEMS or tools  ==
     446
     447
     448# RTEMS Addon Packages
     449# RTEMS Networking Demos
     450==  Level of Familiarity of RTEMS  ==
     451
     452
     453# Ability to configure RTEMS
     454# Ability to start the Hello, World RTEMS Application from the Examples v2.
     455# Ability to build the AVL library using the RTEMS Addon Packages
     456==  Cross-development  ==
     457
     458
     459# Experience using Makefiles and using configurations for cross development.
     460==  Gnu/Linux  ==
     461
     462
     463Ability to use the command line, and modify Makefiles. Knowing "grep -r", "find -name", and a text search feature REALLY helpful.
     464==  Development / Documentation / Testing Tools  ==
     465
     466
     467'''Development Tools:''' RTEMS Addon Packages, RTEMS Source Tree, RTEMS Toolset, Qemu
     468
     469'''Documentation Tools:''' RTEMS Wiki, comments
     470
     471'''Testing Tools:''' Tests provided by GCC and OpenLDAP, possibly other third party open source LDAP testing software...
     472==  Mathematical/algorithmic background  ==
     473
     474
     475None: this is simply putting two pieces of existing software together.
     476==  Other Desirable Skills  ==
     477
     478
     479# Code reuse skills: can research and use other programmer's code.
     480# Exploration skills: a willingness to peruse the RTEMS git, and other projects, to find code maintained by other developers (to make this port successful after the developer moves on).
     481
     482===Resources===
     483
     484Current RTEMS developers, papers, etc that may help you in this project.
     485
     486C Rempel is willing to help
     487
     488===Other sections===
     489
     490
     491If you have more to say about the project that doesn't fit in the proposed sections of this template, feel free to add other sections at will.
     492
     493===Acknowledgements===
     494
     495 *  who helped add did work
     496
     497==Miscellaneous Sections==
     498
     499
     500=== A first stab at building Open LDAP ===
     501
     502
     503As the project progresses, you will need to add build instructions, etc and this page will evolve from a project description into a HOWTO.
     504
     505I am building this package because it is simpler than the Apache Runtime, and so is a good way to gain needed experience.  I started with using the RTEMS Addon Packages. How to use the RTEMS Addon Packages is outlined [http://www.rtems.com/wiki/index.php/RTEMS_Add-On_Packages]. 
     506==  Makefile.openldap  ==
     507
     508
     509I put the following in rtems-addon-packages/Makefile.openldap
     510
     511 #
     512 # Declare supported terminal types.
     513 # This value can be augmented/overridden by the site-configuration file
     514 #
     515 
     516 include ../RTEMS_Makefiles/Makefile.common
     517 
     518 #
     519 # The following will work only if you have the latest ncurses version
     520 # of infocmp installed before trying to build for an RTEMS target.
     521 # The cf_cv_type_of_bool hack works around a bug when configuring
     522 # for a cross-target.
     523 #
     524 all:
     525        ac_cv_member_struct_msghdr_msg_control=no \
     526        ac_cv_func_memcmp_working=yes \
     527        ac_cv_func_initgroups=yes \
     528        ac_cv_func_setuid=no \
     529        ./configure \
     530                --host=$(RTEMS_CPU)-rtems4.11 \
     531                --prefix=$(exec_prefix) \
     532                --libdir=$(exec_prefix)/$(RTEMS_BSP)/lib \
     533                --includedir=$(exec_prefix)/$(RTEMS_BSP)/lib/include \
     534                --without-tls \
     535                --with-yielding_select=yes \
     536                --disable-mdb \
     537                --disable-hdb \
     538                --disable-cleartext \
     539                --disable-overlays \
     540                --enable-proctitle \
     541                --enable-debug \
     542                --disable-shared
     543        make
     544        make install
     545 
     546Next:
     547   $ cd ~/rtems-addon-packages
     548   $ git clone git://git.openldap.org/openldap.git openldap
     549   $ cd openldap
     550   $ make -f RTEMS_Makefiles/Makefile.openldap
     551
     552Note: Very insecure!  But builds OK.
     553==  examples/openldap/BuildTests.sh  ==
     554
     555I'll probably strip out the Examples section altogether... it was interesting and I learned about RTEMS Makefile.leaf s, but I'm thinking a network-demo is a better testing choice, because the openLDAP tests are written in shell script, and SLAPD may not require a shell to run...
     556 
     557To perform a very basic "smoke" test on the build I modified the BuildTest.sh and Makefile to build the programs needed to run the tests, not the tests themselves.  I learned I how to link convenience libraries, and how to use BuildTests.sh, Makefile.leaf and change leaf.cfg.
     558
     559Interestingly, I found it necessary to symbolically link the slapd-common header and source to each test, as well as the libutil library.
     560
     561 ### examples/openldap/BuildTests.sh ###
     562 #!/bin/sh
     563 
     564 set -x
     565 rm -f rtems-grub.cfg
     566 
     567 SRCDIR=../../openldap/tests/progs
     568
     569Make the common source and header files available to all the test programs
     570
     571 ln -s $SRCDIR/slapd-common.h .
     572 ln -s $SRCDIR/slapd-common.c .
     573
     574Make the lutility convenience library available to all the test programs
     575
     576 ln -s $SRCDIR/../../libraries/liblutil/liblutil.a .
     577
     578For every test, delete the old link, make a new link, make the test, delete the link and the object file.
     579
     580 for TEST in \
     581        slapd-tester \
     582        slapd-search \
     583        slapd-read \
     584        slapd-addel \
     585        slapd-modrdn \
     586        slapd-modify \
     587        slapd-bind \
     588        slapd-mtread \
     589        ldif-filter
     590 do
     591        rm -f $TEST.c
     592        ln -s $SRCDIR/$TEST.c .
     593        make TEST=$TEST SRCDIR=$SRCDIR
     594        rm -f $TEST.c $TEST.o
     595 done
     596After making the test programs, remove the links to the common header, source, and lutility library.
     597
     598 rm -f slapd-common.h
     599 rm -f slapd-common.c
     600 rm -f liblutil.a
     601
     602(Optional)
     603Next, to make it easier for the end-user, automate building rtems-grub.cfg
     604 # Generate rtems-grub.cfg
     605
     606Add a comment of what grub is configured to load $(basename $PWD) prints the name of the directory.
     607
     608 echo \# RTEMS Grub configuration for $(basename $PWD) >> rtems-grub.cfg
     609
     610This part was rather interesting... I've never used a pipe inside of a shell-script call before... list everything in o-optimize, that ends with .exe and count them.
     611 
     612 # The number of executables
     613 
     614 echo set default = $(ls o-optimize | \
     615 grep '\.'exe | wc -l) >> rtems-grub.cfg
     616
     617Used the & to use the name of the executable more than once.
     618''Note that sed"s/[a-zA-Z\-]/... has to be all on one line''
     619 # The menuentries
     620 ls o-optimize | \
     621 grep '\.'exe | \
     622 sed "s/[a-zA-Z\-]/menuentry \"RTEMS \-\" \{\n
     623        set root=(hd1,1)\n
     624        multiboot (hd1,1)\/o-optimize\/&/" | \
     625 sed "s/\.exe/\.exe\n\}\n/" >> rtems-grub.cfg
     626
     627# List everything in o-optimize...
     628# that is an executable...
     629# select text composed of letters and format it ...
     630# add a line a right brace, and
     631# another line after any .exe ...
     632
     633=== More tips ===
     634
     635For integrating legacy libraries/applications to RTEMS...
     636I was playing around with adding the sqlite3 command to the multiio shell, and found the following general ideas very helpful.
     637
     638# tear and rebuild ALL applications within the kit every time you make a small change.
     639# link the library into EVERY directory with an RTEMS shell application using your new commands.
     640
     641To tear down and rebuild
     642
     643 rtems-addon-packages$ nano doit.sh
     644
     645 #!/bin/sh
     646 make clean
     647 make
     648 qemu...
     649
     650To link add an entry to the Makefile:
     651
     652you can put the library in with the COBJS,
     653  ../openldap/.libs/libslapd.a
     654
     655or link with LD_LIBS
     656
     657  LD_LIBS += -lslapd
     658or
     659  LD_LIBS += ../openldap/.libs/libslapd.a
     660==  examples/openldap/Makefile  ==
     661
     662
     663I modified the rtems-addon-packages/examples/ncurses/Makefile to turn openldap programs used to test openldap into stand-alone executables.
     664
     665I had to add slapd-common.c to the C_PIECES, and slapd-common.h to the headers.
     666
     667Also, to link the liblutil.a convienience library, I had to add liblutil.a to C_O_FILES (it's an object file after-all :), I linked the ldap libraries using LD_LIBS, and increased the HeapSize by a factor of ten (because it said networking required more -- online research is needed to get an exact size required).
     668
     669Finally, it's worth noting the $(TEST) parameter is used to identify which test to build, and PGMS stands for programs.
     670
     671 #
     672 #  $Id$
     673 #
     674 # Templates/Makefile.leaf
     675 #      Template leaf node Makefile
     676 #
     677 SRCDIR=../../openldap/tests/progs
     678 
     679 # C source names, if any, go here -- minus the .c
     680 C_PIECES=slapd-common $(TEST)
     681 
     682 C_FILES=$(C_PIECES:%=%.c)
     683 C_O_FILES=$(C_PIECES:%=${ARCH}/%.o) liblutil.a
     684 
     685 # C++ source names, if any, go here -- minus the .cc
     686 CC_PIECES=
     687
     688 CC_FILES=$(CC_PIECES:%=%.cc)
     689 CC_O_FILES=$(CC_PIECES:%=${ARCH}/%.o)
     690 
     691 H_FILES= slapd-common.h
     692 
     693 # Assembly source names, if any, go here -- minus the .s
     694 S_PIECES=
     695
     696 S_FILES=$(S_PIECES:%=%.s)
     697 S_O_FILES=$(S_FILES:%.s=${ARCH}/%.o)
     698 
     699 SRCS=$(C_FILES) $(CC_FILES) $(H_FILES) $(S_FILES)
     700 OBJS=$(C_O_FILES) $(CC_O_FILES) $(S_O_FILES)
     701 
     702 PGMS=${ARCH}/$(TEST)
     703 
     704 #
     705 # RTEMS managers go here
     706 #
     707 MANAGERS=io event message semaphore
     708 
     709 include $(RTEMS_MAKEFILE_PATH)/Makefile.inc
     710 include $(RTEMS_CUSTOM)
     711 include $(PROJECT_ROOT)/make/leaf.cfg
     712 
     713 #
     714 # (OPTIONAL) Add local stuff here using +=
     715
     716 #
     717 
     718 DEFINES  +=
     719
     720 CPPFLAGS += -I$(SRCDIR)/../../include
     721 CFLAGS   +=
     722 
     723 LD_PATHS  +=
     724
     725 LD_LIBS   += -lldap -llber -lldap_r
     726 CFLAGS_LD   += -Wl,--defsym -Wl,HeapSize=0x200000 # network needs more space
     727 CFLAGS_DEBUG_V   +=
     728
     729 
     730 #
     731 # Add your list of files to delete here.  The config files
     732 #  already know how to delete some stuff, so you may want
     733 #  to just run 'make clean' first to see what gets missed.
     734 #  'make clobber' already includes 'make clean'
     735 #
     736 
     737 CLEAN_ADDITIONS +=
     738 CLOBBER_ADDITIONS +=
     739
     740 
     741 all:   ${ARCH} $(SRCS) $(PGMS)
     742 
     743 ${ARCH}/$(TEST): ${OBJS} ${LINK_FILES}
     744        $(make-exe)
     745 
     746 # Install the program(s), appending _g or _p as appropriate.
     747 # for include files, just use $(INSTALL)
     748 install:  all
     749        $(INSTALL_VARIANT) -m 555 ${PGMS} /usr/local/tftpboot/bootfiles/m68360/
     750==  leaf.cfg  ==
     751
     752To build the executables without errors, I had to change some lines in /opt/rtems-4.11/make/leaf.cfg.  nm and size work on object files, with the below changes the build process not only gives fewer errors, but generates .bin and .ralf files in examples/openldap/o-optimize!
     753
     754 define default-bsp-post-link
     755 -       $(NM) -g -n $@ > $(basename $@).num
     756 -       $(SIZE) $@
     757 endef
     758
     759nm and size work on executables, the RTEMS 4.11 development tree was updated in June 2012 to correctly configure the RTEMS Addon Packages.  Thanks!!!
     760
     761 define default-bsp-post-link
     762 +       $(NM) -g -n $@.exe > $(basename $@).num
     763 +       $(SIZE) $@.exe
     764 endef
     765
     766Then, I ran BuildTests.sh
     767 ~/rtems-addon-packages/examples/openldap$ ./BuildTests.sh
     768
     769Again this only built the programs used to run the tests as stand-alone executables...
     770==  Non-Forking SLAPD  ==
     771
     772Automatically configure OpenLDAP to NOT fork
     773     slapd -d
     774
     775==== SLAPD Test Idea ===
     776
     777The OpenLDAP [http://www.openldap.org/doc/admin24/quickstart.html quickstart guide] has some interesting ideas.
     778
     779If the test is written based on step 8 forwards, things that would be required would be:
     780# an appropriate configuration file in the rootfs
     781# a call to slapd with the -d option
     782# adding initial entries to a directory
     783# verify the entry exists
     784
     785The shell commands required would be
     786  SHLL [/] slapd -d
     787  SHLL [/] ldapadd -x -D "cn=Manager,dc=ldapdemo,dc=org" -W -f ldapdemo.ldif
     788  SHLL [/] ldapsearch -x -b 'dc=ldapdemo,dc=org' '(objectclass=*)'
     789
     790== Current / Future Directions ==
     791
     792
     793I'll have to verify if the RTEMS-OpenLDAP has the ldapsearch and ldapadd.
    391794
    392795My NEW next steps I will be exploring are:
     
    394797# using the quickstart guide as a guide to modifying the a network-demo to test SLAPD
    395798Hope this helps!
    396 = References =
     799
     800==References==
    397801
    398802 *  [http://www.linuxfromscratch.org/blfs/view/svn/server/openldap.html Beyond Linux From Scratch]