Changes between Version 61 and Version 62 of Projects/OpenLDAP


Ignore:
Timestamp:
Jul 2, 2012, 5:30:05 AM (7 years ago)
Author:
C Rempel
Comment:

/* = SLAPD Test Idea */

Legend:

Unmodified
Added
Removed
Modified
  • Projects/OpenLDAP

    v61 v62  
    381381
    382382Again this only built the programs used to run the tests as stand-alone executables...
    383 ==  ==OC|right}}
    384 
    385 '''Status:''' Exploratory
    386 Light Weight Directory Access Protocol (LDAP) is used to manage distributed directories over a network.
    387 
    388 
    389 According to Wikipedia: ''Historically the OpenLDAP server (slapd, the Standalone LDAP Daemon) architecture was split between a frontend which handles network access and protocol processing, and a backend which deals strictly with data storage. The architecture is modular and many different backends are available for interfacing to other technologies, not just traditional databases.''
    390 
    391 
    392 OpenLDAP has been identified by Dr. Sherrill as a "high-priority" for RTEMS.  According to Beyond Linux from Scratch ''The OpenLDAP package provides an open source implementation of the Lightweight Directory Access Protocol,''.
    393 
    394 
    395 In addition to helping me learn valuable porting skills, this project is on of the first steps towards adding needed modern protocols to RTEMS, so that RTEMS can use LDAP.  One of the more major first steps towards modernizing the networking protocols in RTEMS is the [http://git.rtems.org/rtems-libbsd.git/ RTEMS BSD Porting project].
    396 = =Goal ==
    397 
    398 
    399 Concise statement of the overall goal of the project. Refine this initial statement to include: project deliverable (code, docs, testing), required/suggested methodology, standards of quality, possible goal extensions beyond the main objective.
    400 
    401 C Rempel's main goal for porting OpenLDAP to RTEMS is to develop a method of porting semi-complex existing open source packages to RTEMS that is efficient, elegant and complete.  Efficient meaning the person porting the software can do so in a matter of days or weeks.  Elegant meaning the essence of the original source package, and the RTEMS source are preserved: as few lines of code are modified as possible in both the original source package and the RTEMS source as possible, as well as libraries, programs, and executable are installed in the same locations as they would be if they were built in the RTEMS source tree.  Completeness meaning not only the libraries are ported, but the programs and tests as well.
    402 ==  Deliverables  ==
    403 
    404 Just some ideas from C Rempel:
    405 == = RTEMS Webkit Port  ===
    406 
    407 # A Makefile.openldap that builds the SlapD libraries for RTEMS applications
    408 # A Makefile that builds the programs.
    409 # A program that will run the LDAP testing suite provided by OpenLDAP.  ''Should be automated, as additional tests will be used in the future.''==  Suggested Methodology  ==
    410 
    411 
    412 Start by porting to RTEMS Addon Packages, then adding a "network-demo style test" using a quick start as a guide.
    413 ==  Standards of Quality  ==
    414 
    415 
    416 The standards of quality will be a SLAPD server, with a simple webkit demonstration. 
    417 
    418 Eventually, there should be tests that demonstrate how the SLAPD implementation meets [tools.ietf.org/html/rfc4510 RFC 4510].
    419 ==  Possible Goal Extensions  ==
    420 
    421 
    422 
    423 Explore and (possibly port to RTEMS) the additional packages used by OpenLDAP:
    424 # Integrate [http://www.rtems.org/wiki/index.php/RTEMS_DBKit Database Support]
    425 # Authentication (such as Cyrus SASL) [http://cyrusimap.web.cmu.edu/mediawiki/index.php/Downloads#SASL_Library website][http://git.cyrusimap.org/cyrus-sasl/ git]
    426 # Additional encryption (when configuring openLDAP just type ./configure --help)
    427 
    428 
    429 Explore and (possibly port to RTEMS) the additional packages that use OpenLDAP:
    430 # Apache (did this already, but could be ported more efficiently, elegantly, and completely)
    431 # Others
    432 
    433 
    434 Add an OpenLDAP test to the RTEMS Networking Demos
    435 =  ==Requirements===
    436 
    437 ==  Required Programming Languages  ==
    438 
    439 
    440 C, Bash, Makefile
    441 ==  Specific Areas of RTEMS or tools  ==
    442 
    443 
    444 # RTEMS Addon Packages
    445 # RTEMS Networking Demos
    446 ==  Level of Familiarity of RTEMS  ==
    447 
    448 
    449 # Ability to configure RTEMS
    450 # Ability to start the Hello, World RTEMS Application from the Examples v2.
    451 # Ability to build the AVL library using the RTEMS Addon Packages
    452 ==  Cross-development  ==
    453 
    454 
    455 # Experience using Makefiles and using configurations for cross development.
    456 ==  Gnu/Linux  ==
    457 
    458 
    459 Ability to use the command line, and modify Makefiles. Knowing "grep -r", "find -name", and a text search feature REALLY helpful.
    460 ==  Development / Documentation / Testing Tools  ==
    461 
    462 
    463 '''Development Tools:''' RTEMS Addon Packages, RTEMS Source Tree, RTEMS Toolset, Qemu
    464 
    465 '''Documentation Tools:''' RTEMS Wiki, comments
    466 
    467 '''Testing Tools:''' Tests provided by GCC and OpenLDAP, possibly other third party open source LDAP testing software...
    468 ==  Mathematical/algorithmic background  ==
    469 
    470 
    471 None: this is simply putting two pieces of existing software together.
    472 ==  Other Desirable Skills  ==
    473 
    474 
    475 # Code reuse skills: can research and use other programmer's code.
    476 # Exploration skills: a willingness to peruse the RTEMS git, and other projects, to find code maintained by other developers (to make this port successful after the developer moves on).
    477 
    478 ===Resources===
    479 
    480 Current RTEMS developers, papers, etc that may help you in this project.
    481 
    482 C Rempel is willing to help
    483 
    484 ===Other sections===
    485 
    486 
    487 If you have more to say about the project that doesn't fit in the proposed sections of this template, feel free to add other sections at will.
    488 
    489 ===Acknowledgements===
    490 
    491  *  who helped add did work
    492 
    493 ==Miscellaneous Sections==
    494 
    495 
    496 === A first stab at building Open LDAP ===
    497 
    498 
    499 As the project progresses, you will need to add build instructions, etc and this page will evolve from a project description into a HOWTO.
    500 
    501 I am building this package because it is simpler than the Apache Runtime, and so is a good way to gain needed experience.  I started with using the RTEMS Addon Packages. How to use the RTEMS Addon Packages is outlined [http://www.rtems.com/wiki/index.php/RTEMS_Add-On_Packages]. 
    502 ==  Makefile.openldap  ==
    503 
    504 
    505 I put the following in rtems-addon-packages/Makefile.openldap
    506 
    507  #
    508  # Declare supported terminal types.
    509  # This value can be augmented/overridden by the site-configuration file
    510  #
    511  
    512  include ../RTEMS_Makefiles/Makefile.common
    513  
    514  #
    515  # The following will work only if you have the latest ncurses version
    516  # of infocmp installed before trying to build for an RTEMS target.
    517  # The cf_cv_type_of_bool hack works around a bug when configuring
    518  # for a cross-target.
    519  #
    520  all:
    521         ac_cv_member_struct_msghdr_msg_control=no \
    522         ac_cv_func_memcmp_working=yes \
    523         ac_cv_func_initgroups=yes \
    524         ac_cv_func_setuid=no \
    525         ./configure \
    526                 --host=$(RTEMS_CPU)-rtems4.11 \
    527                 --prefix=$(exec_prefix) \
    528                 --libdir=$(exec_prefix)/$(RTEMS_BSP)/lib \
    529                 --includedir=$(exec_prefix)/$(RTEMS_BSP)/lib/include \
    530                 --without-tls \
    531                 --with-yielding_select=yes \
    532                 --disable-mdb \
    533                 --disable-hdb \
    534                 --disable-cleartext \
    535                 --disable-overlays \
    536                 --enable-proctitle \
    537                 --enable-debug \
    538                 --disable-shared
    539         make
    540         make install
    541  
    542 Next:
    543    $ cd ~/rtems-addon-packages
    544    $ git clone git://git.openldap.org/openldap.git openldap
    545    $ cd openldap
    546    $ make -f RTEMS_Makefiles/Makefile.openldap
    547 
    548 Note: Very insecure!  But builds OK.
    549 ==  examples/openldap/BuildTests.sh  ==
    550 
    551 I'll probably strip out the Examples section altogether... it was interesting and I learned about RTEMS Makefile.leaf s, but I'm thinking a network-demo is a better testing choice, because the openLDAP tests are written in shell script, and SLAPD may not require a shell to run...
    552  
    553 To perform a very basic "smoke" test on the build I modified the BuildTest.sh and Makefile to build the programs needed to run the tests, not the tests themselves.  I learned I how to link convenience libraries, and how to use BuildTests.sh, Makefile.leaf and change leaf.cfg.
    554 
    555 Interestingly, I found it necessary to symbolically link the slapd-common header and source to each test, as well as the libutil library.
    556 
    557  ### examples/openldap/BuildTests.sh ###
    558  #!/bin/sh
    559  
    560  set -x
    561  rm -f rtems-grub.cfg
    562  
    563  SRCDIR=../../openldap/tests/progs
    564 
    565 Make the common source and header files available to all the test programs
    566 
    567  ln -s $SRCDIR/slapd-common.h .
    568  ln -s $SRCDIR/slapd-common.c .
    569 
    570 Make the lutility convenience library available to all the test programs
    571 
    572  ln -s $SRCDIR/../../libraries/liblutil/liblutil.a .
    573 
    574 For every test, delete the old link, make a new link, make the test, delete the link and the object file.
    575 
    576  for TEST in \
    577         slapd-tester \
    578         slapd-search \
    579         slapd-read \
    580         slapd-addel \
    581         slapd-modrdn \
    582         slapd-modify \
    583         slapd-bind \
    584         slapd-mtread \
    585         ldif-filter
    586  do
    587         rm -f $TEST.c
    588         ln -s $SRCDIR/$TEST.c .
    589         make TEST=$TEST SRCDIR=$SRCDIR
    590         rm -f $TEST.c $TEST.o
    591  done
    592 After making the test programs, remove the links to the common header, source, and lutility library.
    593 
    594  rm -f slapd-common.h
    595  rm -f slapd-common.c
    596  rm -f liblutil.a
    597 
    598 (Optional)
    599 Next, to make it easier for the end-user, automate building rtems-grub.cfg
    600  # Generate rtems-grub.cfg
    601 
    602 Add a comment of what grub is configured to load $(basename $PWD) prints the name of the directory.
    603 
    604  echo \# RTEMS Grub configuration for $(basename $PWD) >> rtems-grub.cfg
    605 
    606 This part was rather interesting... I've never used a pipe inside of a shell-script call before... list everything in o-optimize, that ends with .exe and count them.
    607  
    608  # The number of executables
    609  
    610  echo set default = $(ls o-optimize | \
    611  grep '\.'exe | wc -l) >> rtems-grub.cfg
    612 
    613 Used the & to use the name of the executable more than once.
    614 ''Note that sed"s/[a-zA-Z\-]/... has to be all on one line''
    615  # The menuentries
    616  ls o-optimize | \
    617  grep '\.'exe | \
    618  sed "s/[a-zA-Z\-]/menuentry \"RTEMS \-\" \{\n
    619         set root=(hd1,1)\n
    620         multiboot (hd1,1)\/o-optimize\/&/" | \
    621  sed "s/\.exe/\.exe\n\}\n/" >> rtems-grub.cfg
    622 
    623 # List everything in o-optimize...
    624 # that is an executable...
    625 # select text composed of letters and format it ...
    626 # add a line a right brace, and
    627 # another line after any .exe ...
    628 
    629 === More tips ===
    630 
    631 For integrating legacy libraries/applications to RTEMS...
    632 I was playing around with adding the sqlite3 command to the multiio shell, and found the following general ideas very helpful.
    633 
    634 # tear and rebuild ALL applications within the kit every time you make a small change.
    635 # link the library into EVERY directory with an RTEMS shell application using your new commands.
    636 
    637 To tear down and rebuild
    638 
    639  rtems-addon-packages$ nano doit.sh
    640 
    641  #!/bin/sh
    642  make clean
    643  make
    644  qemu...
    645 
    646 To link add an entry to the Makefile:
    647 
    648 you can put the library in with the COBJS,
    649   ../openldap/.libs/libslapd.a
    650 
    651 or link with LD_LIBS
    652 
    653   LD_LIBS += -lslapd
    654 or
    655   LD_LIBS += ../openldap/.libs/libslapd.a
    656 ==  examples/openldap/Makefile  ==
    657 
    658 
    659 I modified the rtems-addon-packages/examples/ncurses/Makefile to turn openldap programs used to test openldap into stand-alone executables.
    660 
    661 I had to add slapd-common.c to the C_PIECES, and slapd-common.h to the headers.
    662 
    663 Also, to link the liblutil.a convienience library, I had to add liblutil.a to C_O_FILES (it's an object file after-all :), I linked the ldap libraries using LD_LIBS, and increased the HeapSize by a factor of ten (because it said networking required more -- online research is needed to get an exact size required).
    664 
    665 Finally, it's worth noting the $(TEST) parameter is used to identify which test to build, and PGMS stands for programs.
    666 
    667  #
    668  #  $Id$
    669  #
    670  # Templates/Makefile.leaf
    671  #      Template leaf node Makefile
    672  #
    673  SRCDIR=../../openldap/tests/progs
    674  
    675  # C source names, if any, go here -- minus the .c
    676  C_PIECES=slapd-common $(TEST)
    677  
    678  C_FILES=$(C_PIECES:%=%.c)
    679  C_O_FILES=$(C_PIECES:%=${ARCH}/%.o) liblutil.a
    680  
    681  # C++ source names, if any, go here -- minus the .cc
    682  CC_PIECES=
    683 
    684  CC_FILES=$(CC_PIECES:%=%.cc)
    685  CC_O_FILES=$(CC_PIECES:%=${ARCH}/%.o)
    686  
    687  H_FILES= slapd-common.h
    688  
    689  # Assembly source names, if any, go here -- minus the .s
    690  S_PIECES=
    691 
    692  S_FILES=$(S_PIECES:%=%.s)
    693  S_O_FILES=$(S_FILES:%.s=${ARCH}/%.o)
    694  
    695  SRCS=$(C_FILES) $(CC_FILES) $(H_FILES) $(S_FILES)
    696  OBJS=$(C_O_FILES) $(CC_O_FILES) $(S_O_FILES)
    697  
    698  PGMS=${ARCH}/$(TEST)
    699  
    700  #
    701  # RTEMS managers go here
    702  #
    703  MANAGERS=io event message semaphore
    704  
    705  include $(RTEMS_MAKEFILE_PATH)/Makefile.inc
    706  include $(RTEMS_CUSTOM)
    707  include $(PROJECT_ROOT)/make/leaf.cfg
    708  
    709  #
    710  # (OPTIONAL) Add local stuff here using +=
    711 
    712  #
    713  
    714  DEFINES  +=
    715 
    716  CPPFLAGS += -I$(SRCDIR)/../../include
    717  CFLAGS   +=
    718  
    719  LD_PATHS  +=
    720 
    721  LD_LIBS   += -lldap -llber -lldap_r
    722  CFLAGS_LD   += -Wl,--defsym -Wl,HeapSize=0x200000 # network needs more space
    723  CFLAGS_DEBUG_V   +=
    724 
    725  
    726  #
    727  # Add your list of files to delete here.  The config files
    728  #  already know how to delete some stuff, so you may want
    729  #  to just run 'make clean' first to see what gets missed.
    730  #  'make clobber' already includes 'make clean'
    731  #
    732  
    733  CLEAN_ADDITIONS +=
    734  CLOBBER_ADDITIONS +=
    735 
    736  
    737  all:   ${ARCH} $(SRCS) $(PGMS)
    738  
    739  ${ARCH}/$(TEST): ${OBJS} ${LINK_FILES}
    740         $(make-exe)
    741  
    742  # Install the program(s), appending _g or _p as appropriate.
    743  # for include files, just use $(INSTALL)
    744  install:  all
    745         $(INSTALL_VARIANT) -m 555 ${PGMS} /usr/local/tftpboot/bootfiles/m68360/
    746 ==  leaf.cfg  ==
    747 
    748 To build the executables without errors, I had to change some lines in /opt/rtems-4.11/make/leaf.cfg.  nm and size work on object files, with the below changes the build process not only gives fewer errors, but generates .bin and .ralf files in examples/openldap/o-optimize!
    749 
    750  define default-bsp-post-link
    751  -       $(NM) -g -n $@ > $(basename $@).num
    752  -       $(SIZE) $@
    753  endef
    754 
    755 nm and size work on executables, the RTEMS 4.11 development tree was updated in June 2012 to correctly configure the RTEMS Addon Packages.  Thanks!!!
    756 
    757  define default-bsp-post-link
    758  +       $(NM) -g -n $@.exe > $(basename $@).num
    759  +       $(SIZE) $@.exe
    760  endef
    761 
    762 Then, I ran BuildTests.sh
    763  ~/rtems-addon-packages/examples/openldap$ ./BuildTests.sh
    764 
    765 Again this only built the programs used to run the tests as stand-alone executables...
    766 
    767 ==== SLAPD Test Idea ===
     383==  SLAPD Test Idea  ==
    768384
    769385The OpenLDAP [http://www.openldap.org/doc/admin24/quickstart.html quickstart guide] has some interesting ideas.
     
    779395  SHLL [/] ldapadd -x -D "cn=Manager,dc=ldapdemo,dc=org" -W -f ldapdemo.ldif
    780396  SHLL [/] ldapsearch -x -b 'dc=ldapdemo,dc=org' '(objectclass=*)'
    781 
    782 == Current / Future Directions ==
     397=  Current / Future Directions  =
    783398
    784399
     
    789404# using the quickstart guide as a guide to modifying the a network-demo to test SLAPD
    790405Hope this helps!
    791 
    792 ==References==
     406= References =
    793407
    794408 *  [http://www.linuxfromscratch.org/blfs/view/svn/server/openldap.html Beyond Linux From Scratch]