source: rtems-source-builder/doc/source-builder.txt @ 0d59530

4.11
Last change on this file since 0d59530 was 0d59530, checked in by Hesham Almatary <heshamelmatary@…>, on Aug 18, 2015 at 1:37:04 PM

doc: zlib1g-dev package needs to be installed on Linux Mint

  • Property mode set to 100644
File size: 144.0 KB
Line 
1RTEMS Source Builder
2====================
3:doctype: book
4:toc2:
5:toclevels: 5
6:icons:
7:numbered:
8
9image:images/rtemswhitebg.jpg["RTEMS",width="30%"]
10
11Chris Johns <chrisj@rtems.org>
121.9, July 2014
13
14RTEMS Tools From Source
15-----------------------
16
17The RTEMS Source Builder is a tool to aid building packages from source used by
18the RTEMS project. It helps consolidate the details you need to build a package
19from source in a controlled and verifiable way. The tool is aimed at developers
20of software who use tool sets for embedded type development and is not limited
21to building just for RTEMS. Embedded development typically uses cross-compiling
22tool chains, debuggers, and debugging aids. Together we call these a 'tool
23set'. The RTEMS Source Builder is not limited to this role but designed to fit
24with-in this specific niche. It can be used outside of the RTEMS project and we
25welcome this happening in other open source or commercial projects.
26
27The RTEMS Source Builder is typically used to build a set of tools or a 'build
28set'. A 'build set' is a collection of packages and a package is a specific
29tool, for example gcc or gdb. The RTEMS Source Builder attempts to support any
30host environment that runs Python and you can build the package on. It is not
31some sort of magic that can take any piece of source code and make it
32build. Someone at some point in time has figured out how to build that package
33from source and taught this tool. The RTEMS Source Builder has been tested on:
34
35[[platform_links]]
36* <<_archlinux,Archlinux>>
37* <<_centos,Centos>>
38* <<_fedora,Fedora>>
39* <<_freebsd,FreeBSD>>
40* <<_netbsd,NetBSD>>
41* <<_macos,MacOS>>
42* <<_mint,Linux Mint>>
43* <<_opensuse,openSUSE>>
44* <<_raspbian,Raspbian>>
45* <<_ubuntu,Ubuntu>>
46* <<_windows,Windows>>
47* <<_ubuntu,Xubuntu>>
48
49The RTEMS Source Builder has two types of configuration data. The first is the
50'build set'. A _build set_ describes a collection of packages that define a set
51of tools you would use when developing software for RTEMS. For example the
52basic GNU tool set is binutils, gcc, and gdb and is the typical base suite of
53tools you need for an embedded cross-development type project. The second type
54of configuration data is the configuration files and they define how a package
55is built. Configuration files are scripts loosely based on the RPM spec file
56format and they detail the steps needed to build a package. The steps are
57'preparation', 'building', and 'installing'. Scripts support macros, shell
58expansion, logic, includes plus many more features useful when build packages.
59
60The RTEMS Source Builder does not interact with any host package management
61systems. There is no automatic dependence checking between various packages you
62build or packages and software your host system you may have installed. We
63assume the build sets and configuration files you are using have been created
64by developers who do. Support is provided for package config or pkgconfg type
65files so you can check and use standard libraries if present. If you have a
66problem please ask on the RTEMS Users mailing list.
67
68This documentation caters for a range of users from new to experienced RTEMS
69developers. New users can follow the Quick Start section up to the "Installing
70and Tar Files" to get a working tools and RTEMS. Users building a binary tool
71set for release can read the "Installing and Tar Files". Users wanting to run
72and test bleeding edge tools or packages, or wanting update or extend the RSB's
73configuration can read the remaining sections.
74
75*************************************************************
76IMPORTANT: If you have a problem please see <<_bugs,the reporting bugs>>
77           section.
78*************************************************************
79
80Quick Start
81-----------
82
83The quick start will show you how to build a set of RTEMS tools for a supported
84architecture. The tools are installed into a build _prefix_. The _prefix_ is the
85top of a group of directories containing all the files needed to develop RTEMS
86applications. Building an RTEMS build set will build all that you need. This
87includes autoconf, automake, assemblers, linkers, compilers, debuggers,
88standard libraries and RTEMS itself.
89
90There is no need to become root or the administrator and we recommend you avoid
91doing this. You can build and install the tools anywhere on the host's file
92system you, as a standard user, have read and write access too. I recommend you
93use your home directory and work under the directory `~/development/rtems`. The
94examples shown here will do this.
95
96You can use the build _prefix_ to install and maintain different versions of
97the tools. Doing this lets you try a new set of tools while not touching your
98proven working production set of tools. Once you have proven the new tools are
99working rebuild with the 'production' prefix switching your development to them.
100
101I also suggest you keep your environment to the bare minimum, particularly the
102path variable. Using environment variables has been proven over the years to be
103difficult to manage in production systems.
104
105.Host Setup
106*************************************************************
107IMPORTANT: Before proceeding to the next section please refer to the
108<<_host_setups,host specific setup>> for your host and install any extra
109packages. The RSB assumes the needed packages are installed and work.
110*************************************************************
111
112.Path to use when building applications
113*************************************************************
114TIP: Do not forget to do this before you use the tools such as build RTEMS.
115
116The RSB by default will install (copy) the executables under the prefix you
117supply. To use the tools once finished just set your path to the 'bin'
118directory under the _prefix_ you use. In the examples that follow the _prefix_
119is `$HOME/development/rtems/4.11` and is set using the `--prefix` option so the
120path you need to configure to build applications can be set with the following
121in a BASH shell:
122
123-------------------------------------------------------------
124$ export PATH=$HOME/development/rtems/4.11/bin:$PATH
125-------------------------------------------------------------
126Make sure you place the RTEMS tool path at the front of your path so they are
127searched first. RTEMS can provide newer versions of some tools your operating
128system provides and placing the RTEMS tools path at the front means it is
129searched first and the RTEMS needed versions of the tools are used.
130*************************************************************
131
132Setup
133~~~~~
134
135Setup a development work space:
136
137-------------------------------------------------------------
138$ cd
139$ mkdir -p development/rtems/src
140$ cd development/rtems/src
141-------------------------------------------------------------
142
143The RTEMS Source Builder is distributed as source. It is Python code so all you
144need to do is head over to the RTEMS GIT repository and clone the code directly
145from git:
146
147-------------------------------------------------------------
148$ git clone git://git.rtems.org/rtems-source-builder.git
149$ cd rtems-source-builder
150-------------------------------------------------------------
151
152Checking
153~~~~~~~~
154
155The next step is to check if your host is set up correctly. The RTEMS Source
156Builder provides a tool to help:
157
158-------------------------------------------------------------
159$ source-builder/sb-check
160warning: exe: absolute exe found in path: (__objcopy) /usr/local/bin/objcopy <1>
161warning: exe: absolute exe found in path: (__objdump) /usr/local/bin/objdump
162error: exe: not found: (_xz) /usr/local/bin/xz <2>
163RTEMS Source Builder environment is not correctly set up
164$ source-builder/sb-check
165RTEMS Source Builder environment is ok <3>
166-------------------------------------------------------------
167
168<1> A tool is in the environment path but does not match the expected path.
169<2> The executable 'xz' is not found.
170<3> The host's environment is set up correct.
171
172The checking tool will output a list of executable files not found if problems
173are detected. Locate those executable files and install them. You may also be
174given a list of warnings about executable files not in the expected location
175however the executable was located somewhere in your environment's path. You
176will need to check each tool to determine if this is an issue. An executable
177not in the standard location may indicate it is not the host operating system's
178standard tool. It maybe ok or it could be buggy, only you can determine this.
179
180The <<_host_setups,Host Setups>> section lists packages you should install for
181common host operating systems. It maybe worth checking if you have those
182installed.
183
184Build Sets
185~~~~~~~~~~
186
187The RTEMS tools can be built within the RTEMS Source Builder's source tree. We
188recommend you do this so lets change into the RTEMS directory in the RTEMS
189Source Builder package:
190
191-------------------------------------------------------------
192$ cd rtems
193-------------------------------------------------------------
194
195If you are unsure how to specify the build set for the architecture you wish to
196build ask the tool:
197
198-------------------------------------------------------------
199$ ../source-builder/sb-set-builder --list-bsets <1>
200RTEMS Source Builder - Set Builder, v0.2.0
201Examining: config <2>
202Examining: ../source-builder/config <2>
203    4.10/rtems-all.bset <3>
204    4.10/rtems-arm.bset <4>
205    4.10/rtems-autotools.bset
206    4.10/rtems-avr.bset
207    4.10/rtems-bfin.bset
208    4.10/rtems-h8300.bset
209    4.10/rtems-i386.bset
210    4.10/rtems-lm32.bset
211    4.10/rtems-m32c.bset
212    4.10/rtems-m32r.bset
213    4.10/rtems-m68k.bset
214    4.10/rtems-mips.bset
215    4.10/rtems-nios2.bset
216    4.10/rtems-powerpc.bset
217    4.10/rtems-sh.bset
218    4.10/rtems-sparc.bset
219    4.11/rtems-all.bset
220    4.11/rtems-arm.bset
221    4.11/rtems-autotools.bset
222    4.11/rtems-avr.bset
223    4.11/rtems-bfin.bset
224    4.11/rtems-h8300.bset
225    4.11/rtems-i386.bset
226    4.11/rtems-lm32.bset
227    4.11/rtems-m32c.bset
228    4.11/rtems-m32r.bset
229    4.11/rtems-m68k.bset
230    4.11/rtems-microblaze.bset
231    4.11/rtems-mips.bset
232    4.11/rtems-moxie.bset
233    4.11/rtems-nios2.bset
234    4.11/rtems-powerpc.bset
235    4.11/rtems-sh.bset
236    4.11/rtems-sparc.bset
237    4.11/rtems-sparc64.bset
238    4.11/rtems-v850.bset
239    4.9/rtems-all.bset
240    4.9/rtems-arm.bset
241    4.9/rtems-autotools.bset
242    4.9/rtems-i386.bset
243    4.9/rtems-m68k.bset
244    4.9/rtems-mips.bset
245    4.9/rtems-powerpc.bset
246    4.9/rtems-sparc.bset
247    gnu-tools-4.6.bset
248    rtems-4.10-base.bset <5>
249    rtems-4.11-base.bset
250    rtems-4.9-base.bset
251    rtems-base.bset <5>
252    unstable/4.11/rtems-all.bset <6>
253    unstable/4.11/rtems-arm.bset
254    unstable/4.11/rtems-avr.bset
255    unstable/4.11/rtems-bfin.bset
256    unstable/4.11/rtems-h8300.bset
257    unstable/4.11/rtems-i386.bset
258    unstable/4.11/rtems-lm32.bset
259    unstable/4.11/rtems-m32c.bset
260    unstable/4.11/rtems-m32r.bset
261    unstable/4.11/rtems-m68k.bset
262    unstable/4.11/rtems-microblaze.bset
263    unstable/4.11/rtems-mips.bset
264    unstable/4.11/rtems-moxie.bset
265    unstable/4.11/rtems-powerpc.bset
266    unstable/4.11/rtems-sh.bset
267    unstable/4.11/rtems-sparc.bset
268    unstable/4.11/rtems-sparc64.bset
269    unstable/4.11/rtems-v850.bset
270-------------------------------------------------------------
271<1> Only option needed is +--list-bsets+
272<2> The paths inspected. See <<X1,+_configdir+>> variable.
273<3> Build all RTEMS 4.10 supported architectures.
274<4> The build set for the ARM architecture on RTEMS 4.10.
275<5> Support build set file with common functionality included by other build
276    set files.
277<6> Unstable tool sets are used by RTEMS developers to test new tool sets. You
278    are welcome to try them but you must remember they are unstable, can change
279    at any point in time and your application may not work. If you have an
280    issue with a production tool it may pay to try the unstable tool to see if
281    it has been resolved.
282
283Building
284~~~~~~~~
285
286In this quick start I will build a SPARC tool set.
287
288-------------------------------------------------------------
289$ ../source-builder/sb-set-builder --log=l-sparc.txt <1> \
290                --prefix=$HOME/development/rtems/4.11 <2> 4.11/rtems-sparc <3>
291Source Builder - Set Builder, v0.2.0
292Build Set: 4.11/rtems-sparc
293config: expat-2.1.0-1.cfg <4>
294package: expat-2.1.0-x86_64-freebsd9.1-1
295building: expat-2.1.0-x86_64-freebsd9.1-1
296config: tools/rtems-binutils-2.22-1.cfg <5>
297package: sparc-rtems4.11-binutils-2.22-1
298building: sparc-rtems4.11-binutils-2.22-1
299config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg <6>
300package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
301building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
302config: tools/rtems-gdb-7.5.1-1.cfg <7>
303package: sparc-rtems4.11-gdb-7.5.1-1
304building: sparc-rtems4.11-gdb-7.5.1-1
305installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <8>
306installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
307installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
308installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
309cleaning: expat-2.1.0-x86_64-freebsd9.1-1 <9>
310cleaning: sparc-rtems4.11-binutils-2.22-1
311cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
312cleaning: sparc-rtems4.11-gdb-7.5.1-1
313Build Set: Time 0:13:43.616383 <10>
314-------------------------------------------------------------
315
316<1> Providing a log file redirects the build output into a file. Logging the
317    build output provides a simple way to report problems.
318<2> The prefix is the location on your file system the tools are installed
319    into. Provide a prefix to a location you have read and write access. You
320    can use the prefix to install different versions or builds of tools. Just
321    use the path to the tools you want to use when building RTEMS.
322<3> The build set. This is the SPARC build set. First a specifically referenced
323    file is checked for and if not found the '%{_configdir} path is
324    searched. The set builder will first look for files with a +.bset+
325    extension and then for a configuration file with a +.cfg+ extension.
326<4> The SPARC build set first builds the expat library as it is used in GDB.
327    This is the expat configuration used.
328<5> The binutils build configuration.
329<6> The GCC and Newlib build configuration.
330<7> The GDB build configuration.
331<8> Installing the built packages to the install prefix.
332<9> All the packages built are cleaned at the end. If the build fails all the
333    needed files are present. You may have to clean up by deleting the build
334    directory if the build fails.
335<10> The time to build the package. This lets you see how different host
336     hardware or configurations perform.
337
338Your SPARC RTEMS 4.11 tool set will be installed and ready to build RTEMS and
339RTEMS applications. When the build runs you will notice the tool fetch the
340source code from the internet. These files are cached in a directory called
341+source+. If you run the build again the cached files are used. This is what
342happened in the shown example before.
343
344TIP: The RSB for releases will automatically build and install RTEMS. The
345development version require adding +--with-rtems+. Use this for a single
346command to get the tools and RTEMS with one command.
347
348The installed tools:
349
350-------------------------------------------------------------
351$ ls $HOME/development/rtems/4.11
352bin         include     lib         libexec     share       sparc-rtems4.11
353$ ls $HOME/development/rtems/4.11/bin
354sparc-rtems4.11-addr2line       sparc-rtems4.11-cpp
355sparc-rtems4.11-gcc-ar          sparc-rtems4.11-gprof
356sparc-rtems4.11-objdump         sparc-rtems4.11-size
357sparc-rtems4.11-ar              sparc-rtems4.11-elfedit
358sparc-rtems4.11-gcc-nm          sparc-rtems4.11-ld
359sparc-rtems4.11-ranlib          sparc-rtems4.11-strings
360sparc-rtems4.11-as              sparc-rtems4.11-g++
361sparc-rtems4.11-gcc-ranlib      sparc-rtems4.11-ld.bfd
362sparc-rtems4.11-readelf         sparc-rtems4.11-strip
363sparc-rtems4.11-c++             sparc-rtems4.11-gcc
364sparc-rtems4.11-gcov            sparc-rtems4.11-nm
365sparc-rtems4.11-run             xmlwf
366sparc-rtems4.11-c++filt         sparc-rtems4.11-gcc-4.7.2
367sparc-rtems4.11-gdb             sparc-rtems4.11-objcopy
368sparc-rtems4.11-sis
369$ $HOME/development/rtems/4.11/bin/sparc-rtems4.11-gcc -v
370Using built-in specs.
371COLLECT_GCC=/home/chris/development/rtems/4.11/bin/sparc-rtems4.11-gcc
372COLLECT_LTO_WRAPPER=/usr/home/chris/development/rtems/4.11/bin/../ \
373libexec/gcc/sparc-rtems4.11/4.7.2/lto-wrapper
374Target: sparc-rtems4.11 <1>
375Configured with: ../gcc-4.7.2/configure <2>
376--prefix=/home/chris/development/rtems/4.11
377--bindir=/home/chris/development/rtems/4.11/bin
378--exec_prefix=/home/chris/development/rtems/4.11
379--includedir=/home/chris/development/rtems/4.11/include
380--libdir=/home/chris/development/rtems/4.11/lib
381--libexecdir=/home/chris/development/rtems/4.11/libexec
382--mandir=/home/chris/development/rtems/4.11/share/man
383--infodir=/home/chris/development/rtems/4.11/share/info
384--datadir=/home/chris/development/rtems/4.11/share
385--build=x86_64-freebsd9.1 --host=x86_64-freebsd9.1 --target=sparc-rtems4.11
386--disable-libstdcxx-pch --with-gnu-as --with-gnu-ld --verbose --with-newlib
387--with-system-zlib --disable-nls --without-included-gettext
388--disable-win32-registry --enable-version-specific-runtime-libs --disable-lto
389--enable-threads --enable-plugin --enable-newlib-io-c99-formats
390--enable-newlib-iconv --enable-languages=c,c++
391Thread model: rtems <3>
392gcc version 4.7.2 20120920 <4>
393 (RTEMS4.11-RSB(cb12e4875c203f794a5cd4b3de36101ff9a88403)-1,gcc-4.7.2/newlib-2.0.0) (GCC)
394-------------------------------------------------------------
395
396<1> The target the compiler is built for.
397<2> The configure options used to build GCC.
398<3> The threading model is always RTEMS. This makes the RTEMS tools difficult
399    for bare metal development more difficult.
400<4> The version string. It contains the Git hash of the RTEMS Source Builder
401    you are using. If your local clone has been modifed that state is also
402    recorded in the version string. The hash allows you to track from a GCC
403    executable back to the original source used to build it.
404
405NOTE: The RTEMS thread model enables specific hooks in GCC so applications
406built with RTEMS tools need the RTEMS runtime to operate correctly. You can use
407RTEMS tools to build bare metal component but it is more difficult than with a
408bare metal tool chain and you need to know what you are doing at a low
409level. The RTEMS Source Builder can build bare metal tool chains as well. Look
410in the top level +bare+ directory.
411
412Distributing and Archiving A Build
413~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
414
415If you wish to create and distribute your build or you want to archive a build
416you can create a tar file. This is a more advanced method for binary packaging
417and installing of tools.
418
419By default the RTEMS Source Builder installs the built packages directly and
420optionally it can also create a _build set tar file_ or a _package tar file_
421per package built. The normal and default behaviour is to let the RTEMS Source
422Builder install the tools. The source will be downloaded, built, installed and
423cleaned up.
424
425The tar files are created with the full build prefix present and if you follow
426the examples given in this documentation the path is absolute. This can cause
427problems if you are installing on a host you do not have super user or
428administrator rights on because the prefix path may references part you do not
429have write access too and tar will not extract the files. You can use the
430+--strip-components+ option in tar if your host tar application supports it to
431remove the parts you do not have write access too or you may need to unpack the
432tar file somewhere and copy the file tree from the level you have write access
433from. Embedding the full prefix path in the tar files lets you know what the
434prefix is and is recommended. For example if
435`/home/chris/development/rtems/4.11` is the prefix used you cannot change
436directory to the root (`/`) and install because the `/home` is root access
437only. To install you would:
438
439-------------------------------------------------------------
440$ cd
441$ tar --strip-components=3 -xjf rtems-4.11-sparc-rtems4.11-1.tar.bz2
442-------------------------------------------------------------
443
444A build set tar file is created by adding `--bset-tar-file` option to the
445`sb-set-builder` command.
446
447-------------------------------------------------------------
448$ ../source-builder/sb-set-builder --log=l-sparc.txt \
449         --prefix=$HOME/development/rtems/4.11 \
450         --bset-tar-file <1> 4.11/rtems-sparc
451Source Builder - Set Builder, v0.2.0
452Build Set: 4.11/rtems-sparc
453config: expat-2.1.0-1.cfg
454package: expat-2.1.0-x86_64-freebsd9.1-1
455building: expat-2.1.0-x86_64-freebsd9.1-1
456config: tools/rtems-binutils-2.22-1.cfg
457package: sparc-rtems4.11-binutils-2.22-1
458building: sparc-rtems4.11-binutils-2.22-1
459config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
460package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
461building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
462config: tools/rtems-gdb-7.5.1-1.cfg
463package: sparc-rtems4.11-gdb-7.5.1-1
464building: sparc-rtems4.11-gdb-7.5.1-1
465installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <2>
466installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
467installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
468installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
469tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
470cleaning: expat-2.1.0-x86_64-freebsd9.1-1
471cleaning: sparc-rtems4.11-binutils-2.22-1
472cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
473cleaning: sparc-rtems4.11-gdb-7.5.1-1
474Build Set: Time 0:15:25.92873
475-------------------------------------------------------------
476
477<1> The option to create a build set tar file.
478<2> The installation still happens unless you specify `--no-install`.
479<3> Creating the build set tar file.
480
481You can also suppress installing the files using the `--no-install` option to
482the `sb-set-builder` command. This is usefu if your prefix is not accessiable
483when building Canadian cross compiled tool sets.
484
485-------------------------------------------------------------
486$ ../source-builder/sb-set-builder --log=l-sparc.txt \
487            --prefix=$HOME/development/rtems/4.11 \
488            --bset-tar-file --no-install <1> 4.11/rtems-sparc
489Source Builder - Set Builder, v0.2.0
490Build Set: 4.11/rtems-sparc
491config: expat-2.1.0-1.cfg
492package: expat-2.1.0-x86_64-freebsd9.1-1
493building: expat-2.1.0-x86_64-freebsd9.1-1
494config: tools/rtems-binutils-2.22-1.cfg
495package: sparc-rtems4.11-binutils-2.22-1
496building: sparc-rtems4.11-binutils-2.22-1
497config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
498package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
499building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
500config: tools/rtems-gdb-7.5.1-1.cfg
501package: sparc-rtems4.11-gdb-7.5.1-1
502building: sparc-rtems4.11-gdb-7.5.1-1
503tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <2>
504cleaning: expat-2.1.0-x86_64-freebsd9.1-1
505cleaning: sparc-rtems4.11-binutils-2.22-1
506cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
507cleaning: sparc-rtems4.11-gdb-7.5.1-1
508Build Set: Time 0:14:11.721274
509$ ls tar
510rtems-4.11-sparc-rtems4.11-1.tar.bz2
511-------------------------------------------------------------
512
513<1> The option to supressing installing the packages.
514<2> Create the build set tar.
515
516A package tar file can be created by adding the +--pkg-tar-files+ to the
517+sb-set-builder+ command. This creates a tar file per package built in the
518build set.
519
520-------------------------------------------------------------
521$ ../source-builder/sb-set-builder --log=l-sparc.txt \
522            --prefix=$HOME/development/rtems/4.11 \
523            --bset-tar-file --pkg-tar-files <1> --no-install 4.11/rtems-sparc
524Source Builder - Set Builder, v0.2.0
525Build Set: 4.11/rtems-sparc
526config: expat-2.1.0-1.cfg
527package: expat-2.1.0-x86_64-freebsd9.1-1
528building: expat-2.1.0-x86_64-freebsd9.1-1
529config: tools/rtems-binutils-2.22-1.cfg
530package: sparc-rtems4.11-binutils-2.22-1
531building: sparc-rtems4.11-binutils-2.22-1
532config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
533package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
534building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
535config: tools/rtems-gdb-7.5.1-1.cfg
536package: sparc-rtems4.11-gdb-7.5.1-1
537building: sparc-rtems4.11-gdb-7.5.1-1
538tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2
539cleaning: expat-2.1.0-x86_64-freebsd9.1-1
540cleaning: sparc-rtems4.11-binutils-2.22-1
541cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
542cleaning: sparc-rtems4.11-gdb-7.5.1-1
543Build Set: Time 0:14:37.658460
544$ ls tar
545expat-2.1.0-x86_64-freebsd9.1-1.tar.bz2           sparc-rtems4.11-binutils-2.22-1.tar.bz2
546sparc-rtems4.11-gdb-7.5.1-1.tar.bz2 <2>           rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
547sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1.tar.bz2
548-------------------------------------------------------------
549
550<1> The option to create packages tar files.
551<2> The GDB package tar file.
552<3> The build set tar file. All the others in a single tar file.
553
554Controlling the Build
555~~~~~~~~~~~~~~~~~~~~~
556
557Build sets can be controlled via the command line to enable and disable various
558features. There is no definitive list of build options that can be listed
559because they are implemented with the configuration scripts. The best way to
560find what is available is to grep the configuration files. for +with+ and
561+without+.
562
563Following are currentlt available:
564
565'--without-rtems':: Do not build RTEMS when building an RTEMS build set.
566'--without-cxx':: Do not build a C++ compiler.
567'--with-objc':: Attempt to build a C++ compiler.
568'--with-fortran':: Attempt to build a Fortran compiler.
569
570Why Build from Source ?
571-----------------------
572
573The RTEMS Source Builder is not a replacement for the binary install systems
574you have with commercial operating systems or open source operating system
575distributions. Those products and distributions are critically important and
576are the base that allows the Source Builder to work. The RTEMS Source Builder
577sits somewhere between you manually entering the commands to build a tool set
578and a tool such as +yum+ or +apt-get+ to install binary packages made
579specifically for your host operating system. Building manually or installing a
580binary package from a remote repository are valid and real alternatives while
581the Source Builder is attempting to provide a specific service of repeatably
582being able to build tool sets from source code.
583
584If you are developing a system or product that has a long shelf life or is used
585in a critical piece of infrastructure that has a long life cycle being able to
586build from source is important. It insulates the project from the fast ever
587changing world of the host development machines. If your tool set is binary and
588you have lost the ability to build it you have lost a degree of control and
589flexibility open source gives you. Fast moving host environments are
590fantastic. We have powerful multi-core computers with huge amounts of memory
591and state of the art operating systems to run on them however the product or
592project you are part of may need to be maintained well past the life time of
593these host. Being able to build from source an important and critical part of
594this process because you can move to a newer host and create an equivalent tool
595set.
596
597Building from source provides you with control over the configuration of the
598package you are building. If all or the most important dependent parts are
599built from source you limit the exposure to host variations. For example the
600GNU C compiler (gcc) currently uses a number of 3rd party libraries internally
601(gmp, mpfr, etc). If your validated compiler generating code for your target
602processor is dynamically linked against the host's version of these libraries
603any change in the host's configuration may effect you. The changes the host's
604package management system makes may be perfectly reasonable in relation to the
605distribution being managed however this may not extend to you and your
606tools. Building your tools from source and controlling the specific version of
607these dependent parts means you are not exposing yourself to unexpected and
608often difficult to resolve problems. On the other side you need to make sure
609your tools build and work with newer versions of the host operating
610system. Given the stability of standards based libraries like 'libc' and ever
611improving support for standard header file locations this task is becoming
612easier.
613
614The RTEMS Source Builder is designed to be audited and incorporated into a
615project's verification and validation process. If your project is developing
616critical applications that needs to be traced from source to executable code in
617the target, you need to also consider the tools and how to track them.
618
619If your IT department maintains all your computers and you do not have suitable
620rights to install binary packages, building from source lets you create your
621own tool set that you install under your home directory. Avoiding installing
622any extra packages as a super user is always helpful in maintaining a secure
623computing environment.
624
625[[_bugs]]
626Bugs, Crashes, and Build Failures
627---------------------------------
628
629The RTEMS Source Builder is a Python program and every care is taken to test
630the code however bugs, crashes, and build failures can and do happen. If you
631find a bug please report it via the RTEMS Bug tracker tool Bugzilla or via
632email on the RTEMS Users list. RTEMS's bugzilla is found at
633https://www.rtems.org/bugzilla/.
634
635Please include the generated RSB report. If you see the following a report has
636been generated:
637
638-------------------------------------------------------------
639 ...
640 ...
641Build FAILED <1>
642  See error report: rsb-report-4.11-rtems-lm32.txt <2>
643-------------------------------------------------------------
644<1> The build has failed.
645<2> The report's file name.
646
647The generated report contains the command line, version of the RSB, your host's
648+uname+ details, the version of Python and the last 200 lines of the log.
649
650If for some reason there is no report please send please report the following:
651
652. Command line,
653. The git hash,
654. Host details with the output of the +uname -a+ command,
655. If you have made any modifications.
656
657If there is a Python crash please cut and paste the Python backtrace into the
658bug report. If the tools fail to build please locate the first error in the log
659file. This can be difficult to find on hosts with many cores so it sometimes
660pays to re-run the command with the +--jobs=none+ option to get a log that is
661correctly sequenced. If searching the log file seach for +error:+ and the error
662should be just above it.
663
664[[_contributing]]
665Contributing
666------------
667
668We welcome all users adding, fixing, updating and upgrading packages and their
669configurations. The RSB is open source and open to contributions. These can be
670bug fixes, new features or new configurations. Please break patches down into
671changes to the core Python code, configuration changes or new configurations.
672
673Please email me patches via git so I can maintain your commit messages so you
674are acknowledged as the contributor.
675
676Most of what follows is related to the development of RSB and it's
677configurations.
678
679Project Sets
680------------
681
682The RTEMS Source Builder supports project configurations. Project
683configurations can be public or private and can be contained in the RTEMS
684Source Builder project if suitable, other projects they use the RTEM Source
685Builder or privately on your local file system.
686
687The configuration file loader searches the macro +_configdir+ and by default
688this is set to +%{\_topdir}/config:%{\_sbdir}/config+ where +_topdir+ is the
689your current working direct, in other words the directory you invoke the RTEMS
690Source Builder command in, and +_sbdir+ is the directory where the RTEMS Source
691Builder command resides. Therefore the +config+ directory under each of these
692is searched so all you need to do is create a +config+ in your project and add
693your configuration files. They do not need to be under the RTEMS Source Builder
694source tree. Public projects are included in the main RTEMS Source Builder such
695as RTEMS.
696
697You can also add your own +patches+ directory next to your +config+ directory
698as the +%patch+ command searches the +_patchdir+ macro variable and it is
699by default set to +%{\_topdir}/patches:%{\_sbdir}/patches+.
700
701The +source-builder/config+ directory provides generic scripts for building
702various tools. You can specialise these in your private configurations to make
703use of them. If you add new generic configurations please contribute them back
704to the project
705
706Bare Metal
707~~~~~~~~~~
708
709The RSB contains a 'bare' configuration tree and you can use this to add
710packages you use on the hosts. For example 'qemu' is supported on a range of
711hosts. RTEMS tools live in the +rtems/config+ directory tree. RTEMS packages
712include tools for use on your host computer as well as packages you can build
713and run on RTEMS.
714
715The 'bare metal' support for GNU Tool chains. An example is the 'lang/gcc491'
716build set. You need to provide a target via the command line '--target'
717option and this is in the standard 2 or 3 tuple form. For example for an ARM
718compiler you would use 'arm-eabi' or 'arm-eabihf', and for SPARC you would
719use 'sparc-elf'.
720
721-------------------------------------------------------------
722$ cd rtems-source-builder/bare
723$../source-builder/sb-set-builder --log=log_arm_eabihf \
724    --prefix=$HOME/development/bare --target=arm-eabihf lang/gcc491
725RTEMS Source Builder - Set Builder, v0.3.0
726Build Set: lang/gcc491
727config: devel/expat-2.1.0-1.cfg
728package: expat-2.1.0-x86_64-apple-darwin13.2.0-1
729building: expat-2.1.0-x86_64-apple-darwin13.2.0-1
730config: devel/binutils-2.24-1.cfg
731package: arm-eabihf-binutils-2.24-1
732building: arm-eabihf-binutils-2.24-1
733config: devel/gcc-4.9.1-newlib-2.1.0-1.cfg
734package: arm-eabihf-gcc-4.9.1-newlib-2.1.0-1
735building: arm-eabihf-gcc-4.9.1-newlib-2.1.0-1
736config: devel/gdb-7.7-1.cfg
737package: arm-eabihf-gdb-7.7-1
738building: arm-eabihf-gdb-7.7-1
739installing: expat-2.1.0-x86_64-apple-darwin13.2.0-1 -> /Users/chris/development/bare
740installing: arm-eabihf-binutils-2.24-1 -> /Users/chris/development/bare
741installing: arm-eabihf-gcc-4.9.1-newlib-2.1.0-1 -> /Users/chris/development/bare
742installing: arm-eabihf-gdb-7.7-1 -> /Users/chris/development/bare
743cleaning: expat-2.1.0-x86_64-apple-darwin13.2.0-1
744cleaning: arm-eabihf-binutils-2.24-1
745cleaning: arm-eabihf-gcc-4.9.1-newlib-2.1.0-1
746cleaning: arm-eabihf-gdb-7.7-1
747-------------------------------------------------------------
748
749RTEMS
750~~~~~
751
752The RTEMS Configurations found in the 'rtems' directory. The configurations are
753grouped by RTEMS version. In RTEMS the tools are specific to a specific version
754because of variations between Newlib and RTEMS. Restructuring in RTEMS and
755Newlib sometimes moves _libc_ functionality between these two parts and this
756makes existing tools incompatible with RTEMS.
757
758RTEMS allows architectures to have different tool versions and patches. The
759large number of architectures RTEMS supports can make it difficult to get a
760common stable version of all the packages. An architecture may require a recent
761GCC because an existing bug has been fixed, however the more recent version may
762have a bug in other architecture. Architecture specific patches should be
763limited to the architecture it relates to. The patch may fix a problem on the
764effect architecture however it could introduce a problem in another
765architecture. Limit exposure limits any possible crosstalk between
766architectures.
767
768RTEMS supports _stable_ and _unstable_. Stable configurations are fixed while
769unstable configurations are supporting using snapshots of user macros and
770reference release candidates or source extracted directly from version
771control. The stable build sets are referenced as +<version>/rtems-<arch>+ and
772include `autoconf` and `automake`.
773
774If you are building a released version of RTEMS the release RTEMS tar file will
775be downloaded and built as part of the build process. If you are building a
776tool set for use with the development branch of RTEMS, the development branch
777will be cloned directly from the RTEMS GIT repository and built.
778
779When building RTEMS within the RTEMS Source Builder it needs a suitable working
780`autoconf` and `automake`. These packages need to built and installed in their
781prefix in order for them to work. The RTEMS Source Builder installs all
782packages only after they have been built so if you host does not have a
783recent enough version of `autoconf` and `automake` you first need to build them
784and install them then build your tool set. The commands are:
785
786-------------------------------------------------------------
787$ ../source-builder/sb-set-builder --log=l-4.11-at.txt \
788   --prefix=$HOME/development/rtems/4.11 4.11/rtems-autotools
789$ export PATH=~/development/rtems/4.11/bin:$PATH <1>
790$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
791   --prefix=$HOME/development/rtems/4.11 4.11/rtems-sparc
792-------------------------------------------------------------
793<1> Setting the path.
794
795TIP: If this is your first time building the tools and RTEMS it pays to add the
796`--dry-run` option. This will run through all the configuration files and if
797any checks fail you will see this quickly rather than waiting for until the
798build fails a check.
799
800To build snapshots for testing purposes you use the available macro maps
801passing them on the command line using the `--macros` option. For RTEMS these
802are held in `config/snapshots` directory. The following builds _newlib_ from
803CVS:
804
805-------------------------------------------------------------
806$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
807   --prefix=$HOME/development/rtems/4.11 \
808   --macros=snapshots/newlib-head.mc \
809   4.11/rtems-sparc
810-------------------------------------------------------------
811
812and the following uses the version control heads for _binutils_, _gcc_,
813_newlib_, _gdb_ and _RTEMS_:
814
815-------------------------------------------------------------
816$ ../source-builder/sb-set-builder --log=l-heads-sparc.txt \
817   --prefix=$HOME/development/rtems/4.11-head \
818   --macros=snapshots/binutils-gcc-newlib-gdb-head.mc \
819   4.11/rtems-sparc
820-------------------------------------------------------------
821
822Patches
823~~~~~~~
824
825Packages being built by the RSB need patches from time to time and the RSB
826supports patching upstream packages. The patches are held in a seperate
827directory called +patches+ relative to the configuration directory you are
828building. For example +%{\_topdir}/patches:%{\_sbdir}/patches+. Patches are
829declared in the configuraiton files in a similar manner to the package's source
830so please refer to the +%source+ documentation. Patches, like the source, are
831to be made publically available for configurations that live in the RSB package
832and are downloaded on demand.
833
834If a package has a patch management tool it is recommended you reference the
835package's patch management tools directly. If the RSB does not support the
836specific patch manage tool please contact the mailing list to see if support
837can be added.
838
839Patches for packages developed by the RTEMS project can be placed in the RTEMS
840Tools Git repository. The +tools+ directory in the repository has various
841places a patch can live. The tree is broken down in RTEMS releases and then
842tools within that release. If the package is not specific to any release the
843patch can be added closer to the top under the package's name. Patches to fix
844specific tool related issues for a specific architecture should be grouped
845under the specific architecture and only applied when building that
846architecture avoiding a patch breaking an uneffected architecture.
847
848Patches in the RTEMS Tools repository need to be submitted to the upstream
849project. It should not be a clearing house for patches that will not be
850accepted upstream.
851
852Patches are added to a component's name and in the +%prep:+ section the patches
853can be set up, meaning they are applied to source. The patches are applied in
854the order they are added. If there is a dependency make sure you order the
855patches correctly when you add them. You can add any number of patches and the
856RSB will handle them efficently.
857
858Patches can have options. These are added before the patch URL. If no options
859are provided the patch's setup default options are used.
860
861Patches can be declared in build set up files.
862
863This examples shows how to declare a patch for gdb in the +lm32+ architecture:
864
865-------------------------------------------------------------
866%patch add <1> gdb <2> %{rtems_gdb_patches}/lm32/gdb-sim-lm32uart.diff <3>
867-------------------------------------------------------------
868<1> The patch's +add+ command.
869<2> The group of patches this patch belongs too.
870<3> The patch's URL. It is downloaded from here.
871
872Patches require a checksum to avoid a warning. The +%hash+ directive can be
873used to add a checksum for a patch that is used to verify the patch:
874
875-------------------------------------------------------------
876%hash md5 <1> gdb-sim-lm32uart.diff <2> 77d070878112783292461bd6e7db17fb <3>
877-------------------------------------------------------------
878<1> The type of checksum, in the case an MD5 hash.
879<2> The patch file the checksum is for.
880<3> The MD5 hash.
881
882The patches are applied when a patch +setup+ command is issued in the +%prep:+
883section. All patches in the group are applied. To apply the GDB patch above
884use:
885
886-------------------------------------------------------------
887%patch setup <1> gdb <2> -p1 <3>
888-------------------------------------------------------------
889<1> The patch's +setup+ command.
890<2> The group of patches to apply.
891<3> The patch group's default options. If no option is given with the patch
892these options are used.
893
894Architecture specific patches live in the architecture build set file isolating
895the patch to that specific architecture. If a patch is common to a tool it
896resides in the RTEMS tools configuration file. Do not place patches for tools
897in the +source-builder/config+ template configuration files.
898
899To test a patch simply copy it to your local +patches+ directory. The RSB will
900see the patch is present and will not attempt to download it. Once you are
901happy with the patch submit it to the project and a core developer will review
902it and add it to the RTEMS Tools git repository.
903
904File Checksums
905
906Cross and Canadian Cross Building
907---------------------------------
908
909Cross building and Canadian Cross building is the process of building on one
910machine an executable that runs on another machine. An example is building a
911set of RTEMS tools on Linux to run on Windows. The RSB supports cross building
912and Canadian cross building.
913
914This sections details how to the RSB to cross and Canadian cross build.
915
916Cross Building
917~~~~~~~~~~~~~~
918
919Cross building is where the _build_ machine and _host_ are different. The
920_build_ machine runs the RSB and the _host_ machine is where the output from
921the build runs. An example is building a package such as NTP for RTEMS on your
922development machine.
923
924To build the NTP package for RTEMS you enter the RSB command:
925
926-------------------------------------------------------------
927$ ../source-builder/sb-set-builder \
928   --log=log_ntp_arm.txt \
929   --prefix=$HOME/development/rtems/4.11 <1> \
930   --host=arm-rtems4.11 <2> \
931   --with-rtems-bsp=xilinx_zynq_zc706 <3> \
932   4.11/net/ntp
933-------------------------------------------------------------
934<1> The tools and the RTEMS BSP are installed under the same prefix.
935<2> The +--host+ command is the RTEMS architecture and version.
936<3> The BSP is built and installed in the prefix. The arhcitecture must
937    match the +--host+ architecture.
938
939TIP: If you install BSPs into a different path to the prefix use the
940+--with-tools+ option to specify the path to the tools. Do not add the 'bin'
941directory at the end of the path.
942
943Canadian Cross Building
944~~~~~~~~~~~~~~~~~~~~~~~
945
946A Canadian cross builds are where the _build_, _host_ and _target_ machines all
947differ. For example building an RTEMS compiler for an ARM processor that runs
948on Windows is built using a Linux machine. The process is controlled by setting
949the build triplet to the host you are building, the host triplet to the host
950the tools will run on and the target to the RTEMS architecture you require. The
951tools needed by the RSB are:
952
953. Build host C and C++ compiler
954. Host C and C++ cross compiler
955
956The RTEMS Source Builder requires you provide the build host C and C\++
957compiler and the final host C and C\++ cross-compiler. The RSB will build the
958build host RTEMS compiler and the final host RTEMS C and C++ compiler, the
959output of this process.
960
961The Host C and C++ compiler is a cross-compiler that builds executables for
962the host you want the tools for. You need to provide these tools. For Windows a
963number of Unix operating systems provide MinGW tool sets as packages.
964
965The RSB will build an RTEMS tool set for the build host. This is needed when
966building the final host's RTEMS compiler as it needs to build RTEMS runtime
967code such as _libc_ on the build host.
968
969TIP: Make sure the host's cross-compiler tools are in your path before run the
970RSB build command.
971
972TIP: Canadian Cross built tools will not run on the machine being used to build
973them so you should provide the +--bset-tar-files+ and +--no-install+
974options. The option to not install the files lets you provide a prefix that
975does not exist or you cannot access.
976
977To perform a cross build add +--host=+ to the command line. For example to
978build a MinGW tool set on FreeBSD for Windows add +--host=mingw32+ if the cross
979compiler is +mingw32-gcc+:
980
981-------------------------------------------------------------
982$ ../source-builder/sb-set-builder --host=mingw32 \
983   --log=l-mingw32-4.11-sparc.txt \
984   --prefix=$HOME/development/rtems/4.11 \
985   4.11/rtems-sparc
986-------------------------------------------------------------
987
988If you are on a Linux Fedora build host with the MinGW packages installed the
989command line is:
990
991-------------------------------------------------------------
992$ ../source-builder/sb-set-builder --host=i686-w64-mingw32 \
993   --log=l-mingw32-4.11-sparc.txt \
994   --prefix=$HOME/development/rtems/4.11 \
995   4.11/rtems-sparc
996-------------------------------------------------------------
997
998RTEMS 3rd Party Packages
999------------------------
1000
1001This section describes how to build and add an RTEMS 3rd party package to the
1002RSB.
1003
1004A 3rd party package is a library or software package built to run on RTEMS,
1005examples are NTP, Net-Snmp, libjpeg or Python. These pieces of software can be
1006used to help build RTEMS applications. The package is built for a specific
1007BSP and so requires a working RTEMS tool chain and an installed RTEMS Board
1008Support Package (BSP).
1009
1010The RSB support for building 3rd part packages is based around the pkconfig
1011files (PC) installed with the BSP. The pkgconfig support in RTEMS is considered
1012experimental and can have some issues for some BSPs. This issue is rooted deep
1013in the RTEMS build system. If you have any issues with this support please ask
1014on the RTEMS developers mailing list.
1015
1016Building
1017~~~~~~~~
1018
1019To build a package you need to have a suitable RTEMS tool chain and RTEMS BSP
1020installed. The set builder command line requires you provide the tools path,
1021the RTEMS host, and the prefix path to the installed RTEMS BSP. The prefix
1022needs to be the same as the prefix used to build RTEMS.
1023
1024To build Net-SNMP the command is:
1025
1026-------------------------------------------------------------
1027cd rtems-source-builder/rtems
1028$ ../source-builder/sb-set-builder --log=log_sis_net_snmp \
1029    --prefix=$HOME/development/rtems/bsps/4.11 \
1030    --with-tools=$HOME/development/rtems/4.11 \
1031    --host=sparc-rtems4.11 --with-rtems-bsp=sis 4.11/net-mgmt/net-snmp
1032RTEMS Source Builder - Set Builder, v0.3.0
1033Build Set: 4.11/net-mgmt/net-snmp
1034config: net-mgmt/net-snmp-5.7.2.1-1.cfg
1035package: net-snmp-5.7.2.1-sparc-rtems4.11-1
1036building: net-snmp-5.7.2.1-sparc-rtems4.11-1
1037installing: net-snmp-5.7.2.1-sparc-rtems4.11-1 -> /Users/chris/development/rtems/bsps/4.11
1038cleaning: net-snmp-5.7.2.1-sparc-rtems4.11-1
1039Build Set: Time 0:01:10.651926
1040-------------------------------------------------------------
1041
1042Adding
1043~~~~~~
1044
1045Adding a package requires you first build it manually by downloading the source
1046for the package and building it for RTEMS using the command line of a standard
1047shell. If the package has not been ported to RTEMS you will need to port it and
1048this may require you asking questions on the package's user or development
1049support lists as well as RTEMS's developers list. Your porting effort may end
1050up with a patch. RTEMS requires a patch be submitted upstream to the project's
1051community as well as RTEMS so it can be added to the RTEMS Tools git
1052repository. A patch in the RTEMS Tools git reposiitory can then be referenced
1053by an RSB configuration file.
1054
1055A package may create executables, for example NTP normally creates executables
1056such as +ntdp+, +ntpupdate+, or +ntpdc+. These executables can be useful when
1057testing the package however they are of limited use by RTEMS users because they
1058cannot be directly linked into a user application. Users need to link to the
1059functions in these executables or even the executable as a function placed in
1060libraries. If the package does not export the code in a suitable manner please
1061contact the project's commuinity and see if you can work them to provide a way
1062for the code to be exported. This may be difficult because exporting internal
1063headers and functions opens the project up to API compatibility issues they did
1064not have before. In the simplest case attempting to get the code into a static
1065library with a single call entry point exported in a header would give RTEMS
1066user's access to the package's main functionality.
1067
1068A package requires 3 files to be created:
1069
1070. The first file is the RTEMS build set file and it resides in the
1071  +$$rtems/config/%{rtems_version}$$+ path in a directory tree based on the
1072  FreeBSD ports collection. For the NTP package and RTEMS 4.11 this is
1073  +rtems/config/4.11/net/ntp.bset+. If you do not know the FreeBSD port path
1074  for the package you are adding please ask. The build set file references a
1075  specific configuration file therefore linking the RTEMS version to a specific
1076  version of the package you are adding. Updating the package to a new version
1077  requires changing the build set to the new configuration file.
1078
1079. The second file is an RTEMS version specific configuration file and it
1080  includes the RSB RTEMS BSP support. These configuration files reside in the
1081  +rtems/config+ tree again under the FreeBSP port's path name. For example the
1082  NTP package is found in the +net+ directory of the FreeBSD ports tree so the
1083  NTP configuration path is +$$rtems/config/net/ntp-4.2.6p5-1.cfg$$+ for that
1084  specific version. The configuration file name typically provides version
1085  specific references and the RTEMS build set file references a specific
1086  version. This configuration file references the build configuration file held
1087  in the common configuration file tree.
1088
1089. The build configuration. This is a common script that builds the package. It
1090  resides in the +source-builder/config+ directory and typically has the
1091  packages's name with the major version number. If the build script does not
1092  change for each major version number a _common_ base script can be created
1093  and included by each major version configuration script. The _gcc_ compiler
1094  configuration is an example. This approach lets you branch a version if
1095  something changes that is not backwards compatible. It is important to keep
1096  existing versions building. The build configuration should be able to build a
1097  package for the build host as well as RTEMS as the RSB abstracts the RTEMS
1098  specific parts. See <<H1,+_Configuration_+>> for more details.
1099
1100BSP Support
1101~~~~~~~~~~~
1102
1103The RSB provides support to help build packages for RTEMS. RTEMS applications
1104can be viewed as statically linked executables operating in a single address
1105space. As a result only the static libraries a package builds are required and
1106these libraries need to be ABI compatible with the RTEMS kernel and application
1107code meaning compiler ABI flags cannot be mixed when building code. The 3rd
1108party package need to use the same compiler flags as the BSP used to build
1109RTEMS.
1110
1111[TIP]
1112=============================================================
1113
1114RTEMS's dynamic loading support does not use the standard shared library
1115support found in Unix and the ELF standard. RTEMS's loader uses static
1116libraries and the runtime link editor performs a similar function to a host
1117based static linker. RTEMS will only reference static libraries even if dynamic
1118libraries are created and installed.
1119
1120=============================================================
1121
1122The RSB provides the configuration file +rtems/config/rtems-bsp.cfg+ to support
1123building 3rd party packages and you need to include this file in your RTEMS
1124version specific configuration file. For example the Net-SNMP configuration
1125file:
1126
1127.rtems/config/net-mgmt/net-snmp-5.7.2.1-1.cfg
1128-------------------------------------------------------------
1129#
1130# NetSNMP 5.7.2.1
1131#
1132
1133%if %{release} == %{nil}
1134 %define release 1 <1>
1135%endif
1136
1137%include %{_configdir}/rtems-bsp.cfg <2>
1138
1139#
1140# NetSNMP Version
1141#
1142%define net_snmp_version 5.7.2.1 <3>
1143
1144#
1145# We need some special flags to build this version.
1146#
1147%define net_snmp_cflags <4> -DNETSNMP_CAN_USE_SYSCTL=1 -DARP_SCAN_FOUR_ARGUMENTS=1 -DINP_IPV6=0
1148
1149#
1150# Patch for RTEMS support.
1151#
1152%patch add net-snmp %{rtems_git_tools}/net-snmp/rtems-net-snmp-5.7.2.1-20140623.patch <5>
1153
1154#
1155# NetSNMP Build configuration
1156#
1157%include %{_configdir}/net-snmp-5-1.cfg <6>
1158-------------------------------------------------------------
1159<1> The release number.
1160<2> Include the RSB RTEMS BSP support.
1161<3> The Net-SNMP package's version.
1162<4> Add specific CFLAGS to the build process. See the +net-snmp-5.7.2.1-1.cfg+
1163    for details.
1164<5> The RTEMS Net-SNMP patch downloaded from the RTEMS Tools git repo.
1165<6> The Net-SNMP standard build configuration.
1166
1167The RSB RTEMS BSP support file +rtems/config/rtems-bsp.cfg+ checks to make sure
1168standard command line options are provided. These include `--host` and
1169`--with-rtems-bsp`. If the `--with-tools` command line option is not given the
1170+${\_prefix}+ is used.
1171
1172.rtems/config/rtems-bsp.cfg
1173-------------------------------------------------------------
1174%if %{_host} == %{nil} <1>
1175 %error No RTEMS target specified: --host=host
1176%endif
1177
1178%ifn %{defined with_rtems_bsp} <2>
1179 %error No RTEMS BSP specified: --with-rtems-bsp=bsp
1180%endif
1181
1182%ifn %{defined with_tools} <3>
1183 %define with_tools %{_prefix}
1184%endif
1185
1186#
1187# Set the path to the tools.
1188#
1189%{path prepend %{with_tools}/bin} <4>
1190-------------------------------------------------------------
1191<1> Check the host has been set.
1192<2> Check a BSP has been specified.
1193<3> If no tools path has been provided assume they are under the %{\_prefix}.
1194<4> Add the tools +bin+ path to the system path.
1195
1196RTEMS exports the build flags used in pkgconfig (.pc) files and the RSB can
1197read and manage them even when there is no pkgconfig support installed on your
1198build machine. Using this support we can obtain a BSP's configuration and set
1199some standard macros variables:
1200
1201.rtems/config/rtems-bsp.cfg
1202-------------------------------------------------------------
1203%{pkgconfig prefix %{_prefix}/lib/pkgconfig} <1>
1204%{pkgconfig crosscompile yes} <2>
1205%{pkgconfig filter-flags yes} <3>
1206
1207#
1208# The RTEMS BSP Flags
1209#
1210%define rtems_bsp           %{with_rtems_bsp}
1211%define rtems_bsp_ccflags   %{pkgconfig ccflags %{_host}-%{rtems_bsp}} <4>
1212%define rtems_bsp_cflags    %{pkgconfig cflags  %{_host}-%{rtems_bsp}}
1213%define rtems_bsp_ldflags   %{pkgconfig ldflags %{_host}-%{rtems_bsp}}
1214%define rtems_bsp_libs      %{pkgconfig libs    %{_host}-%{rtems_bsp}}
1215-------------------------------------------------------------
1216<1> Set the path to the BSP's pkgconfig file.
1217<2> Let pkgconfig know this is a cross-compile build.
1218<3> Filter flags such as warnings. Warning flags are specific to a package.
1219<4> Ask pkgconfig for the various items we require.
1220
1221The flags obtain by pkgconfig and given a `rtems_bsp_` prefix and we uses these
1222to set the RSB host support CFLAGS, LDFLAGS and LIBS flags. When we build a 3rd
1223party library your host computer is the _build_ machine and RTEMS is the _host_
1224machine therefore we set the `host` variables:
1225
1226.rtems/config/rtems-bsp.cfg
1227-------------------------------------------------------------
1228%define host_cflags  %{rtems_bsp_cflags}
1229%define host_ldflags %{rtems_bsp_ldflags}
1230%define host_libs    %{rtems_bsp_libs}
1231-------------------------------------------------------------
1232
1233Finally we provide all the paths you may require when configuring a
1234package. Packages by default consider the `_prefix` the base and install
1235various files under this tree. The package you are building is specific to a
1236BSP and so needs to install into the specific BSP path under the
1237`_prefix`. This allows more than BSP build of this package to be install under
1238the same `_prefix` at the same time:
1239
1240.rtems/config/rtems-bsp.cfg
1241-------------------------------------------------------------
1242%define rtems_bsp_prefix  %{_prefix}/%{_host}/%{rtems_bsp} <1>
1243%define _exec_prefix      %{rtems_bsp_prefix}
1244%define _bindir           %{_exec_prefix}/bin
1245%define _sbindir          %{_exec_prefix}/sbin
1246%define _libexecdir       %{_exec_prefix}/libexec
1247%define _datarootdir      %{_exec_prefix}/share
1248%define _datadir          %{_datarootdir}
1249%define _sysconfdir       %{_exec_prefix}/etc
1250%define _sharedstatedir   %{_exec_prefix}/com
1251%define _localstatedir    %{_exec_prefix}/var
1252%define _includedir       %{_libdir}/include
1253%define _lib              lib
1254%define _libdir           %{_exec_prefix}/%{_lib}
1255%define _libexecdir       %{_exec_prefix}/libexec
1256%define _mandir           %{_datarootdir}/man
1257%define _infodir          %{_datarootdir}/info
1258%define _localedir        %{_datarootdir}/locale
1259%define _localedir        %{_datadir}/locale
1260%define _localstatedir    %{_exec_prefix}/var
1261-------------------------------------------------------------
1262<1> The path to the BSP.
1263
1264When you configure a package you can reference these paths and the RSB will
1265provide sensible default or in this case map them to the BSP:
1266
1267.source-builder/config/ntp-4-1.cfg
1268-------------------------------------------------------------
1269  ../${source_dir_ntp}/configure \ <1>
1270    --host=%{_host} \
1271    --prefix=%{_prefix} \
1272    --bindir=%{_bindir} \
1273    --exec_prefix=%{_exec_prefix} \
1274    --includedir=%{_includedir} \
1275    --libdir=%{_libdir} \
1276    --libexecdir=%{_libexecdir} \
1277    --mandir=%{_mandir} \
1278    --infodir=%{_infodir} \
1279    --datadir=%{_datadir} \
1280    --disable-ipv6 \
1281    --disable-HOPFPCI
1282-------------------------------------------------------------
1283<1> The configure command for NTP.
1284
1285RTEMS BSP Configuration
1286~~~~~~~~~~~~~~~~~~~~~~~
1287
1288To build a package for RTEMS you need to build it with the matching BSP
1289configuration. A BSP can be built with specific flags that require all code
1290being used needs to be built with the same flags.
1291
1292
1293[[H1]]
1294Configuration
1295-------------
1296
1297The RTEMS Source Builder has two types of configuration data:
1298
1299. Build Sets
1300. Package Build Configurations
1301
1302By default these files can be located in two separate directories and
1303searched. The first directory is +config+ in your current working directory
1304(+_topdir+) and the second is +config+ located in the base directory of the
1305RTEMS Source Builder command you run (+_sbdir+). The RTEMS directory +rtems+
1306located at the top of the RTEMS Source Builder source code is an example of a
1307specific build configuration directory. You can create custom or private build
1308configurations and if you run the RTEMS Source Builder command from that
1309directory your configurations will be used.
1310
1311[[X1]] The configuration search path is a macro variable and is reference as
1312+%\{_configdir\}+. It's default is defined as:
1313
1314-------------------------------------------------------------
1315_configdir   : dir  optional<2>  %{_topdir}/config:%{_sbdir}/config <1>
1316-------------------------------------------------------------
1317
1318<1> The +_topdir+ is the directory you run the command from and +_sbdir+ is the
1319location of the RTEMS Source Builder command.
1320<2> A macro definition in a macro file has 4 fields, the label, type,
1321constraint and the definition.
1322
1323Build set files have the file extension +.bset+ and the package build
1324configuration files have the file extension of +.cfg+. The +sb-set-builder+
1325command will search for _build sets_ and the +sb-builder+ commands works with
1326package build configuration files.
1327
1328Both types of configuration files use the \'#' character as a comment
1329character. Anything after this character on the line is ignored. There is no
1330block comment.
1331
1332Source and Patches
1333~~~~~~~~~~~~~~~~~~
1334
1335The RTEMS Source Builder provides a flexible way to manage source. Source and
1336patches are declare in configurations file using the +source+ and +patch+
1337directives. These are a single line containing a Universal Resource Location or
1338URL and can contain macros and shell expansions. The <<_prep,%prep>> section
1339details the source and patch directives
1340
1341The URL can reference remote and local source and patch resources. The
1342following schemes are provided:
1343
1344'http':: Remote access using the HTTP protocol.
1345'https':: Remote access using the Secure HTTP protocol.
1346'ftp':: Remote access using the FTP protocol.
1347'git':: Remote access to a GIT repository.
1348'cvs':: Remote access to a CVS repository.
1349'pm':: Remote access to a patch management repository.
1350'file':: Local access to an existing source directory.
1351
1352HTTP, HTTPS, and FTP
1353^^^^^^^^^^^^^^^^^^^^
1354
1355Remote access to TAR or ZIP files is provided using HTTP, HTTPS and FTP
1356protocols. The full URL provided is used to access the remote file including
1357any query components. The URL is parsed to extract the file component and the
1358local source directory is checked for that file. If the file is located locally
1359the remote file is not downloaded. Currently no other checks are made. If a
1360download fails you need to manually remove the file from the source directory
1361and start the build process again.
1362
1363The URL can contain macros. These are expanded before issuing the request to
1364download the file. The standard GNU GCC compiler source URL is:
1365
1366-------------------------------------------------------------
1367%source set<1> gcc<2> ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
1368-------------------------------------------------------------
1369<1> The +%source+ command's set command sets the source. The first is set and
1370following sets are ignored.
1371<2> The source is part of the +gcc+ group.
1372
1373The type of compression is automatically detected from the file extension. The
1374supported compression formats are:
1375
1376'gz':: GNU ZIP
1377'bzip2':: BZIP2
1378'zip':: ZIP
1379'xy':: XY
1380
1381The output of the decompression tool is feed to the standard `tar` utility if
1382not a ZIP file and unpacked into the build directory. ZIP files are unpacked by
1383the decompression tool and all other files must be in the tar file format.
1384
1385The +%source+ directive typically supports a single source file tar or zip
1386file. The +set+ command is used to set the URL for a specific source group. The
1387first set command encoutner is registered and any further set commands are
1388ignored. This allows you to define a base standard source location and override
1389it in build and architecture specific files. You can also add extra source
1390files to a group. This is typically done when a collection of source is broken
1391down in a number of smaller files and you require the full package. The
1392source's +setup+ command must reide in the +%prep:+ section and it unpacks the
1393source code ready to be built.
1394
1395If the source URL references the GitHub API server 'https://api.github.com/' a
1396tarball of the specified version is download. For example the URL for the
1397STLINK project on GitHub and version is:
1398
1399-------------------------------------------------------------
1400%define stlink_version 3494c11
1401%source set stlink https://api.github.com/repos/texane/stlink/texane-stlink-%{stlink_version}.tar.gz
1402-------------------------------------------------------------
1403
1404GIT
1405^^^
1406
1407A GIT repository can be cloned and used as source. The GIT repository resides
1408in the 'source' directory under the `git` directory. You can edit, update and
1409use the repository as you normally do and the results will used to build the
1410tools. This allows you to prepare and test patches in the build environment the
1411tools are built in. The GIT URL only supports the GIT protocol. You can control
1412the repository via the URL by appending options and arguments to the GIT
1413path. The options are delimited by `?` and option arguments are delimited from
1414the options with `=`. The options are:
1415
1416`protocol`:: Use a specific protocol. The supported values are _ssh_, _git_,
1417_http_, _https_, _ftp_, _ftps_, _rsync_, and _none_.
1418`branch`:: Checkout the specified branch.
1419`pull`:: Perform a pull to update the repository.
1420`fetch`:: Perform a fetch to get any remote updates.
1421`reset`:: Reset the repository. Useful to remove any local changes. You can
1422pass the `hard` argument to force a hard reset.
1423
1424-------------------------------------------------------------
1425%source set gcc git://gcc.gnu.org/git/gcc.git?branch=gcc-4_7-branch?reset=hard
1426-------------------------------------------------------------
1427
1428This will clone the GCC git repository and checkout the 4.7-branch and perform
1429a hard reset. You can select specific branches and apply patches. The
1430repository is cleaned up before each build to avoid various version control
1431errors that can arise.
1432
1433The protocol option lets you set a specific protocol. The 'git://' prefix used
1434by the RSB to select a git repository can be removed using _none_ or replaced
1435with one of the standard git protcols.
1436
1437CVS
1438^^^
1439
1440A CVS repository can be checked out. CVS is more complex than GIT to handle
1441because of the modules support. This can effect the paths the source ends up
1442in. The CVS URL only supports the CVS protocol. You can control the repository
1443via the URL by appending options and arguments to the CVS path. The options are
1444delimited by `?` and option arguments are delimited from the options with
1445`=`. The options are:
1446
1447`module`:: The module to checkout.
1448`src-prefix`:: The path into the source where the module starts.
1449`tag`:: The CVS tag to checkout.
1450`date`:: The CVS date to checkout.
1451
1452-------------------------------------------------------------
1453%source set newlib cvs://pserver:anoncvs@sourceware.org/cvs/src?module=newlib?src-prefix=src
1454-------------------------------------------------------------
1455
1456Macros and Defaults
1457~~~~~~~~~~~~~~~~~~~
1458
1459The RTEMS Source Builder uses tables of _macros_ read in when the tool
1460runs. The initial global set of macros is called the _defaults_. These values
1461are read from a file called `defaults.mc` and modified to suite your host. This
1462host specific adaption lets the Source Builder handle differences in the build
1463hosts.
1464
1465Build set and configuration files can define new values updating and extending
1466the global macro table. For example builds are given a release number. This is
1467typically a single number at the end of the package name. For example:
1468
1469-------------------------------------------------------------
1470%define release 1
1471-------------------------------------------------------------
1472
1473Once defined if can be accessed in a build set or package configuration file
1474with:
1475
1476-------------------------------------------------------------
1477%{release}
1478-------------------------------------------------------------
1479
1480The +sb-defaults+ command lists the defaults for your host. I will not include
1481the output of this command because of its size.
1482
1483-------------------------------------------------------------
1484$ ../source-builder/sb-defaults
1485-------------------------------------------------------------
1486
1487A nested build set is given a separate copy of the global macro maps. Changes
1488in one change set are not seen in other build sets. That same happens with
1489configuration files unless inline includes are used. Inline includes are seen
1490as part of the same build set and configuration and changes are global to that
1491build set and configuration.
1492
1493Macro Maps and Files
1494^^^^^^^^^^^^^^^^^^^^
1495
1496Macros are read in from files when the tool starts. The default settings are
1497read from the defaults macro file called `defaults.mc` located in the top level
1498RTEMS Source Builder command directory. User macros can be read in at start up
1499by using the `--macros` command line option.
1500
1501The format for a macro in macro files is:
1502
1503[options="header,compact",width="50%",cols="15%,15%,15%,55%"]
1504|=================================
1505| Name | Type | Attribute | String
1506|=================================
1507
1508where 'Name' is a case insensitive macro name, the 'Type' field is:
1509
1510[horizontal]
1511`none`:: Nothing, ignore.
1512`dir`:: A directory path.
1513`exe`:: An executable path.
1514`triplet`:: A GNU style architecture, platform, operating system string.
1515
1516the 'Attribute' field is:
1517
1518[horizontal]
1519`none`:: Nothing, ignore
1520`required`:: The host check must find the executable or path.
1521`optional`:: The host check generates a warning if not found.
1522`override`:: Only valid outside of the `global` map to indicate this macro
1523             overrides the same one in the `global` map when the map containing
1524             it is selected.
1525`undefine`:: Only valid outside of the `global` map to undefine the macro if it
1526             exists in the `global` map when the map containing it is
1527             selected. The `global` map's macro is not visible but still
1528             exists.
1529
1530and the 'String' field is a single or tripled multiline quoted string. The
1531'String' can contain references to other macros. Macro that loop are not
1532currently detected and will cause the tool to lock up.
1533
1534Maps are declared anywhere in the map using the map directive:
1535
1536-------------------------------------------------------------
1537# Comments
1538[my-special-map] <1>
1539_host:  none, override, 'abc-xyz'
1540multiline: none, override, '''First line,
1541second line,
1542and finally the last line'''
1543-------------------------------------------------------------
1544<1> The map is set to `my-special-map`.
1545
1546Any macro defintions following a map declaration are placed in that map and the
1547default map is `global` when loading a file. Maps are selected in configuration
1548files by using the `%select` directive.
1549
1550-------------------------------------------------------------
1551%select my-special-map
1552-------------------------------------------------------------
1553
1554Selecting a map means all requests for a macro first check the selected map and
1555if present return that value else the `global` map is used. Any new macros or
1556changes update only the `global` map. This may change in future releases so
1557please make sure you use the `override` attribute.
1558
1559The macro files specificed on the command line are looked for in the
1560`_configdir` paths. See <<X1,+_configdir+>> variable for details. Included
1561files need to add the `%{_configdir}` macro to the start of the file.
1562
1563Macro map files can include other macro map files using the `%include`
1564directive. The macro map to build _binutils_, _gcc_, _newlib_, _gdb_ and
1565_RTEMS_ from version control heads is:
1566
1567-------------------------------------------------------------
1568# <1>
1569# Build all tool parts from version control head.
1570#
1571%include %{_configdir}/snapshots/binutils-head.mc
1572%include %{_configdir}/snapshots/gcc-head.mc
1573%include %{_configdir}/snapshots/newlib-head.mc
1574%include %{_configdir}/snapshots/gdb-head.mc
1575-------------------------------------------------------------
1576<1> The file is `config/snapshots/binutils-gcc-newlib-gdb-head.mc`.
1577
1578The macro map defaults to `global` at the start of each included file and the
1579map setting of the macro file including the other macro files does not change.
1580
1581Personal Macros
1582^^^^^^^^^^^^^^^
1583
1584When the tools start to run they will load personal macros. Personal macros are
1585in the standard format for macros in a file. There are two places personal
1586macros can be configured. The first is the environment variable
1587`RSB_MACROS`. If present the macros from the file the environment variable
1588points to are loaded. The second is a file called `.rsb_macros` in your home
1589directory. You need to have the environment variable `HOME` defined for this
1590work.
1591
1592Report Mailing
1593~~~~~~~~~~~~~~
1594
1595The build reports can be mailed to a specific email address to logging and
1596monitoring. Mailing requires a number of parameters to function. These are:
1597
1598. To mail address
1599. From mail address
1600. SMTP host
1601
1602.To Mail Address
1603
1604The +to+ mail address is taken from the macro `%{_mail_tools_to}` and the
1605default is _rtems-tooltestresults at rtems.org_. You can override the default
1606with a personal or user macro file or via the command line option _--mail-to_.
1607
1608.From Mail Address
1609
1610The +from+ mail address is taken from:
1611
1612. GIT configuration
1613. User `.mailrc` file
1614. Command line
1615
1616If you have configured an email and name in git it will be used used. If you do
1617not a check is made for a `.mailrc` file. The environment variable _MAILRC_ is
1618used if present else your home directory is check. If found the file is scanned
1619for the `from` setting:
1620
1621  set from="Foo Bar <foo@bar>"
1622
1623You can also support a from address on the command line with the _--mail-from_
1624option.
1625
1626.SMTP Host
1627
1628The SMTP host is taken from the macro `%{_mail_smtp_host}` and the default is
1629`localhost`. You can override the default with a personal or user macro file or
1630via the command line option _--smtp-host_.
1631
1632Build Set Files
1633~~~~~~~~~~~~~~~
1634
1635Build set files lets you list the packages in the build set you are defining
1636and have a file extension of +.bset+. Build sets can define macro variables,
1637inline include other files and reference other build set or package
1638configuration files.
1639
1640Defining macros is performed with the +%define+ macro:
1641
1642-------------------------------------------------------------
1643%define _target m32r-rtems4.11
1644-------------------------------------------------------------
1645
1646Inline including another file with the +%include+ macro continues processing
1647with the specified file returning to carry on from just after the include
1648point.
1649
1650-------------------------------------------------------------
1651%include rtems-4.11-base.bset
1652-------------------------------------------------------------
1653
1654This includes the RTEMS 4.11 base set of defines and checks. The configuration
1655paths as defined by +_configdir+ are scanned. The file extension is optional.
1656
1657You reference build set or package configuration files by placing the file name
1658on a single line.
1659
1660-------------------------------------------------------------
1661tools/rtems-binutils-2.22-1
1662-------------------------------------------------------------
1663
1664The +_configdir+ path is scanned for +tools/rtems-binutils-2.22-1.bset+ or
1665+tools/rtems-binutils-2.22-1.cfg+. Build set files take precedent over package
1666configuration files. If +tools/rtems-binutils-2.22-1+ is a build set a new
1667instance of the build set processor is created and if the file is a package
1668configuration the package is built with the package builder. This all happens
1669once the build set file has finished being scanned.
1670
1671Configuration Control
1672~~~~~~~~~~~~~~~~~~~~~
1673
1674The RTEMS Souce Builder is designed to fit within most verification and
1675validation processes. All of the RTEMS Source Builder is source code. The
1676Python code is source and comes with a commercial friendly license. All
1677configuration data is text and can be read or parsed with standard text based
1678tools.
1679
1680File naming provides configuration management. A specific version of a package
1681is captured in a specific set of configuration files. The top level
1682configuration file referenced in a _build set_ or passed to the +sb-builder+
1683command relates to a specific configuration of the package being built. For
1684example the RTEMS configuration file +rtems-gcc-4.7.2-newlib-2.0.0-1.cfg+
1685creates an RTEMS GCC and Newlib package where the GCC version is 4.7.2, the
1686Newlib version is 2.0.0, plus any RTEMS specific patches that related to this
1687version. The configuration defines the version numbers of the various parts
1688that make up this package:
1689
1690-------------------------------------------------------------
1691%define gcc_version    4.7.2
1692%define newlib_version 2.0.0
1693%define mpfr_version   3.0.1
1694%define mpc_version    0.8.2
1695%define gmp_version    5.0.5
1696-------------------------------------------------------------
1697
1698The package build options, if there are any are also defined:
1699
1700-------------------------------------------------------------
1701%define with_threads 1
1702%define with_plugin  0
1703%define with_iconv   1
1704-------------------------------------------------------------
1705
1706The generic configuration may provide defaults in case options are not
1707specified. The patches this specific version of the package requires can be
1708included:
1709
1710-------------------------------------------------------------
1711Patch0: gcc-4.7.2-rtems4.11-20121026.diff
1712-------------------------------------------------------------
1713
1714Finally including the GCC 4.7 configuration script:
1715
1716-------------------------------------------------------------
1717%include %{_configdir}/gcc-4.7-1.cfg
1718-------------------------------------------------------------
1719
1720The +gcc-4.7-1.cfg+ file is a generic script to build a GCC 4.7 compiler with
1721Newlib. It is not specific to RTEMS. A bare no operating system tool set can be
1722built with this file.
1723
1724The +-1+ part of the file names is a revision. The GCC 4.7 script maybe revised
1725to fix a problem and if this fix effects an existing script the file is copied
1726and given a +-2+ revision number. Any dependent scripts referencing the earlier
1727revision number will not be effected by the change. This locks down a specific
1728configuration over time.
1729
1730Personal Configurations
1731~~~~~~~~~~~~~~~~~~~~~~~
1732
1733The RSB supports personal configurations. You can view the RTEMS support in the
1734+rtems+ directory as a private configuration tree that resides within the RSB
1735source. There is also the +bare+ set of configurations. You can create your own
1736configurations away from the RSB source tree yet use all that the RSB provides.
1737
1738To create a private configuration change to a suitable directory:
1739
1740-------------------------------------------------------------
1741$ cd ~/work
1742$ mkdir test
1743$ cd test
1744$ mkdir config
1745-------------------------------------------------------------
1746
1747and create a +config+ directory. Here you can add a new configuration or build
1748set file. The section 'Adding New Configurations' details how to add a new
1749confguration.
1750
1751New Configurations
1752~~~~~~~~~~~~~~~~~~
1753
1754This section describes how to add a new configuration to the RSB. We will add a
1755configuration to build the Device Tree Compiler. The Device Tree Compiler or
1756DTC is part of the Flattened Device Tree project and compiles Device Tree
1757Source (DTS) files into Device Tree Blobs (DTB). DTB files can be loaded by
1758operating systems and used to locate the various resources such as base
1759addresses of devices or interrupt numbers allocated to devices. The Device Tree
1760Compiler source code can be downloaded from http://www.jdl.com/software. The
1761DTC is supported in the RSB and you can find the configuration files under the
1762+bare/config+ tree. I suggest you have a brief look over these files.
1763
1764Layering by Including
1765^^^^^^^^^^^^^^^^^^^^^
1766
1767Configurations can be layered using the +%include+ directive. The user invokes
1768the outer layers which include inner layers until all the required
1769configuration is present and the package can be built. The outer layers can
1770provide high level details such as the version and the release and the inner
1771layers provide generic configuration details that do not change from one
1772release to another. Macro variables are used to provide the specific
1773configuration details.
1774
1775Configuration File Numbering
1776^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1777
1778Configuration files have a number at the end. This is a release number for that
1779configuration and it gives us the ability to track a specific configuration for
1780a specific version. For example lets say the developers of the DTC package
1781change the build system from a single makefile to autoconf and automake between
1782version 1.3.0 and version 1.4.0. The configuration file used to build the
1783package would change have to change. If we did not number the configuration
1784files the ability to build 1.1.0, 1.2.0 or 1.3.0 would be lost if we update a
1785common configuration file to build an autoconf and automake version. For
1786version 1.2.0 the same build script can be used so we can share the same
1787configuration file between version 1.1.0 and version 1.2.0. An update to any
1788previous release lets us still build the package.
1789
1790Common Configuration Scripts
1791^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1792
1793Common configuration scripts that are independent of version, platform and
1794architecture are useful to everyone. These live in the Source Builder's
1795configuration directory. Currently there are scripts to build binutils, expat,
1796DTC, GCC, GDB and libusb. These files contain the recipes to build these
1797package without the specific details of the versions or patches being
1798built. They expect to be wrapped by a configuration file that ties the package
1799to a specific version and optionally specific patches.
1800
1801DTC Example
1802^^^^^^^^^^^
1803
1804We will be building the DTC for your host rather than a package for RTEMS. We
1805will create a file called +source-builder/config/dtc-1-1.cfg+. This is a common
1806script that can be used to build a specific version using a general recipe. The
1807file name is 'dtc-1-1.cfg' where the 'cfg' extension indicates this is a
1808configuration file. The first *1* says this is for the major release 1 of the
1809package and the last *1* is the build configuration version.
1810
1811The file starts with some comments that detail the configuration. If there is
1812anything unusual about the configuration it is a good idea to add something in
1813the comments here. The comments are followed by a check for the release. In
1814this case if a release is not provided a default of 1 is used.
1815
1816-------------------------------------------------------------
1817#
1818# DTC 1.x.x Version 1.
1819#
1820# This configuration file configure's, make's and install's DTC.
1821#
1822
1823%if %{release} == %{nil}
1824%define release 1
1825%endif
1826-------------------------------------------------------------
1827
1828The next section defines some information about the package. It does not effect
1829the build and is used to annotate the reports. It is recommended this
1830information is kept updated and accurate.
1831
1832-------------------------------------------------------------
1833Name:      dtc-%{dtc_version}-%{_host}-%{release}
1834Summary:   Device Tree Compiler v%{dtc_version} for target %{_target} on host %{_host}
1835Version:   %{dtc_version}
1836Release:   %{release}
1837URL:       http://www.jdl.com/software/
1838BuildRoot: %{_tmppath}/%{name}-root-%(%{__id_u} -n)
1839-------------------------------------------------------------
1840
1841The next section defines the source and any patches. In this case there is a
1842single source package and it can be downloaded using the HTTP protocol. The RSB
1843knows this is GZip'ped tar file. If more than one package package is needed add
1844them increasing the index. The +gcc-4.8-1.cfg+ configuration contains examples
1845of more than one source package as well as conditionally including source
1846packages based on the outer configuration options.
1847
1848-------------------------------------------------------------
1849#
1850# Source
1851#
1852%source set dtc http://www.jdl.com/software/dtc-v%{dtc_version}.tgz
1853-------------------------------------------------------------
1854
1855The remainder of the script is broken in to the various phases of a build. They
1856are:
1857
1858. Preperation
1859. Bulding
1860. Installing, and
1861. Cleaning
1862
1863Preparation is the unpacking of the source, applying any patches as well as any
1864package specific set ups. This part of the script is a standard Unix shell
1865script. Be careful with the use of '%' and '$'. The RSB uses '%' while the
1866shell scripts use '$'.
1867
1868A standard pattern you will observe is the saving of the build's top
1869directory. This is used instead of changing into a subdirectory and then
1870changing to the parent when finished. Some hosts will change in a subdirectory
1871that is a link however changing to the parent does not change back to the
1872parent of the link rather it changes to the parent of the target of the link
1873and that is something the RSB nor you can track easily. The RSB configuration
1874script's are a collection of various subtle issues so please ask if you are
1875unsure why something is being done a particular way.
1876
1877The preparation phase will often include source and patch setup commands. Outer
1878layers can set the source package and add patches as needed while being able to
1879use a common recipe for the build. Users can override the standard build and
1880supply a custom patch for testing using the user macro command line interface.
1881
1882-------------------------------------------------------------
1883#
1884# Prepare the source code.
1885#
1886%prep
1887  build_top=$(pwd)
1888
1889  %source setup dtc -q -n dtc-v%{dtc_version}
1890  %patch setup dtc -p1
1891
1892  cd ${build_top}
1893-------------------------------------------------------------
1894
1895The configuration file 'gcc-common-1.cfg' is a complex example of source
1896preparation. It contains a number of source packages and patches and it
1897combines these into a single source tree for building. It uses links to map
1898source into the GCC source tree so GCC can be built using the _single source
1899tree_ method. It also shows how to fetch source code from version
1900control. Newlib is taken directly from its CVS repository.
1901
1902Next is the building phase and for the DTC example this is simply a matter of
1903running +make+. Note the use of the RSB macros for commands. In the case of
1904'%{\__make}' it maps to the correct make for your host. In the case of BSD
1905systems we need to use the GNU make and not the GNU make.
1906
1907If your package requires a configuration stage you need to run this before the
1908make stage. Again the GCC common configuration file provides a detailed example.
1909
1910-------------------------------------------------------------
1911%build
1912  build_top=$(pwd)
1913
1914  cd dtc-v%{dtc_version}
1915
1916  %{build_build_flags}
1917
1918  %{__make} PREFIX=%{_prefix}
1919
1920  cd ${build_top}
1921-------------------------------------------------------------
1922
1923You can invoke make with the macro '%{?_smp_flags}' as a command line
1924argument. This macro is controlled by the '--jobs' command line option and the
1925host CPU detection support in the RSB. If you are on a multicore host you can
1926increase the build speed using this macro. It also lets you disabled building on
1927multicores to aid debugging when testing.
1928
1929Next is the install phase. This phase is a little more complex because you may
1930be building a tar file and the end result of the build is never actually
1931installed into the prefix on the build host and you may not even have
1932permissions to perform a real install. Most packages install to the +prefix+
1933and the prefix is typically supplied via the command to the RSB or the
1934package's default is used. The default can vary depending on the host's
1935operating system. To install to a path that is not the prefix the +DESTDIR+
1936make variable is used. Most packages should honour the +DISTDIR+ make variables
1937and you can typically specify it on the command line to make when invoking the
1938install target. This results in the package being installed to a location that
1939is not the prefix but one you can control. The RSB provides a shell variable
1940called +SB_BUILD_ROOT+ you can use. In a build set where you are building a
1941number of packages you can collect all the built packages in a single tree that
1942is captured in the tar file.
1943
1944Also note the use of the macro +%{\__rmdir}+. The use of these macros allow the
1945RSB to vary specific commands based on the host. This can help on hosts like
1946Windows where bugs can effect the standard commands such as 'rm'. There are
1947many many macros to help you. You can find these listed in the +defaults.mc+
1948file and in the trace output. If you are new to creating and editing
1949configurations learning these can take a little time.
1950
1951-------------------------------------------------------------
1952%install
1953  build_top=$(pwd)
1954
1955  %{__rmdir} -rf $SB_BUILD_ROOT
1956
1957  cd dtc-v%{dtc_version}
1958  %{__make} DESTDIR=$SB_BUILD_ROOT PREFIX=%{_prefix} install
1959
1960  cd ${build_top}
1961-------------------------------------------------------------
1962
1963Finally there is an optional clean section. The RSB will run this section if
1964+--no-clean+ has not been provided on the command line. The RSB does clean up
1965for you.
1966
1967Once we have the configuration files we can execute the build using the
1968`sb-builder` command. The command will perform the build and create a tar file
1969in the +tar+ directory.
1970
1971-------------------------------------------------------------
1972$  ../source-builder/sb-builder --prefix=/usr/local \
1973     --log=log_dtc devel/dtc-1.2.0
1974RTEMS Source Builder, Package Builder v0.2.0
1975config: devel/dtc-1.2.0
1976package: dtc-1.2.0-x86_64-freebsd9.1-1
1977download: http://www.jdl.com/software/dtc-v1.2.0.tgz -> sources/dtc-v1.2.0.tgz
1978building: dtc-1.2.0-x86_64-freebsd9.1-1
1979$ ls tar
1980dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2
1981-------------------------------------------------------------
1982
1983If you want to have the package installed automatically you need to create a
1984build set. A build set can build one or more packages from their configurations
1985at once to create a single package. For example the GNU tools is typically seen
1986as binutils, GCC and GDB and a build set will build each of these packages and
1987create a single build set tar file or install the tools on the host into the
1988prefix path.
1989
1990The DTC build set file is called +dtc.bset+ and contains:
1991
1992-------------------------------------------------------------
1993#
1994# Build the DTC.
1995#
1996
1997%define release 1
1998
1999devel/dtc-1.2.0.cfg
2000-------------------------------------------------------------
2001
2002To build this you can use something similar to:
2003
2004-------------------------------------------------------------
2005$ ../source-builder/sb-set-builder --prefix=/usr/local --log=log_dtc \
2006   --trace --bset-tar-file --no-install dtc
2007RTEMS Source Builder - Set Builder, v0.2.0
2008Build Set: dtc
2009config: devel/dtc-1.2.0.cfg
2010package: dtc-1.2.0-x86_64-freebsd9.1-1
2011building: dtc-1.2.0-x86_64-freebsd9.1-1
2012tarball: tar/x86_64-freebsd9.1-dtc-set.tar.bz2
2013cleaning: dtc-1.2.0-x86_64-freebsd9.1-1
2014Build Set: Time 0:00:02.865758
2015$ ls tar
2016dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2   x86_64-freebsd9.1-dtc-set.tar.bz2
2017-------------------------------------------------------------
2018
2019The build is for a FreeBSD host and the prefix is for user installed
2020packages. In this example I cannot let the source builder perform the install
2021because I never run the RSB with root priviledges so a build set or bset tar
2022file is created. This can then be installed using root privildges.
2023
2024The command also supplies the --trace option. The output in the log file will
2025contian all the macros.
2026
2027Debugging
2028^^^^^^^^^
2029
2030New configuration files require debugging. There are two types of
2031debugging. The first is debugging RSB script bugs. The +--dry-run+ option is
2032used here. Suppling this option will result in most of the RSB processing to be
2033performed and suitable output placed in the log file. This with the +--trace+
2034option should help you resolve any issues.
2035
2036The second type of bug to fix are related to the execution of one of
2037phases. These are usually a mix of shell script bugs or package set up or
2038configuration bugs. Here you can use any normal shell script type debug
2039technique such as +set -x+ to output the commands or +echo+
2040statements. Debugging package related issues may require you start a build with
2041teh RSB and supply +--no-clean+ option and then locate the build directories
2042and change directory into them and manually run commands until to figure what
2043the package requires.
2044
2045Snapshot Testing
2046~~~~~~~~~~~~~~~~
2047
2048_This section needs to be updated once I sort out snapshot testing._
2049
2050Testing of release canidates and snapshots is important to those helping
2051maintain tool sets. The RTEMS Source Builder helps by providing a simple and
2052flexible way to use existing build sets and configuration without needing to
2053change them or creating new temporary build sets and configurations.
2054
2055The process uses snapshot macro files loaded via the command line option
2056`--macros`. These files provide macros that override the standard build set and
2057configuration file macros.
2058
2059Lets consider testing a GCC 4.7 snapshot for RTEMS 4.11. Lets assume the
2060current RTEMS 4.11 tools reference GCC 4.7.3 with a patch as the stable tool
2061set. We want to use a recent snapshot with no patches. In the
2062`rtems/config/snapshots` directoy create a file called `gcc-4.7-snapshot.mc`
2063containing:
2064
2065-------------------------------------------------------------
2066[gcc-4.7-snapshot]
2067GCC_Version: none, override, '4.7-20130413'
2068Source:     none, override, 'http://mirrors.kernel.org/sources.redhat.com/gcc/
2069snapshots/%{gcc_version}/gcc-%{gcc_version}.tar.bz2'
2070Patch0:      none, udefine,  ''
2071-------------------------------------------------------------
2072
2073In the standard configuration file `source-builder/config/gcc-4.7-1.cfg` the
2074map is selected with:
2075
2076-------------------------------------------------------------
2077#
2078# Select the GCC 4.7 Snapshot Macro Map
2079#
2080%select gcc-4.7-snapshot
2081-------------------------------------------------------------
2082
2083On the command line add `--macros=snapshots/gcc-4.7-snapshot.mc` and this
2084snapshot will be built. With careful use of the `--prefix` option you can
2085locate the tools in a specific directory and test them without needing to
2086effect your production environment.
2087
2088Scripting
2089~~~~~~~~~
2090
2091Configuration files specify how to build a package. Configuration files are
2092scripts and have a +.cfg+ file extension. The script format is based loosely on
2093the RPM spec file format however the use and purpose in this tool does not
2094compare with the functionality and therefore the important features of the spec
2095format RPM needs and uses.
2096
2097The script language is implemented in terms of macros. The built-in list is:
2098
2099[horizontal]
2100+%{}+:: Macro expansion with conditional logic.
2101+%()+:: Shell expansion.
2102+%prep+:: The source preparation shell commands.
2103+%build+:: The build shell commands.
2104+%install+:: The package install shell commands.
2105+%clean+:: The package clean shell commands.
2106+%include+:: Inline include another configuration file.
2107+%name+:: The name of the package.
2108+%summary+:: A brief package description. Useful when reporting about a build.
2109+%release+:: The package release. A number that is the release as built by this tool.
2110+%version+:: The package's version string.
2111+%buildarch+:: The build architecture.
2112+%source+:: Define a source code package. This macro has a number appended.
2113+%patch+:: Define a patch. This macro has a is number appended.
2114+%hash+:: Define a checksum for a source or patch file.
2115+%echo+:: Print the following string as a message.
2116+%warning+:: Print the following string as a warning and continue.
2117+%error+:: Print the following string as an error and exit.
2118+%select+:: Select the macro map. If there is no map nothing is reported.
2119+%define+:: Define a macro. Macros cannot be redefined, you must first undefine it.
2120+%undefine+:: Undefine a macro.
2121+%if+:: Start a conditional logic block that ends with a +%endif+.
2122+%ifn+:: Inverted start of a conditional logic block.
2123+%ifarch+:: Test the architecture against the following string.
2124+%ifnarch+:: Inverted test of the architecture
2125+%ifos+:: Test the host operating system.
2126+%else+:: Start the _else_ conditional logic block.
2127+%endfi+:: End the conditional logic block.
2128+%bconf_with+:: Test the build condition _with_ setting. This is the +--with-*+
2129command line option.
2130+%bconf_without+:: Test the build condition _without_ setting. This is the
2131+--without-*+ command line option.
2132
2133Expanding
2134^^^^^^^^^
2135
2136A macro can be `%{string}` or the equivalent of `%string`. The following macro
2137expansions supported are:
2138
2139`%{string}`;;
2140Expand the 'string' replacing the entire macro text with the text in the table
2141for the entry 'string . For example if 'var' is 'foo' then `${var}` would
2142become `foo`.
2143
2144`%{expand: string}`;;
2145Expand the 'string' and then use it as a ``string'' to the macro expanding the
2146macro. For example if _foo_ is set to 'bar' and 'bar' is set to 'foobar' then
2147`%{expand:foo}` would result in `foobar`. Shell expansion can also be used.
2148
2149`%{with string}`;;
2150Expand the macro to '1' if the macro `with_`'string' is defined else expand to
2151_0_. Macros with the name `with_`'string' can be define with command line
2152arguments to the RTEMS Source Builder commands.
2153
2154`%{defined string}`;;
2155Expand the macro to '1' if a macro of name 'string' is defined else expand to '0'.
2156
2157`%{?string: expression}`;;
2158Expand the macro to 'expression' if a macro of name 'string' is defined else expand to `%{nil}`.
2159
2160`%{!?string: expression}`;;
2161Expand the macro to 'expression' if a macro of name 'string' is not defined. If
2162the macro is define expand to `%{nil}`.
2163
2164`%(expression)`;;
2165Expand the macro to the result of running the 'expression' in a host shell. It
2166is assumed this is a Unix type shell. For example `%(whoami)` will return your
2167user name and `%(date)` will return the current date string.
2168
2169%prep
2170^^^^^
2171
2172The +%prep+ macro starts a block that continues until the next block macro. The
2173_prep_ or preparation block defines the setup of the package's source and is a
2174mix of RTEMS Source Builder macros and shell scripting. The sequence is
2175typically +%source+ macros for source, +%patch+ macros to patch the source
2176mixed with some shell commands to correct any source issues.
2177
2178-------------------------------------------------------------
2179              <1>                <2>      <3>
2180%source setup gcc -q -c -T -n %{name}-%{version}
2181-------------------------------------------------------------
2182
2183<1> The source group to set up.
2184<2> The source's name.
2185<3> The version of the source.
2186
2187The source set up are declared with the source +set+ and +add+ commands. For
2188example:
2189
2190-------------------------------------------------------------
2191%source set gdb http://ftp.gnu.org/gnu/gdb/gdb-%{gdb_version}.tar.bz2
2192-------------------------------------------------------------
2193
2194This URL is the primary location of the GNU GDB source code and the RTEMS
2195Source Builder can download the file from this location and by inspecting the
2196file extension use +bzip2+ decompression with +tar+. When the +%prep+ section
2197is processed a check of the local +source+ directory is made to see if the file
2198has already been downloaded. If not found in the source cache directory the
2199package is downloaded from the URL. You can append other base URLs via the
2200command line option +--url+. This option accepts a comma delimited list of
2201sites to try.
2202
2203You could optionally have a few source files that make up the package. For
2204example GNU's GCC was a few tar files for a while and it is now a single tar
2205file. Support for multiple source files can be conditionally implemented with
2206the following scripting:
2207
2208-------------------------------------------------------------
2209%source set gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-code-%{gcc_version}.tar.bz2
2210%source add gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-g++-%{gcc_version}.tar.bz2
2211%source setup gcc -q -T -D -n gcc-%{gcc_version}
2212-------------------------------------------------------------
2213
2214Separate modules use separate source groups. The GNU GCC compiler for RTEMS
2215uses Newlib, MPFR, MPC, and GMP source packages. You define the source with:
2216
2217-------------------------------------------------------------
2218%source set gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
2219%source set newlib ftp://sourceware.org/pub/newlib/newlib-%{newlib_version}.tar.gz
2220%source set mpfr http://www.mpfr.org/mpfr-%{mpfr_version}/mpfr-%{mpfr_version}.tar.bz2
2221%source set mpc http://www.multiprecision.org/mpc/download/mpc-%{mpc_version}.tar.gz
2222%source set gmp ftp://ftp.gnu.org/gnu/gmp/gmp-%{gmp_version}.tar.bz2
2223-------------------------------------------------------------
2224
2225and set up with:
2226
2227-------------------------------------------------------------
2228%source setup gcc -q -n gcc-%{gcc_version}
2229%source setup newlib -q -D -n newlib-%{newlib_version}
2230%source setup mpfr -q -D -n mpfr-%{mpfr_version}
2231%source setup mpc -q -D -n mpc-%{mpc_version}
2232%source setup gmp -q -D -n gmp-%{gmp_version}
2233-------------------------------------------------------------
2234
2235Patching also occurs during the preparation stage. Patches are handled in a
2236similar way to the source packages except you only +add+ patches. Patches are
2237applied using the +setup+ command. The +setup+ command takes the default patch
2238option. You can provide options with each patch by adding them as arguments
2239before the patch URL. Patches with no options uses the +setup+ default.
2240
2241-------------------------------------------------------------
2242%patch add gdb %{rtems_gdb_patches}/gdb-sim-arange-inline.diff
2243%patch add gdb -p0 <1> %{rtems_gdb_patches}/gdb-sim-cgen-inline.diff
2244-------------------------------------------------------------
2245<1> This patch has a custom option.
2246
2247To apply these patches:
2248
2249-------------------------------------------------------------
2250%patch setup gdb -p1 <1>
2251-------------------------------------------------------------
2252<1> The default options.
2253
2254%build
2255^^^^^^
2256
2257The +%build+ macro starts a block that continues until the next block
2258macro. The build block is a series of shell commands that execute to build the
2259package. It assumes all source code has been unpacked, patch and adjusted so
2260the build will succeed.
2261
2262The following is an example take from the GutHub STLink project:
2263
2264NOTE: STLink is a JTAG debugging device for the ST ARM family of processors.
2265
2266-------------------------------------------------------------
2267%build
2268  export PATH="%{_bindir}:${PATH}" <1>
2269
2270  cd texane-stlink-%{stlink_version} <2>
2271
2272  ./autogen.sh <3>
2273
2274%if "%{_build}" != "%{_host}"
2275  CFLAGS_FOR_BUILD="-g -O2 -Wall" \ <4>
2276%endif
2277  CPPFLAGS="-I $SB_TMPPREFIX/include/libusb-1.0" \ <5>
2278  CFLAGS="$SB_OPT_FLAGS" \
2279  LDFLAGS="-L $SB_TMPPREFIX/lib" \
2280  ./configure \ <6>
2281    --build=%{_build} --host=%{_host} \
2282    --verbose \
2283    --prefix=%{_prefix} --bindir=%{_bindir} \
2284    --exec-prefix=%{_exec_prefix} \
2285    --includedir=%{_includedir} --libdir=%{_libdir} \
2286    --mandir=%{_mandir} --infodir=%{_infodir}
2287
2288  %{__make} %{?_smp_mflags} all <7>
2289
2290  cd ..
2291-------------------------------------------------------------
2292
2293<1> Setup the PATH environment variable. This is not always needed.
2294<2> This package builds in the source tree so enter it.
2295<3> The package is actually checked directly out from the github project and so
2296    it needs its autoconf and automake files generated.
2297<4> Flags for a cross-compiled build.
2298<5> Various settings passed to configure to customise the build. In this
2299    example an include path is being set to the install point of _libusb_. This
2300    package requires _libusb_ is built before it.
2301<6> The +configure+ command. The RTEMS Source Builder provides all the needed
2302    paths as macro variables. You just need to provide them to +configure+.
2303<7> Running make. Do not use +make+ directly, use the RTEMS Source Builder's
2304    defined value. This value is specific to the host. A large number of
2305    packages need GNU make and on BSD systems this is +gmake+. You can
2306    optionally add the SMP flags if the packages build system can handle
2307    parallel building with multiple jobs. The +_smp_mflags+ value is
2308    automatically setup for SMP hosts to match the number of cores the host has.
2309
2310%install
2311^^^^^^^^
2312
2313The +%install+ macro starts a block that continues until the next block
2314macro. The install block is a series of shell commands that execute to install
2315the package. You can assume the package has build correctly when this block
2316starts executing.
2317
2318Never install the package to the actual _prefix_ the package was built
2319with. Always install to the RTEMS Source Builder's temporary path defined in
2320the macro variable +\__tmpdir+. The RTEMS Source Builder sets up a shell
2321environment variable called +SB_BUILD_ROOT+ as the standard install point. Most
2322packages support adding +DESTDIR=+ to the _make install_ command.
2323
2324Looking at the same example as in <<_build, %build>>:
2325
2326-------------------------------------------------------------
2327%install
2328  export PATH="%{_bindir}:${PATH}" <1>
2329  rm -rf $SB_BUILD_ROOT <2>
2330
2331  cd texane-stlink-%{stlink_version} <3>
2332  %{__make} DESTDIR=$SB_BUILD_ROOT install <4>
2333
2334  cd ..
2335-------------------------------------------------------------
2336
2337<1> Setup the PATH environment variable. This is not always needed.
2338<2> Clean any installed files. This make sure the install is just what
2339    the package installs and not any left over files from a broken build or
2340    install.
2341<3> Enter the build directory. In this example it just happens to be the source
2342    directory.
2343<4> Run +make install+ to install the package overriding the +DESTDIR+ make
2344    variable.
2345
2346%clean
2347^^^^^^
2348
2349The +%clean+ macro starts a block that continues until the next block
2350macro. The clean block is a series of shell commands that execute to clean up
2351after a package has been built and install. This macro is currenly not been
2352used because the RTEMS Source Builder automatically cleans up.
2353
2354%include
2355^^^^^^^^
2356
2357The +%include+ macro inline includes the specific file. The +\__confdir+
2358path is searched. Any relative path component of the include file is appended
2359to each part of the +\__configdir+. Adding an extension is optional as files
2360with +.bset+ and +.cfg+ are automatically searched for.
2361
2362Inline including means the file is processed as part of the configuration at
2363the point it is included. Parsing continues from the next line in the
2364configuration file that contains the +%include+ macro.
2365
2366Including files allow a kind of configuration file reuse. The outer
2367configuration files provide specific information such as package version
2368numbers and patches and then include a generic configuration script which
2369builds the package.
2370
2371-------------------------------------------------------------
2372%include %{_configdir}/gcc-4.7-1.cfg
2373-------------------------------------------------------------
2374
2375%name
2376^^^^^
2377
2378The name of the package being built. The name typically contains the components
2379of the package and their version number plus a revision number. For the GCC
2380with Newlib configuration the name is typically:
2381
2382-------------------------------------------------------------
2383Name: %{_target}-gcc-%{gcc_version}-newlib-%{newlib_version}-%{release}
2384-------------------------------------------------------------
2385
2386%summary
2387^^^^^^^^
2388
2389The +%summary+ is a brief description of the package. It is useful when
2390reporting. This information is not capture in the package anywhere. For the GCC
2391with Newlib configuration the summary is typically:
2392
2393-------------------------------------------------------------
2394Summary: GCC v%{gcc_version} and Newlib v%{newlib_version} for target %{_target} on host %{_host}
2395-------------------------------------------------------------
2396
2397%release
2398^^^^^^^^
2399
2400The +%release+ is packaging number that allows revisions of a package to happen
2401where none package versions change. This value typically increases when the
2402configuration building the package changes.
2403
2404-------------------------------------------------------------
2405%define release 1
2406-------------------------------------------------------------
2407
2408%version
2409^^^^^^^^
2410
2411The +%version% macro sets the version the package. If the package is a single
2412component it tracks that component's version number. For example in the
2413_libusb_ configuration the +%version+ is the same as +%libusb_version+, however
2414in a GCC with Newlib configuration there is no single version number. In this
2415case the GCC version is used.
2416
2417-------------------------------------------------------------
2418Version: %{gcc_version}
2419-------------------------------------------------------------
2420
2421%buildarch
2422^^^^^^^^^^
2423
2424The +%buildarch+ macro is set to the architecture the package contains. This is
2425currently not used in the RTEMS Source Builder and may go away. This macro is
2426more important in a real packaging system where the package could end up on the
2427wrong architecture.
2428
2429%source
2430^^^^^^^
2431
2432The +%source+ macro has 3 commands that controls what it does. You can +set+
2433the source files, +add+ source files to a source group, and +setup+ the source
2434file group getting it ready to be used.
2435
2436Source files are source code files in tar or zip files that are unpacked,
2437copied or symbolically linked into the package's build tree. Building a package
2438requires one or more dependent packages. These are typically the packages
2439source code plus dependent libraries or modules. You can create any number of
2440these source groups and set each of them up with a separe source group for each
2441needed library or module. Each source group normally has a single tar, zip or
2442repository and the +set+ defines this. Some projects split the source code into
2443separate tar or zip files and you install them by using the +add+ command.
2444
2445The first instance of a +set+ command creates the source group and sets the
2446source files to be set up. Subsequence +set+ commands for the same source group
2447are ignored. this lets you define the standard source files and override them
2448for specific releases or snapshots.. To set a source file group:
2449
2450-------------------------------------------------------------
2451%source set gcc <1> ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
2452-------------------------------------------------------------
2453<1> The source group is +gcc+.
2454
2455To add another source package to be installed into the same source tree you use
2456the +add+ command:
2457
2458-------------------------------------------------------------
2459%source add gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/g++-%{gcc_version}.tar.bz2
2460-------------------------------------------------------------
2461
2462The source +setup+ command can only be issued in the +%prep:+ section. The
2463setup is:
2464
2465-------------------------------------------------------------
2466%source gcc setup -q -T -D -n %{name}-%{version}
2467-------------------------------------------------------------
2468
2469Accepted options are:
2470
2471[horizontal]
2472*Switch*:: *Description*
2473+-n+:: The -n option is used to set the name of the software's build
2474directory. This is necessary only when the source archive unpacks into a
2475directory named other than +<name>-<version>+.
2476+-c+:: The -c option is used to direct %setup to create the top-level build
2477directory before unpacking the sources.
2478+-D+:: The -D option is used to direct %setup to not delete the build directory
2479prior to unpacking the sources. This option is used when more than one source
2480archive is to be unpacked into the build directory, normally with the +-b+ or
2481+-a+ options.
2482+-T+:: The -T option is used to direct %setup to not perform the default
2483unpacking of the source archive specified by the first Source: macro. It is used
2484with the +-a+ or +-b+ options.
2485+-b <n>+:: The -b option is used to direct %setup to unpack the source archive
2486specified on the nth Source: macro line before changing directory into the build
2487directory.
2488
2489%patch
2490^^^^^^
2491
2492The +%patch+ macro has the same 3 command as the +%source+ command however the
2493+set+ commands is not really that useful with the with command. You add patches
2494with the +add+ command and +setup+ applies the patches. Patch options can be
2495added to each patch by placing them before the patch URL. If no patch option is
2496provided the default options passed to the +setup+ command are used. An option
2497starts with a '-'. The +setup+ command must reside inside the +%prep+ section.
2498
2499Patches are grouped in a similar way to the +%source+ macro so you can control
2500applying a group of patches to a specific source tree.
2501
2502The +__patchdir+ path is search.
2503
2504To add a patch:
2505
2506-------------------------------------------------------------
2507%patch add gcc <1>  gcc-4.7.2-rtems4.11-20121026.diff
2508%patch add gcc -p0 <2>  gcc-4.7.2-rtems4.11-20121101.diff
2509-------------------------------------------------------------
2510<1> The patch group is +gcc+.
2511<2> Option for this specific patch.
2512
2513Placing +%patch setup+ in the +%prep+ section will apply the groups patches.
2514
2515-------------------------------------------------------------
2516%patch setup gcc <1>  -p1 <2>
2517-------------------------------------------------------------
2518<1> The patch group.
2519<2> The default option used to apply the patch.
2520
2521%hash
2522^^^^^
2523
2524The +%hash+ macro requires 3 arguments and defines a checksum for a specific
2525file. The checksum is not applied until the file is checked before downloading
2526and once downloaded. A patch or source file that does not has a hash defined
2527generates a warning.
2528
2529A file to be checksum must be unqiue in the any of the source and patch
2530directories. The basename of the file is used as the key for the hash.
2531
2532The hash algorthim can be 'md5', 'sha1', 'sha224', 'sha256', 'sha384', and
2533'sha512' and we typically use 'md5'.
2534
2535To add a hash:
2536
2537-------------------------------------------------------------
2538%hash md5 <1> net-snmp-%{net_snmp_version}.tar.gz <2> 7db683faba037249837b226f64d566d4 <3>
2539-------------------------------------------------------------
2540<1> The type of checksum.
2541<2> The file to checksum. It can contain macros that are expanded for you.
2542<3> The MD5 hash for the Net-SNMP file +net-snmp-5.7.2.1.tar.gz+.
2543
2544Do not include a path with the file name. Only the basename is required. Files
2545can be searched for from a number of places and having a path conponent would
2546create confusion. This does mean files with hashes must be unique.
2547
2548Downloading of repositories such as git and cvs cannot be checksumed. It is
2549assumed those protocols and tools manage the state of the files.
2550
2551%echo
2552^^^^^
2553
2554The +%echo+ macro outputs the following string to stdout. This can also be used
2555as `%{echo: message}`.
2556
2557%warning
2558^^^^^^^^
2559
2560The +%warning+ macro outputs the following string as a warning. This can also
2561be used as `%{warning: message}`.
2562
2563%error
2564^^^^^^
2565
2566The +%error+ macro outputs the follow string as an error and exits the RTEMS
2567Source Builder. This can also be used as `%{error: message}`.
2568
2569%select
2570^^^^^^^
2571
2572The +%select+ macro selects the map specified. If there is no map no error or
2573warning is generated. Macro maps provide a simple way for a user to override
2574the settings is a configuration file without having to edit it. The changes are
2575recorded in the build report so can be traced.
2576
2577Configuration use different maps so macro overrides can target a specific
2578package.
2579
2580The default map is `global'.
2581
2582-------------------------------------------------------------
2583%select gcc-4.8-snapshot <1>
2584%define one_plus_one 2 <2>
2585-------------------------------------------------------------
2586
2587<1> The map switches to `gcc-4.8-snapshot`. Any overrides in this map will be
2588    used.
2589<2> Defining macros only updates the `global` map and not the selected map.
2590
2591%define
2592^^^^^^^
2593
2594The +%define+ macro defines a new macro or updates an existing one. If no value
2595is given it is assumed to be 1.
2596
2597-------------------------------------------------------------
2598%define foo bar
2599%define one_plus_one 2
2600%define one <1>
2601-------------------------------------------------------------
2602
2603<1> The macro _one_ is set to 1.
2604
2605%undefine
2606^^^^^^^^^
2607
2608The +%undefine+ macro removes a macro if it exists. Any further references to
2609it will result in an undefine macro error.
2610
2611%if
2612^^^
2613
2614The +%if+ macro starts a conditional logic block that can optionally have a
2615_else_ section. A test follows this macro and can have the following operators:
2616
2617[horizontal]
2618*Operator*:: *Description*
2619+%{}+:: Check the macro is set or _true_, ie non-zero.
2620+
2621-------------------------------------------------------------
2622%if ${foo}
2623 %warning The test passes, must not be empty or is non-zero
2624%else
2625 %error The test fails, must be empty or zero
2626%endif
2627-------------------------------------------------------------
2628+!+:: The _not_ operator inverts the test of the macro.
2629+
2630-------------------------------------------------------------
2631%if ! ${foo}
2632 %warning The test passes, must be empty or zero
2633%else
2634 %error The test fails, must not be empty or is non-zero
2635%endif
2636-------------------------------------------------------------
2637+==+:: The left hand size must equal the right hand side. For example:
2638+
2639-------------------------------------------------------------
2640%define one 1
2641%if ${one} == 1
2642 %warning The test passes
2643%else
2644 %error The test fails
2645%endif
2646-------------------------------------------------------------
2647+
2648You can also check to see if a macro is empty:
2649+
2650-------------------------------------------------------------
2651%if ${nothing} == %{nil}
2652 %warning The test passes
2653%else
2654 %error The test fails
2655%endif
2656-------------------------------------------------------------
2657+!=+:: The left hand size does not equal the right hand side. For example:
2658+
2659-------------------------------------------------------------
2660%define one 1
2661%if ${one} != 2
2662 %warning The test passes
2663%else
2664 %error The test fails
2665%endif
2666-------------------------------------------------------------
2667+
2668You can also check to see if something is set:
2669+
2670-------------------------------------------------------------
2671%if ${something} != %{nil}
2672 %warning The test passes
2673%else
2674 %error The test fails
2675%endif
2676-------------------------------------------------------------
2677+>+:: The left hand side is numerically greater than the right hand side.
2678+>=+:: The left hand side is numerically greater than or equal to the right
2679hand side.
2680+<+:: The left hand side is numerically less than the right hand side.
2681+\<=+:: The left hand side is numerically less than or equal to the right hand
2682side.
2683
2684%ifn
2685^^^^
2686
2687The +%ifn+ macro inverts the normal +%if+ logic. It avoids needing to provide
2688empty _if_ blocks followed by _else_ blocks. It is useful when checking if a
2689macro is defined:
2690
2691-------------------------------------------------------------
2692%ifn %{defined foo}
2693 %define foo bar
2694%endif
2695-------------------------------------------------------------
2696
2697%ifarch
2698^^^^^^^
2699
2700The +%ifarch+ is a short cut for "+%if %{\_arch} == i386+". Currently not used.
2701
2702%ifnarch
2703^^^^^^^^
2704
2705The +%ifnarch+ is a short cut for "+%if %{\_arch} != i386+". Currently not
2706used.
2707
2708%ifos
2709^^^^^
2710
2711The +%ifos+ is a short cut for "+%if %{\_os} != mingw32+". It allows
2712conditional support for various operating system differences when building
2713packages.
2714
2715%else
2716^^^^^
2717
2718The +%else+ macro starts the conditional _else_ block.
2719
2720%endfi
2721^^^^^^
2722
2723The +%endif+ macro ends a conditional logic block.
2724
2725%bconf_with
2726^^^^^^^^^^^
2727
2728The +%bconf_with+ macro provides a way to test if the user has passed a
2729specific option on the command line with the +--with-<label>+ option. This
2730option is only available with the +sb-builder+ command.
2731
2732%bconf_without
2733^^^^^^^^^^^^^^
2734
2735The +%bconf_without+ macro provides a way to test if the user has passed a
2736specific option on the command line with the +--without-<label>+ option. This
2737option is only available with the +sb-builder+ command.
2738
2739Commands
2740--------
2741
2742Checker (sb-check)
2743~~~~~~~~~~~~~~~~~~
2744
2745This commands checks your system is set up correctly. Most options are ignored.
2746
2747-------------------------------------------------------------
2748$ ../source-builder/sb-check --help
2749sb-check: [options] [args]
2750RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2751Options and arguments:
2752--force                : Force the build to proceed
2753--quiet                : Quiet output (not used)
2754--trace                : Trace the execution
2755--dry-run              : Do everything but actually run the build
2756--warn-all             : Generate warnings
2757--no-clean             : Do not clean up the build tree
2758--always-clean         : Always clean the build tree, even with an error
2759--jobs                 : Run with specified number of jobs, default: num CPUs.
2760--host                 : Set the host triplet
2761--build                : Set the build triplet
2762--target               : Set the target triplet
2763--prefix path          : Tools build prefix, ie where they are installed
2764--topdir path          : Top of the build tree, default is $PWD
2765--configdir path       : Path to the configuration directory, default: ./config
2766--builddir path        : Path to the build directory, default: ./build
2767--sourcedir path       : Path to the source directory, default: ./source
2768--tmppath path         : Path to the temp directory, default: ./tmp
2769--macros file[,[file]  : Macro format files to load after the defaults
2770--log file             : Log file where all build out is written too
2771--url url[,url]        : URL to look for source
2772--no-download          : Disable the source downloader
2773--targetcflags flags   : List of C flags for the target code
2774--targetcxxflags flags : List of C++ flags for the target code
2775--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2776--with-<label>         : Add the --with-<label> to the build
2777--without-<label>      : Add the --without-<label> to the build
2778--regression           : Set --no-install, --keep-going and --always-clean
2779$ ../source-builder/sb-check
2780RTEMS Source Builder - Check, v0.2.0
2781Environment is ok
2782-------------------------------------------------------------
2783
2784Defaults (sb-defaults)
2785~~~~~~~~~~~~~~~~~~~~~~
2786
2787This commands outputs and the default macros for your when given no
2788arguments. Most options are ignored.
2789
2790-------------------------------------------------------------
2791$ ../source-builder/sb-defaults --help
2792sb-defaults: [options] [args]
2793RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2794Options and arguments:
2795--force                : Force the build to proceed
2796--quiet                : Quiet output (not used)
2797--trace                : Trace the execution
2798--dry-run              : Do everything but actually run the build
2799--warn-all             : Generate warnings
2800--no-clean             : Do not clean up the build tree
2801--always-clean         : Always clean the build tree, even with an error
2802--jobs                 : Run with specified number of jobs, default: num CPUs.
2803--host                 : Set the host triplet
2804--build                : Set the build triplet
2805--target               : Set the target triplet
2806--prefix path          : Tools build prefix, ie where they are installed
2807--topdir path          : Top of the build tree, default is $PWD
2808--configdir path       : Path to the configuration directory, default: ./config
2809--builddir path        : Path to the build directory, default: ./build
2810--sourcedir path       : Path to the source directory, default: ./source
2811--tmppath path         : Path to the temp directory, default: ./tmp
2812--macros file[,[file]  : Macro format files to load after the defaults
2813--log file             : Log file where all build out is written too
2814--url url[,url]        : URL to look for source
2815--no-download          : Disable the source downloader
2816--targetcflags flags   : List of C flags for the target code
2817--targetcxxflags flags : List of C++ flags for the target code
2818--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2819--with-<label>         : Add the --with-<label> to the build
2820--without-<label>      : Add the --without-<label> to the build
2821--regression           : Set --no-install, --keep-going and --always-clean
2822-------------------------------------------------------------
2823
2824Set Builder (sb-set-builder)
2825~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2826
2827This command builds a set.
2828
2829-------------------------------------------------------------
2830$ ../source-builder/sb-set-builder --help
2831RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2832Options and arguments:
2833--force                : Force the build to proceed
2834--quiet                : Quiet output (not used)
2835--trace                : Trace the execution
2836--dry-run              : Do everything but actually run the build
2837--warn-all             : Generate warnings
2838--no-clean             : Do not clean up the build tree
2839--always-clean         : Always clean the build tree, even with an error
2840--regression           : Set --no-install, --keep-going and --always-clean
2841---jobs                 : Run with specified number of jobs, default: num CPUs.
2842--host                 : Set the host triplet
2843--build                : Set the build triplet
2844--target               : Set the target triplet
2845--prefix path          : Tools build prefix, ie where they are installed
2846--topdir path          : Top of the build tree, default is $PWD
2847--configdir path       : Path to the configuration directory, default: ./config
2848--builddir path        : Path to the build directory, default: ./build
2849--sourcedir path       : Path to the source directory, default: ./source
2850--tmppath path         : Path to the temp directory, default: ./tmp
2851--macros file[,[file]  : Macro format files to load after the defaults
2852--log file             : Log file where all build out is written too
2853--url url[,url]        : URL to look for source
2854--no-download          : Disable the source downloader
2855--no-install           : Do not install the packages to the prefix
2856--targetcflags flags   : List of C flags for the target code
2857--targetcxxflags flags : List of C++ flags for the target code
2858--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2859--with-<label>         : Add the --with-<label> to the build
2860--without-<label>      : Add the --without-<label> to the build
2861--mail-from            : Email address the report is from.
2862--mail-to              : Email address to send the email too.
2863--mail                 : Send email report or results.
2864--smtp-host            : SMTP host to send via.
2865--no-report            : Do not create a package report.
2866--report-format        : The report format (text, html, asciidoc).
2867--bset-tar-file        : Create a build set tar file
2868--pkg-tar-files        : Create package tar files
2869--list-bsets           : List available build sets
2870--list-configs         : List available configurations
2871--list-deps            : List the dependent files.
2872-------------------------------------------------------------
2873
2874.Arguments
2875The +[args]+ are a list build sets to build.
2876
2877.Options
2878+--force+;;
2879Force the build to proceed even if the host check fails. Typically this happens
2880if executable files are found in the path at a different location to the host
2881defaults.
2882+--trace+;;
2883Trace enable printing of debug information to stdout. It is really only of use
2884to RTEMS Source Builder's developers.
2885+--dry-run+;;
2886Do everything but actually run the build commands. This is useful when checking
2887a new configuration parses cleanly.
2888+--warn-all+;;
2889Generate warnings.
2890+--no-clean+;;
2891Do not clean up the build tree during the cleaning phase of the build. This
2892leaves the source and the build output on disk so you can make changes, or
2893amend or generate new patches. It also allows you to review configure type
2894output such as +config.log+.
2895+--always-clean+;;
2896Clean away the results of a build even if the build fails. This is normally
2897used with `--keep-going` when regression testing to see which build sets
2898fail to build. It keeps the disk usage down.
2899+--jobs+;;
2900Control the number of jobs make is given. The jobs can be 'none' for only 1
2901job, 'half' so the number of jobs is half the number of detected cores, a
2902fraction such as '0.25' so the number of jobs is a quarter of the number of
2903detected cores and a number such as '25' which forces the number of jobs to
2904that number.
2905+--host+;;
2906Set the host triplet value. Be careful with this option.
2907+--build+;;
2908Set the build triplet. Be careful with this option.
2909+--target+;;
2910Set the target triplet. Be careful with this option. This is useful if you have
2911a generic configuration script that can work for a range of architectures.
2912+--prefix path+;;
2913Tools build prefix, ie where they are installed.
2914+--topdir path+;;
2915Top of the build tree, that is the current directory you are in.
2916+--configdir path+;;
2917Path to the configuration directory. This overrides the built in defaults.
2918+--builddir path+;;
2919Path to the build directory. This overrides the default of +build+.
2920+--sourcedir path+;;
2921Path to the source directory. This overrides the default of +source+.
2922+--tmppath path+;;
2923Path to the temporary directory. This overrides the default of +tmp+.
2924+--macros files+;;
2925Macro files to load. The configuration directory path is searched.
2926+--log file+;;
2927Log all the output from the build process. The output is directed to +stdout+
2928if no log file is provided.
2929+--url url+;;
2930URL to look for source when downloading. This is can be comma separate list.
2931+--no-download+;;
2932Disable downloading of source and patches. If the source is not found an error
2933is raised.
2934+--targetcflags flags+;;
2935List of C flags for the target code. This allows for specific local
2936customisation when testing new variations.
2937+--targetcxxflags flags+;;
2938List of C++ flags for the target code. This allows for specific local
2939customisation when testing new variations.
2940+--libstdcxxflags flags+;;
2941List of C\++ flags to build the target libstdc++ code. This allows for specific
2942local customisation when testing new variations.
2943+--with-<label>+;;
2944Add the --with-<label> to the build. This can be tested for in a script with
2945the +%bconf_with+ macro.
2946+--without-<label>+;;
2947Add the --without-<label> to the build. This can be tested for in a script with
2948the +%bconf_without+ macro.
2949+--mail-from+;;
2950Set the from mail address if report mailing is enabled.
2951+--mail-to+;;
2952Set the to mail address if report mailing is enabled. The report is mailed to
2953this address.
2954+--mail+;;
2955Mail the build report to the mail to address.
2956+--smtp-host+;;
2957The SMTP host to use to send the email. The default is +localhost+.
2958+--no-report+;;
2959Do not create a report format.
2960+--report-format format+;;
2961The report format can be 'text' or 'html'. The default is 'html'.
2962+--keep-going+;;
2963Do not stop on error. This is useful if your build sets performs a large number
2964of testing related builds and there are errors.
2965+--always-clean+.
2966Always clean the build tree even with a failure.
2967+--no-install+;;
2968Do not install the packages to the prefix. Use this if you are only after the
2969tar files.
2970+--regression+;;
2971A convenience option which is the same as +--no-install+, +--keep-going+ and
2972+--bset-tar-file+;;
2973Create a build set tar file. This is a single tar file of all the packages in
2974the build set.
2975+--pkg-tar-files+;;
2976Create package tar files. A tar file will be created for each package built in
2977a build set.
2978+--list-bsets+;;
2979List available build sets.
2980+--list-configs+;;
2981List available configurations.
2982+--list-deps+;;
2983Print a list of dependent files used by a build set. Dependent files have a
2984'dep[?]' prefix where '?' is a number. The files are listed alphabetically.
2985
2986Set Builder (sb-builder)
2987~~~~~~~~~~~~~~~~~~~~~~~~
2988
2989This command builds a configuration as described in a configuration
2990file. Configuration files have the extension of +.cfg+.
2991
2992-------------------------------------------------------------
2993$ ./source-builder/sb-builder --help
2994sb-builder: [options] [args]
2995RTEMS Source Builder, an RTEMS Tools Project (c) 2012 Chris Johns
2996Options and arguments:
2997--force                : Force the build to proceed
2998--quiet                : Quiet output (not used)
2999--trace                : Trace the execution
3000--dry-run              : Do everything but actually run the build
3001--warn-all             : Generate warnings
3002--no-clean             : Do not clean up the build tree
3003--always-clean         : Always clean the build tree, even with an error
3004--jobs                 : Run with specified number of jobs, default: num CPUs.
3005--host                 : Set the host triplet
3006--build                : Set the build triplet
3007--target               : Set the target triplet
3008--prefix path          : Tools build prefix, ie where they are installed
3009--topdir path          : Top of the build tree, default is $PWD
3010--configdir path       : Path to the configuration directory, default: ./config
3011--builddir path        : Path to the build directory, default: ./build
3012--sourcedir path       : Path to the source directory, default: ./source
3013--tmppath path         : Path to the temp directory, default: ./tmp
3014--macros file[,[file]  : Macro format files to load after the defaults
3015--log file             : Log file where all build out is written too
3016--url url[,url]        : URL to look for source
3017--targetcflags flags   : List of C flags for the target code
3018--targetcxxflags flags : List of C++ flags for the target code
3019--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
3020--with-<label>         : Add the --with-<label> to the build
3021--without-<label>      : Add the --without-<label> to the build
3022--list-configs         : List available configurations
3023-------------------------------------------------------------
3024
3025Host Setups
3026-----------
3027
3028The host versions are listed. If a later version of the host operating system
3029exists it should work unless listed.
3030
3031Please provide patches to update these sections if they are wrong or need
3032updating. I cannot install and test each one and rely on getting your feedback.
3033
3034Linux
3035~~~~~
3036
3037A number of different Linux distrubutions are known to work. The following have
3038been tested and report as working.
3039
3040Archlinux
3041^^^^^^^^^
3042
3043The following packages are required on a fresh Archlinux 64bit installation:
3044
3045--------------------------------------------------------------
3046# pacman -S base-devel gdb xz unzip ncurses git zlib
3047--------------------------------------------------------------
3048
3049Archlinux, by default installs `texinfo-5` which is incompatible for building
3050GCC 4.7 tree. You will have to obtain `texinfo-legacy` from `AUR` and provide
3051a manual override.
3052
3053--------------------------------------------------------------
3054# pacman -R texinfo
3055$ yaourt -S texinfo-legacy
3056# ln -s /usr/bin/makeinfo-4.13a /usr/bin/makeinfo
3057--------------------------------------------------------------
3058
3059CentOS
3060^^^^^^
3061
3062The following packages are required on a minimal CentOS 6.3 64bit installation:
3063
3064-------------------------------------------------------------
3065# yum install autoconf automake binutils gcc gcc-c++ gdb make patch \
3066bison flex xz unzip ncurses-devel texinfo zlib-devel python-devel git
3067-------------------------------------------------------------
3068
3069The minimal CentOS distribution is a specific DVD that installs a minimal
3070system. If you use a full system some of these packages may have been
3071installed.
3072
3073Fedora
3074^^^^^^
3075
3076The RTEMS Source Builder has been tested on Fedora 19 64bit with the following packages.
3077
3078-------------------------------------------------------------
3079# yum install ncurses-devel python-devel git bison gcc cvs gcc-c++ \
3080         flex texinfo patch perl-Text-ParseWords zlib-devel
3081-------------------------------------------------------------
3082
3083Raspbian
3084^^^^^^^^
3085
3086The is the Debian distribution for the Raspberry Pi. The following packages are
3087required.
3088
3089-------------------------------------------------------------
3090$ sudo apt-get install autoconf automake bison flex binutils gcc g++ gdb \
3091texinfo unzip ncurses-dev python-dev git
3092-------------------------------------------------------------
3093
3094It is recommended you get Model B of the Pi with 512M of memory and to mount a
3095remote disk over the network. The tools can be build with a prefix under your
3096home directory as recommended and end up on the SD card.
3097
3098Ubuntu
3099^^^^^^
3100
3101The latest testing was with Ubuntu 13.10 64bit. This section also includes Xubuntu. A
3102minimal installation was used and the following packages installed.
3103
3104-------------------------------------------------------------
3105$ sudo apt-get build-dep binutils gcc g++ gdb unzip git python2.7-dev
3106-------------------------------------------------------------
3107
3108FreeBSD
3109~~~~~~~
3110
3111The RTEMS Source Builder has been tested on FreeBSD 9.1 and 10.0 64bit. You
3112need to install some ports. They are:
3113
3114-------------------------------------------------------------
3115# cd /usr/ports
3116# portinstall --batch lang/python27
3117-------------------------------------------------------------
3118
3119If you wish to build Windows (mingw32) tools please install the following
3120ports:
3121
3122-------------------------------------------------------------
3123# cd /usr/ports
3124# portinstall --batch devel/mingw32-binutils devel/mingw32-gcc
3125# portinstall --batch devel/mingw32-zlib devel/mingw32-pthreads
3126-------------------------------------------------------------
3127
3128The +zlip+ and +pthreads+ ports for MinGW32 are used for builiding a Windows
3129QEMU.
3130
3131If you are on FreeBSD 10.0 and you have pkgng installed you can use 'pkg
3132install' rather than 'portinstall'.
3133
3134NetBSD
3135~~~~~~
3136
3137The RTEMS Source Builder has been tested on NetBSD 6.1 i386. Packages to add
3138are:
3139
3140-------------------------------------------------------------
3141# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/devel/gmake-3.82nb7.tgz
3142# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/devel/bison-2.7.1.tgz
3143# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/archivers/xz-5.0.4.tgz
3144-------------------------------------------------------------
3145
3146MacOS
3147~~~~~
3148
3149The RTEMS Source Builder has been tested on Mountain Lion. You will need to
3150install the Xcode app using the _App Store_ tool, run Xcode and install the
3151Developers Tools package within Xcode.
3152
3153Linux Mint
3154^^^^^^^^^^
3155
3156zlib package is required on Linux Mint. It has a different name (other
3157than the usual zlib-dev):
3158
3159-------------------------------------------------------------
3160# sudo apt-get install zlib1g-dev
3161-------------------------------------------------------------
3162
3163Mavericks
3164^^^^^^^^^
3165
3166The RSB works on Mavericks and the GNU tools can be built for RTEMS using the
3167Mavericks clang LLVM tool chain. You will need to build and install a couple of
3168packages to make the RSB pass the +sb-check+. These are CVS and XZ. You can get
3169these tools from a packaging tool for MacOS such as _MacPorts_ or _HomeBrew_.
3170
3171I do not use 3rd party packaging on MacOS and prefer to build the packages from
3172source using a prefix of '/usr/local'. There are good 3rd party packages around
3173however they sometimes bring in extra dependence and that complicates my build
3174environment and I want to know the minimal requirements when building
3175tools. The following are required:
3176
3177. The XZ package's home page is http://tukaani.org/xz/ and I use version
3178  5.0.5. XZ builds and installs cleanly.
3179
3180. CVS can be found at http://cvs.nongnu.org/ and I use version 1.11.23. CVS
3181  requires the following patch
3182  http://www.rtems.org/ftp/pub/rtems/people/chrisj/source-builder/cvs-1.11.23-osx-maverick.diff
3183  to build. Place the diff in the same directory as the unpacked cvs-1.11.23
3184  and apply with +patch -p0 < cvs-1.11.23-osx-maverick.diff+.
3185
3186Windows
3187~~~~~~~
3188
3189Windows tool sets are supported. The tools are native Windows executable which
3190means they do not need an emulation layer to run once built. The tools
3191understand and use standard Windows paths and integrate easily into Windows IDE
3192environments because they understand and use standard Windows paths. Native
3193Windows tools have proven over time to be stable and reliable with good
3194performance. If you are a Windows user or you are required to use Windows you
3195can still develop RTEMS application as easily as a Unix operating system. Some
3196debugging experiences may vary and if this is an issue please raised the topic
3197on the RTEMS Users mailing list.
3198
3199Building the tools or some other packages may require a Unix or POSIX type
3200shell. There are a few options, Cygwin and MSYS2. I recommend MSYS2.
3201
3202.Ready To Go Windows Tools
3203NOTE: From time to time I provide tools for Windows at
3204http://ftp.rtems.org/pub/rtems/people/chrisj/source-builder/4.11/mingw32/
3205
3206MSYS2
3207
3208This is a new version of the old MinGW project's original MSYS based around the
3209Arch Linux pacman packager. MSYS and MSYS2 are a specific fork of the Cygwin
3210project with some fundamental changes in the handling of paths and mounts that
3211allow easy interaction between the emulated POSIX environment and the native
3212Windows environment.
3213
3214Install MSYS2 using the installer you can download from
3215https://msys2.github.io/. Follow the instructions on the install page and make
3216sure you remove any global path entries to any other Cygwin, MinGW, MSYS or
3217packages that may uses a Cygwin DLL, for example some ports of Git.
3218
3219To build the tools you need install the following packages using pacman:
3220
3221 $ pacman -S git cvs bison make texinfo patch unzip diffutils tar \
3222          mingw64/mingw-w64-x86_64-gcc mingw64/mingw-w64-x86_64-binutils
3223
3224To build make sure you add '--without-python --jobs=none' to the standard RSB
3225command line. MSYS2 has a temp file name issue and so the GNU AR steps on
3226itself when running in parallel on SMP hardware which means we have to set the
3227jobs option to none.
3228
3229Install a suitable version of Python from http://www.python.org/ and add it to
3230the start of your path. The MSYS2 python does not work with waf.
3231
3232Cygwin
3233
3234Building on Windows is a little more complicated because the Cygwin shell is
3235used rather than the MSYS2 shell. The MSYS2 shell is simpler because the
3236detected host triple is MinGW so the build is standard cross-compiler build.
3237A Canadian cross-build using Cygwin is supported if you would like native tools.
3238
3239Install a recent Cygwin version using the Cygwin setup tool. Select and install
3240the groups and packages listed:
3241
3242.Cygwin Packages
3243[options="header,compact",width="50%",cols="20%,80%"]
3244|================================
3245|Group   |Package
3246|Archive |bsdtar
3247|        |unzip
3248|        |xz
3249|Devel   |autoconf
3250|        |autoconf2.1
3251|        |autoconf2.5
3252|        |automake
3253|        |binutils
3254|        |bison
3255|        |flex
3256|        |gcc4-core
3257|        |gcc4-g++
3258|        |git
3259|        |make
3260|        |mingw64-x86_64-binutils
3261|        |mingw64-x86_64-gcc-core
3262|        |mingw64-x86_64-g++
3263|        |mingw64-x86_64-runtime
3264|        |mingw64-x86_64-zlib
3265|        |patch
3266|        |zlib-devel
3267|MinGW   |mingw-zlib-devel
3268|Python  |python
3269|================================
3270
3271The setup tool will add a number of dependent package and it is ok to accept
3272them.
3273
3274I have found turning off Windows Defender improves performance if you have
3275another up to date virus detection tool installed and enabled. I used the
3276excellent `Process Hacker 2` tool to monitor the performance and I found the
3277Windows Defender service contributed a high load. In my case I had a 3rd party
3278virus tool installed so the Windows Defender service was not needed.
3279
3280A Canadian cross-compile (Cxc) is required on Cygwin because the host is Cygwin
3281therefore a traditional cross-compile will result in Cygiwn binaries. With a
3282Canadian cross-compile a Cygwin cross-compiler is built as well as the MinGW
3283RTEMS cross-compiler. The Cygwin cross-compiler is required to build the C
3284runtime for the RTEMS target because we are building under Cygiwn. The build
3285output for an RTEMS 4.10 ARM tool set is:
3286
3287-------------------------------------------------------------
3288chris@cygthing ~/development/rtems/src/rtems-source-builder/rtems
3289$ ../source-builder/sb-set-builder --log=l-arm.txt --prefix=$HOME/development/rtems/4.10 4.10/rtems-arm
3290RTEMS Source Builder - Set Builder, v0.2
3291Build Set: 4.10/rtems-arm
3292config: expat-2.1.0-1.cfg
3293package: expat-2.1.0-x86_64-w64-mingw32-1
3294building: expat-2.1.0-x86_64-w64-mingw32-1
3295reporting: expat-2.1.0-1.cfg -> expat-2.1.0-x86_64-w64-mingw32-1.html
3296config: tools/rtems-binutils-2.20.1-1.cfg
3297package: arm-rtems4.10-binutils-2.20.1-1 <1>
3298building: arm-rtems4.10-binutils-2.20.1-1
3299package: (Cxc) arm-rtems4.10-binutils-2.20.1-1 <2>
3300building: (Cxc) arm-rtems4.10-binutils-2.20.1-1
3301reporting: tools/rtems-binutils-2.20.1-1.cfg ->
3302arm-rtems4.10-binutils-2.20.1-1.html
3303config: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg
3304package: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
3305building: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
3306package: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
3307building: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
3308reporting: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg ->
3309arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1.html
3310config: tools/rtems-gdb-7.3.1-1.cfg
3311package: arm-rtems4.10-gdb-7.3.1-1
3312building: arm-rtems4.10-gdb-7.3.1-1
3313reporting: tools/rtems-gdb-7.3.1-1.cfg -> arm-rtems4.10-gdb-7.3.1-1.html
3314config: tools/rtems-kernel-4.10.2.cfg
3315package: arm-rtems4.10-kernel-4.10.2-1
3316building: arm-rtems4.10-kernel-4.10.2-1
3317reporting: tools/rtems-kernel-4.10.2.cfg -> arm-rtems4.10-kernel-4.10.2-1.html
3318installing: expat-2.1.0-x86_64-w64-mingw32-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
3319installing: arm-rtems4.10-binutils-2.20.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10 <3>
3320installing: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
3321installing: arm-rtems4.10-gdb-7.3.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
3322installing: arm-rtems4.10-kernel-4.10.2-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
3323cleaning: expat-2.1.0-x86_64-w64-mingw32-1
3324cleaning: arm-rtems4.10-binutils-2.20.1-1
3325cleaning: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
3326cleaning: arm-rtems4.10-gdb-7.3.1-1
3327cleaning: arm-rtems4.10-kernel-4.10.2-1
3328Build Set: Time 10:09:42.810547 <4>
3329-------------------------------------------------------------
3330
3331<1> The Cygwin version of the ARM cross-binutils.
3332<2> The +(Cxc)+ indicates this is the MinGW build of the package.
3333<3> Only the MinGW version is installed.
3334<4> Cygwin is slow so please be patient. This time was on an AMD Athlon 64bit
3335    Dual Core 6000+ running at 3GHz with 4G RAM running Windows 7 64bit.
3336
3337CAUTION: Cygwin documents the 'Big List Of Dodgy Apps' or 'BLODA'. The link is
3338http://cygwin.com/faq/faq.html#faq.using.bloda and it is worth a
3339look. You will see a large number of common pieces of software found on Windows
3340systems that can cause problems. My testing has been performed with NOD32
3341running and I have seen some failures. The list is for all of Cygwin so I am
3342not sure which of the listed programs effect the RTEMS Source Biulder. The
3343following FAQ item talks about +fork+ failures and presents some technical
3344reasons they cannot be avoided in all cases. Cygwin and it's fork MSYS are
3345fantastic pieces of software in a difficult environment. I have found building
3346a single tool tends to work, building all at once is harder.
3347
3348
3349Build Status By Host
3350~~~~~~~~~~~~~~~~~~~~
3351
3352This table lists the current build and testing status for reported hosts:
3353
3354[grid="rows",format="csv"]
3355[options="header",cols="<,<,<,<,<,<"]
3356|===========================
3357OS,Uname,4.9,4.10,4.11,Comments
3358include::host-results.csv[]
3359|===========================
3360
3361[NOTE]
3362==================================================================
3363Report any unlisted hosts as a patch.
3364==================================================================
3365
3366History
3367-------
3368
3369The RTEMS Source Builder is a stand alone tool based on another tool called the
3370'SpecBuilder'. The SpecBuilder was written for the RTEMS project to give me a
3371way to build tools on hosts that did not support RPMs. At the time the RTEMS
3372tools maintainer only used spec files to create various packages. This meant I
3373had either spec files, RPM files or SRPM files. The RPM and SPRM files where
3374useless because you needed an 'rpm' type tool to extract and manage them. There
3375are versions of 'rpm' for a number of non-RPM hosts however these proved to be
3376in various broken states and randomly maintained. The solution I settled on was
3377to use spec files so I wrote a Python based tool that parsed the spec file
3378format and allowed me to create a shell script I could run to build the
3379package. This approach proved successful and I was able to track the RPM
3380version of the RTEMS tools on a non-RPM host over a number of years. however
3381the SpecBuilder tool did not help me build tools or other packages not related
3382to the RTEMS project where there was no spec file I could use so I needed
3383another tool. Rather than start again I decided to take the parsing code for
3384the spec file format and build a new tool called the RTEMS Source Builder.
Note: See TracBrowser for help on using the repository browser.