source: rtems-source-builder/doc/source-builder.txt @ e119c6a

4.104.114.9
Last change on this file since e119c6a was e119c6a, checked in by Chris Johns <chrisj@…>, on May 14, 2014 at 6:38:24 AM

doc: Update the documentation for the new source and patch.

  • Property mode set to 100644
File size: 123.3 KB
Line 
1RTEMS Source Builder
2====================
3:doctype: book
4:toc2:
5:toclevels: 5
6:icons:
7:numbered:
8
9image:images/rtemswhitebg.jpg["RTEMS",width="30%"]
10
11Chris Johns <chrisj@rtems.org>
121.7, May 2014
13
14RTEMS Tools From Source
15-----------------------
16
17The RTEMS Source Builder is a tool to aid building packages from source used by
18the RTEMS project. It helps consolidate the details you need to build a package
19from source in a controlled and verifiable way. The tool is aimed at developers
20of software who use tool sets for embedded type development and is not limited
21to building just for RTEMS. Embedded development typically uses cross-compiling
22tool chains, debuggers, and debugging aids. Together we call these a 'tool
23set'. The RTEMS Source Builder is not limited to this role but designed to fit
24with-in this specific niche. It can be used outside of the RTEMS project and we
25welcome this happening in other open source or commercial projects.
26
27The RTEMS Source Builder is typically used to build a set of tools or a 'build
28set'. A 'build set' is a collection of packages and a package is a specific
29tool, for example gcc or gdb. The RTEMS Source Builder attempts to support any
30host environment that runs Python and you can build the package on. It is not
31some sort of magic that can take any piece of source code and make it
32build. Someone at some point in time has figured out how to build that package
33from source and taught this tool. The RTEMS Source Builder has been tested on:
34
35[[platform_links]]
36* <<_archlinux,Archlinux>>
37* <<_centos,Centos>>
38* <<_fedora,Fedora>>
39* <<_freebsd,FreeBSD>>
40* <<_netbsd,NetBSD>>
41* <<_macos,MacOS>>
42* <<_mint,Linux Mint>>
43* <<_opensuse,openSUSE>>
44* <<_raspbian,Raspbian>>
45* <<_ubuntu,Ubuntu>>
46* <<_windows,Windows>>
47* <<_ubuntu,Xubuntu>>
48
49The RTEMS Source Builder has two types configuration data. The first is the
50'build set'. A _build set_ describes a collection of packages that define a set
51of tools you would use when developing software for RTEMS. For example the
52basic GNU tool set is binutils, gcc, and gdb and is the typical base suite of
53tools you need for an embedded cross-development type project. The second type
54of configuration data is the configuration files and they define how a package
55is built. Configuration files are scripts loosely based on the RPM spec file
56format and detail the steps needed to build a package. The steps are
57'preparation', 'building', and 'installing'. Scripts support macros, shell
58expansion, logic, includes plus many more features useful when build packages.
59
60The RTEMS Source Builder does not interact with any host package management
61systems. There is no automatic dependence checking between various packages you
62build or packages and software your host system you may have installed. We
63assume the build sets and configuration files you are using have been created
64by developers who do. If you have a problem please ask on the RTEMS Users
65mailing list.
66
67*************************************************************
68IMPORTANT: If you have a problem please see <<_bugs,the reporting bugs>>
69           section.
70*************************************************************
71
72Quick Start
73-----------
74
75The quick start will show you how to build a set of RTEMS tools for a supported
76architecture. The tools are installed into a build _prefix_ and this is top of
77a group of directories containing all the files needed to develop RTEMS
78applications. Building an RTEMS build set will build all that you need. This
79includes autoconf, automake, assemblers, linkers, compilers, debuggers,
80standard libraries and RTEMS itself.
81
82There is no need to become root or the administrator and we recommend you avoid
83doing this. You can build and install the tools anywhere on the host's file
84system you, as a standard user, have read and write access too. I recommend you
85use your home directory and work under the directory `~/development/rtems`. The
86examples shown here will do this.
87
88You can use the build _prefix_ to install and maintain different versions of
89the tools. Doing this lets you try a new set of tools while not touching your
90proven working production set of tools. Once you have proven the new tools are
91working rebuild with the 'production' prefix switching your development to them.
92
93I also suggest you keep your environment to the bare minimum, particularly the
94path variable. Using environment variables has been proven over the years to be
95difficult to manage in production systems.
96
97.Host Setup
98*************************************************************
99IMPORTANT: Before proceeding to the next section please refer to the
100<<_host_setups,host specific setup>> for your host and install any extra
101packages. The RSB assumes the needed packages are installed and work.
102*************************************************************
103
104.Path to use when building applications
105*************************************************************
106TIP: Do not forget to do this before you use the tools such as build RTEMS.
107
108The RSB by default will install (copy) the executables under the prefix you
109supply. To use the tools once finished just set your path to the 'bin'
110directory under the _prefix_ you use. In the examples that follow the _prefix_
111is `$HOME/development/rtems/4.11` and is set using the `--prefix` option so the
112path you need to configure to build applications can be set with the following
113in a BASH shell:
114
115-------------------------------------------------------------
116$ export PATH=$HOME/development/rtems/4.11/bin:$PATH
117-------------------------------------------------------------
118Make sure you place the RTEMS tool path at the front of your path so they are
119searched first. RTEMS can provide newer versions of some tools your operating
120system provides and placing the RTEMS tools path at the front means it is
121searched first and the RTEMS needed versions of the tools are used.
122*************************************************************
123
124Setup
125~~~~~
126
127Setup a development work space:
128
129-------------------------------------------------------------
130$ cd
131$ mkdir -p development/rtems/src
132$ cd development/rtems/src
133-------------------------------------------------------------
134
135The RTEMS Source Builder is distributed as source. It is Python code so all you
136need to do is head over to the RTEMS GIT repository and clone the code directly
137from git:
138
139-------------------------------------------------------------
140$ git clone git://git.rtems.org/rtems-source-builder.git
141$ cd rtems-source-builder
142-------------------------------------------------------------
143
144Checking
145~~~~~~~~
146
147The next step is to check if your host is set up correctly. The RTEMS Source
148Builder provides a tool to help:
149
150-------------------------------------------------------------
151$ source-builder/sb-check
152warning: exe: absolute exe found in path: (__objcopy) /usr/local/bin/objcopy <1>
153warning: exe: absolute exe found in path: (__objdump) /usr/local/bin/objdump
154error: exe: not found: (_xz) /usr/local/bin/xz <2>
155RTEMS Source Builder environment is not correctly set up
156$ source-builder/sb-check
157RTEMS Source Builder environment is ok <3>
158-------------------------------------------------------------
159
160<1> A tool is in the environment path but does not match the expected path.
161<2> The executable 'xz' is not found.
162<3> The host's environment is set up correct.
163
164The checking tool will output a list of executable files not found if problems
165are detected. Locate those executable files and install them. You may also be
166given a list of warnings about executable files not in the expected location
167however the executable was located somewhere in your environment's path. You
168will need to check each tool to determine if this is an issue. An executable
169not in the standard location may indicate it is not the host operating system's
170standard tool. It maybe ok or it could be buggy, only you can determine this.
171
172The <<_host_setups,Host Setups>> section lists packages you should install for
173common host operating systems. It maybe worth checking if you have those
174installed.
175
176Build Sets
177~~~~~~~~~~
178
179The RTEMS tools can be built within the RTEMS Source Builder's source tree. We
180recommend you do this so lets change into the RTEMS directory in the RTEMS
181Source Builder package:
182
183-------------------------------------------------------------
184$ cd rtems
185-------------------------------------------------------------
186
187If you are unsure how to specify the build set for the architecture you wish to
188build ask the tool:
189
190-------------------------------------------------------------
191$ ../source-builder/sb-set-builder --list-bsets <1>
192RTEMS Source Builder - Set Builder, v0.2.0
193Examining: config <2>
194Examining: ../source-builder/config <2>
195    4.10/rtems-all.bset <3>
196    4.10/rtems-arm.bset <4>
197    4.10/rtems-autotools.bset
198    4.10/rtems-avr.bset
199    4.10/rtems-bfin.bset
200    4.10/rtems-h8300.bset
201    4.10/rtems-i386.bset
202    4.10/rtems-lm32.bset
203    4.10/rtems-m32c.bset
204    4.10/rtems-m32r.bset
205    4.10/rtems-m68k.bset
206    4.10/rtems-mips.bset
207    4.10/rtems-nios2.bset
208    4.10/rtems-powerpc.bset
209    4.10/rtems-sh.bset
210    4.10/rtems-sparc.bset
211    4.11/rtems-all.bset
212    4.11/rtems-arm.bset
213    4.11/rtems-autotools.bset
214    4.11/rtems-avr.bset
215    4.11/rtems-bfin.bset
216    4.11/rtems-h8300.bset
217    4.11/rtems-i386.bset
218    4.11/rtems-lm32.bset
219    4.11/rtems-m32c.bset
220    4.11/rtems-m32r.bset
221    4.11/rtems-m68k.bset
222    4.11/rtems-microblaze.bset
223    4.11/rtems-mips.bset
224    4.11/rtems-moxie.bset
225    4.11/rtems-nios2.bset
226    4.11/rtems-powerpc.bset
227    4.11/rtems-sh.bset
228    4.11/rtems-sparc.bset
229    4.11/rtems-sparc64.bset
230    4.11/rtems-v850.bset
231    4.9/rtems-all.bset
232    4.9/rtems-arm.bset
233    4.9/rtems-autotools.bset
234    4.9/rtems-i386.bset
235    4.9/rtems-m68k.bset
236    4.9/rtems-mips.bset
237    4.9/rtems-powerpc.bset
238    4.9/rtems-sparc.bset
239    gnu-tools-4.6.bset
240    rtems-4.10-base.bset <5>
241    rtems-4.11-base.bset
242    rtems-4.9-base.bset
243    rtems-base.bset <5>
244    unstable/4.11/rtems-all.bset <6>
245    unstable/4.11/rtems-arm.bset
246    unstable/4.11/rtems-avr.bset
247    unstable/4.11/rtems-bfin.bset
248    unstable/4.11/rtems-h8300.bset
249    unstable/4.11/rtems-i386.bset
250    unstable/4.11/rtems-lm32.bset
251    unstable/4.11/rtems-m32c.bset
252    unstable/4.11/rtems-m32r.bset
253    unstable/4.11/rtems-m68k.bset
254    unstable/4.11/rtems-microblaze.bset
255    unstable/4.11/rtems-mips.bset
256    unstable/4.11/rtems-moxie.bset
257    unstable/4.11/rtems-powerpc.bset
258    unstable/4.11/rtems-sh.bset
259    unstable/4.11/rtems-sparc.bset
260    unstable/4.11/rtems-sparc64.bset
261    unstable/4.11/rtems-v850.bset
262-------------------------------------------------------------
263<1> Only option needed is +--list-bsets+
264<2> The paths inspected. See <<X1,+_configdir+>> variable.
265<3> Build all RTEMS 4.10 supported architectures.
266<4> The build set for the ARM architecture on RTEMS 4.10.
267<5> Support build set file with common functionality included by other build
268    set files.
269<6> Unstable tool sets are used by RTEMS developers to test new tool sets. You
270    are welcome to try them but you must remember they are unstable, can change
271    at any point in time and your application may not work. If you have an
272    issue with a production tool it may pay to try the unstable tool to see if
273    it has been resolved.
274
275Building
276~~~~~~~~
277
278In this quick start I will build a SPARC tool set.
279
280-------------------------------------------------------------
281$ ../source-builder/sb-set-builder --log=l-sparc.txt <1> \
282                --prefix=$HOME/development/rtems/4.11 <2> 4.11/rtems-sparc <3>
283Source Builder - Set Builder, v0.2.0
284Build Set: 4.11/rtems-sparc
285config: expat-2.1.0-1.cfg <4>
286package: expat-2.1.0-x86_64-freebsd9.1-1
287building: expat-2.1.0-x86_64-freebsd9.1-1
288config: tools/rtems-binutils-2.22-1.cfg <5>
289package: sparc-rtems4.11-binutils-2.22-1
290building: sparc-rtems4.11-binutils-2.22-1
291config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg <6>
292package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
293building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
294config: tools/rtems-gdb-7.5.1-1.cfg <7>
295package: sparc-rtems4.11-gdb-7.5.1-1
296building: sparc-rtems4.11-gdb-7.5.1-1
297installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <8>
298installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
299installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
300installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
301cleaning: expat-2.1.0-x86_64-freebsd9.1-1 <9>
302cleaning: sparc-rtems4.11-binutils-2.22-1
303cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
304cleaning: sparc-rtems4.11-gdb-7.5.1-1
305Build Set: Time 0:13:43.616383 <10>
306-------------------------------------------------------------
307
308<1> Providing a log file redirects the build output into a file. Logging the
309    build output provides a simple way to report problems.
310<2> The prefix is the location on your file system the tools are installed
311    into. Provide a prefix to a location you have read and write access. You
312    can use the prefix to install different versions or builds of tools. Just
313    use the path to the tools you want to use when building RTEMS.
314<3> The build set. This is the SPARC build set. First a specifically referenced
315    file is checked for and if not found the '%{_configdir} path is
316    searched. The set builder will first look for files with a +.bset+
317    extension and then for a configuration file with a +.cfg+ extension.
318<4> The SPARC build set first builds the expat library as is used in GDB. This
319    is the expat configuration used.
320<5> The binutils build configuration.
321<6> The GCC and Newlib build configuration.
322<7> The GDB build configuration.
323<8> Installing the built packages to the install prefix.
324<9> All the packages built are cleaned at the end. If the build fails all the
325    needed files are present. You may have to clean up by deleting the build
326    directory if the build fails.
327<10> The time to build the package. This lets you see how different host
328     hardware or configurations perform.
329
330Your SPARC RTEMS 4.11 tool set will be installed and ready to build RTEMS and
331RTEMS applications. When the build runs you will notice the tool fetch the
332source code from the internet. These files are cached in a directory called
333+source+. If you run the build again the cached files are used. This is what
334happened in the show example before.
335
336The installed tools:
337
338-------------------------------------------------------------
339$ ls $HOME/development/rtems/4.11
340bin         include     lib         libexec     share       sparc-rtems4.11
341$ ls $HOME/development/rtems/4.11/bin
342sparc-rtems4.11-addr2line       sparc-rtems4.11-cpp
343sparc-rtems4.11-gcc-ar          sparc-rtems4.11-gprof
344sparc-rtems4.11-objdump         sparc-rtems4.11-size
345sparc-rtems4.11-ar              sparc-rtems4.11-elfedit
346sparc-rtems4.11-gcc-nm          sparc-rtems4.11-ld
347sparc-rtems4.11-ranlib          sparc-rtems4.11-strings
348sparc-rtems4.11-as              sparc-rtems4.11-g++
349sparc-rtems4.11-gcc-ranlib      sparc-rtems4.11-ld.bfd
350sparc-rtems4.11-readelf         sparc-rtems4.11-strip
351sparc-rtems4.11-c++             sparc-rtems4.11-gcc
352sparc-rtems4.11-gcov            sparc-rtems4.11-nm
353sparc-rtems4.11-run             xmlwf
354sparc-rtems4.11-c++filt         sparc-rtems4.11-gcc-4.7.2
355sparc-rtems4.11-gdb             sparc-rtems4.11-objcopy
356sparc-rtems4.11-sis
357$ $HOME/development/rtems/4.11/bin/sparc-rtems4.11-gcc -v
358Using built-in specs.
359COLLECT_GCC=/home/chris/development/rtems/4.11/bin/sparc-rtems4.11-gcc
360COLLECT_LTO_WRAPPER=/usr/home/chris/development/rtems/4.11/bin/../ \
361libexec/gcc/sparc-rtems4.11/4.7.2/lto-wrapper
362Target: sparc-rtems4.11 <1>
363Configured with: ../gcc-4.7.2/configure <2>
364--prefix=/home/chris/development/rtems/4.11
365--bindir=/home/chris/development/rtems/4.11/bin
366--exec_prefix=/home/chris/development/rtems/4.11
367--includedir=/home/chris/development/rtems/4.11/include
368--libdir=/home/chris/development/rtems/4.11/lib
369--libexecdir=/home/chris/development/rtems/4.11/libexec
370--mandir=/home/chris/development/rtems/4.11/share/man
371--infodir=/home/chris/development/rtems/4.11/share/info
372--datadir=/home/chris/development/rtems/4.11/share
373--build=x86_64-freebsd9.1 --host=x86_64-freebsd9.1 --target=sparc-rtems4.11
374--disable-libstdcxx-pch --with-gnu-as --with-gnu-ld --verbose --with-newlib
375--with-system-zlib --disable-nls --without-included-gettext
376--disable-win32-registry --enable-version-specific-runtime-libs --disable-lto
377--enable-threads --enable-plugin --enable-newlib-io-c99-formats
378--enable-newlib-iconv --enable-languages=c,c++
379Thread model: rtems <3>
380gcc version 4.7.2 20120920 <4>
381 (RTEMS4.11-RSB(cb12e4875c203f794a5cd4b3de36101ff9a88403)-1,gcc-4.7.2/newlib-2.0.0) (GCC)
382-------------------------------------------------------------
383
384<1> The target the compiler is built for.
385<2> The configure options used to build GCC.
386<3> The threading model is always RTEMS. This makes the RTEMS tools difficult
387    for bare metal development more difficult.
388<4> The version string. It contains the Git hash of the RTEMS Source Builder
389    you are using. If your local clone has been modifed that state is also
390    recorded in the version string. The hash allows you to track from a GCC
391    executable back to the original source used to build it.
392
393NOTE: The RTEMS thread model enables specific hooks in GCC so applications
394built with RTEMS tools need the RTEMS runtime to operate correctly. You can use
395RTEMS tools to build bare metal component but it is more difficult than with a
396bare metal tool chain and you need to know what you are doing at a low
397level. The RTEMS Source Builder can build bare metal tool chains.
398
399Installing and Tar Files
400~~~~~~~~~~~~~~~~~~~~~~~~
401
402The RTEMS Source Builder can install the built packages directly and optionally
403it can create a build set tar file or a tar file per package built. The normal
404and default behaviour is to let the RTEMS Source Builder install the tools. The
405source will be downloaded, built, installed and cleaned up.
406
407The tar files are created with the full build prefix present and if you follow
408the examples given in this documentation the path is absolute. This can cause
409problems if you are installing on a host you do not have super user or
410administrator rights on because the prefix path may references part you do not
411have write access too and tar will not extract the files. You can use the
412+--strip-components+ option in tar if your host tar application supports it to
413remove the parts you do not have write access too or you may need to unpack the
414tar file somewhere and copy the file tree from the level you have write access
415from. Embedding the full prefix path in the tar files lets you know what the
416prefix is and is recommended. For example if
417`/home/chris/development/rtems/4.11` is the prefix used you cannot change
418directory to the root (`/`) and install because the `/home` is root access
419only. To install you would:
420
421-------------------------------------------------------------
422$ cd
423$ tar --strip-components=3 -xjf rtems-4.11-sparc-rtems4.11-1.tar.bz2
424-------------------------------------------------------------
425
426A build set tar file is created by adding `--bset-tar-file` option to the
427`sb-set-builder` command.
428
429-------------------------------------------------------------
430$ ../source-builder/sb-set-builder --log=l-sparc.txt \
431         --prefix=$HOME/development/rtems/4.11 \
432         --bset-tar-file <1> 4.11/rtems-sparc
433Source Builder - Set Builder, v0.2.0
434Build Set: 4.11/rtems-sparc
435config: expat-2.1.0-1.cfg
436package: expat-2.1.0-x86_64-freebsd9.1-1
437building: expat-2.1.0-x86_64-freebsd9.1-1
438config: tools/rtems-binutils-2.22-1.cfg
439package: sparc-rtems4.11-binutils-2.22-1
440building: sparc-rtems4.11-binutils-2.22-1
441config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
442package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
443building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
444config: tools/rtems-gdb-7.5.1-1.cfg
445package: sparc-rtems4.11-gdb-7.5.1-1
446building: sparc-rtems4.11-gdb-7.5.1-1
447installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <2>
448installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
449installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
450installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
451tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
452cleaning: expat-2.1.0-x86_64-freebsd9.1-1
453cleaning: sparc-rtems4.11-binutils-2.22-1
454cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
455cleaning: sparc-rtems4.11-gdb-7.5.1-1
456Build Set: Time 0:15:25.92873
457-------------------------------------------------------------
458
459<1> The option to create a build set tar file.
460<2> The installation still happens unless you specify `--no-install`.
461<3> Creating the build set tar file.
462
463You can also suppress installing the files using the `--no-install` option to
464the `sb-set-builder` command. This is usefu if your prefix is not accessiable
465when building Canadian cross compiled tool sets.
466
467-------------------------------------------------------------
468$ ../source-builder/sb-set-builder --log=l-sparc.txt \
469            --prefix=$HOME/development/rtems/4.11 \
470            --bset-tar-file --no-install <1> 4.11/rtems-sparc
471Source Builder - Set Builder, v0.2.0
472Build Set: 4.11/rtems-sparc
473config: expat-2.1.0-1.cfg
474package: expat-2.1.0-x86_64-freebsd9.1-1
475building: expat-2.1.0-x86_64-freebsd9.1-1
476config: tools/rtems-binutils-2.22-1.cfg
477package: sparc-rtems4.11-binutils-2.22-1
478building: sparc-rtems4.11-binutils-2.22-1
479config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
480package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
481building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
482config: tools/rtems-gdb-7.5.1-1.cfg
483package: sparc-rtems4.11-gdb-7.5.1-1
484building: sparc-rtems4.11-gdb-7.5.1-1
485tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <2>
486cleaning: expat-2.1.0-x86_64-freebsd9.1-1
487cleaning: sparc-rtems4.11-binutils-2.22-1
488cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
489cleaning: sparc-rtems4.11-gdb-7.5.1-1
490Build Set: Time 0:14:11.721274
491$ ls tar
492rtems-4.11-sparc-rtems4.11-1.tar.bz2
493-------------------------------------------------------------
494
495<1> The option to supressing installing the packages.
496<2> Create the build set tar.
497
498A package tar file can be created by adding the +--pkg-tar-files+ to the
499+sb-set-builder+ command. This creates a tar file per package built in the
500build set.
501
502-------------------------------------------------------------
503$ ../source-builder/sb-set-builder --log=l-sparc.txt \
504            --prefix=$HOME/development/rtems/4.11 \
505            --bset-tar-file --pkg-tar-files <1> --no-install 4.11/rtems-sparc
506Source Builder - Set Builder, v0.2.0
507Build Set: 4.11/rtems-sparc
508config: expat-2.1.0-1.cfg
509package: expat-2.1.0-x86_64-freebsd9.1-1
510building: expat-2.1.0-x86_64-freebsd9.1-1
511config: tools/rtems-binutils-2.22-1.cfg
512package: sparc-rtems4.11-binutils-2.22-1
513building: sparc-rtems4.11-binutils-2.22-1
514config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
515package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
516building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
517config: tools/rtems-gdb-7.5.1-1.cfg
518package: sparc-rtems4.11-gdb-7.5.1-1
519building: sparc-rtems4.11-gdb-7.5.1-1
520tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2
521cleaning: expat-2.1.0-x86_64-freebsd9.1-1
522cleaning: sparc-rtems4.11-binutils-2.22-1
523cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
524cleaning: sparc-rtems4.11-gdb-7.5.1-1
525Build Set: Time 0:14:37.658460
526$ ls tar
527expat-2.1.0-x86_64-freebsd9.1-1.tar.bz2           sparc-rtems4.11-binutils-2.22-1.tar.bz2
528sparc-rtems4.11-gdb-7.5.1-1.tar.bz2 <2>           rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
529sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1.tar.bz2
530-------------------------------------------------------------
531
532<1> The option to create packages tar files.
533<2> The GDB package tar file.
534<3> The build set tar file. All the others in a single tar file.
535
536Controlling the Build
537~~~~~~~~~~~~~~~~~~~~~
538
539Build sets can be controlled via the command line to enable and disable various
540features. There is no definitive list of build options that can be listed
541because they are implemented with the configuration scripts. The best way to
542find what is available is to grep the configuration files. for +with+ and
543+without+.
544
545Following are currentlt available:
546
547'--without-rtems':: Do not build RTEMS when building an RTEMS build set.
548'--without-cxx':: Do not build a C++ compiler.
549'--with-objc':: Attempt to build a C++ compiler.
550'--with-fortran':: Attempt to build a Fortran compiler.
551
552Why Build from Source ?
553-----------------------
554
555The RTEMS Source Builder is not a replacement for the binary install systems
556you have with commercial operating systems or open source operating system
557distributions. Those products and distributions are critically important and
558are the base that allows the Source Builder to work. The RTEMS Source Builder
559sits somewhere between you manually entering the commands to build a tool set
560and a tool such as +yum+ or +apt-get+ to install binary packages made
561specifically for your host operating system. Building manually or installing a
562binary package from a remote repository are valid and real alternatives while
563the Source Builder is attempting to provide a specific service of repeatably
564being able to build tool sets from source code.
565
566If you are developing a system or product that has a long shelf life or is used
567in a critical piece of infrastructure that has a long life cycle being able to
568build from source is important. It insulates the project from the fast ever
569changing world of the host development machines. If your tool set is binary and
570you have lost the ability to build it you have lost a degree of control and
571flexibility open source gives you. Fast moving host environments are
572fantastic. We have powerful multi-core computers with huge amounts of memory
573and state of the art operating systems to run on them however the product or
574project you are part of may need to be maintained well past the life time of
575these host. Being able to build from source an important and critical part of
576this process because you can move to a newer host and create an equivalent tool
577set.
578
579Building from source provides you with control over the configuration of the
580package you are building. If all or the most important dependent parts are
581built from source you limit the exposure to host variations. For example the
582GNU C compiler (gcc) currently uses a number of 3rd party libraries internally
583(gmp, mpfr, etc). If your validated compiler generating code for your target
584processor is dynamically linked against the host's version of these libraries
585any change in the host's configuration may effect you. The changes the host's
586package management system makes may be perfectly reasonable in relation to the
587distribution being managed however this may not extend to you and your
588tools. Building your tools from source and controlling the specific version of
589these dependent parts means you are not exposing yourself to unexpected and
590often difficult to resolve problems. On the other side you need to make sure
591your tools build and work with newer versions of the host operating
592system. Given the stability of standards based libraries like 'libc' and ever
593improving support for standard header file locations this task is becoming
594easier.
595
596The RTEMS Source Builder is designed to be audited and incorporated into a
597project's verification and validation process. If your project is developing
598critical applications that needs to be traced from source to executable code in
599the target, you need to also consider the tools and how to track them.
600
601If your IT department maintains all your computers and you do not have suitable
602rights to install binary packages, building from source lets you create your
603own tool set that you install under your home directory. Avoiding installing
604any extra packages as a super user is always helpful in maintaining a secure
605computing environment.
606
607[[_bugs]]
608Bugs, Crashes, and Build Failures
609---------------------------------
610
611The RTEMS Source Builder is a Python program and every care is taken to test
612the code however bugs, crashes, and build failures can and do happen. If you
613find a bug please report it via the RTEMS Bug tracker tool Bugzilla or via
614email on the RTEMS Users list. RTEMS's bugzilla is found at
615https://www.rtems.org/bugzilla/.
616
617Please include the generated RSB report. If you see the following a report has
618been generated:
619
620-------------------------------------------------------------
621 ...
622 ...
623Build FAILED <1>
624  See error report: rsb-report-4.11-rtems-lm32.txt <2>
625-------------------------------------------------------------
626<1> The build has failed.
627<2> The report's file name.
628
629The generated report contains the command line, version of the RSB, your host
630+uname+ details, the version of Python and the last 200 lines of the log.
631
632If there is no report please send for some reason and something has falied
633please report the following:
634
635. Command line,
636. The git hash,
637. Host details with the output of the +uname -a+ command,
638. If you have made any modifications.
639
640If there is a Python crash please cut and paste the Python backtrace into the
641bug report. If the tools fail to build please locate the first error in the log
642file. This can be difficult to find on hosts with many cores so it sometimes
643pays to run the command with the +--jobs=none+ option to get a log that is
644correctly sequenced. If searching the log file seach for +error:+ and the error
645should be just above it.
646
647[[_contributing]]
648Contributing
649------------
650
651The RSB is open source and open to contributions. These can be bug fixes, new
652features or new configurations. Please break patches down into changes to the
653core Python code, configuration changes or new configurations.
654
655Please email me patches via git so I can maintain your commit messages so you
656are acknowledged as the contributor.
657
658Packages
659~~~~~~~~
660
661We welcome users adding, fixing, updating and upgrading packages and their
662configurations. The RSB contains the 'bare' configuration tree and you can use
663this to add packages you use on the hosts. For example 'qemu' is support on a
664range of hosts. RTEMS tools live in the +rtems/config+ directory tree.
665
666Project Sets
667------------
668
669The RTEMS Source Builder supports project configurations. Project
670configurations can be public or private and can be contained in the RTEMS
671Source Builder project if suitable, other projects they use the RTEM Source
672Builder or privately on your local file system.
673
674The configuration file loader searches the macro +_configdir+ and by default
675this is set to +%{\_topdir}/config:%{\_sbdir}/config+ where +_topdir+ is the
676your current working direct, in other words the directory you invoke the RTEMS
677Source Builder command in, and +_sbdir+ is the directory where the RTEMS Source
678Builder command resides. Therefore the +config+ directory under each of these
679is searched so all you need to do is create a +config+ in your project and add
680your configuration files. They do not need to be under the RTEMS Source Builder
681source tree. Public projects are included in the main RTEMS Source Builder such
682as RTEMS.
683
684You can also add your own +patches+ directory next to your +config+ directory
685as the +%patch+ command searches the +_patchdir+ macro variable and it is
686by default set to +%{\_topdir}/patches:%{\_sbdir}/patches+.
687
688The +source-builder/config+ directory provides generic scripts for building
689various tools. You can specialise these in your private configurations to make
690use of them. If you add new generic configurations please contribute them back
691to the project
692
693RTEMS Configurations
694~~~~~~~~~~~~~~~~~~~~
695
696The RTEMS Configurations are grouped by RTEMS version. In RTEMS the tools are
697specific to a specific version because of variations between Newlib and
698RTEMS. Restructuring in RTEMS and Newlib sometimes moves _libc_ functionality
699between these two parts and this makes existing tools incompatible with RTEMS.
700
701RTEMS allows architectures to have different tool versions and patches. The
702large number of architectures RTEMS supports can make it difficult to get a
703common stable version of all the packages. An architecture may require a recent
704GCC because an existing bug has been fixed, however the more recent version may
705have a bug in other architecture. Architecture specific patches should be
706limited to the architecture it relates to. The patch may fix a problem on the
707effect architecture however it could introduce a problem in another
708architecture. Limit exposure limits any possible crosstalk between
709architectures.
710
711RTEMS supports _stable_ and _unstable_. Stable configurations are fixed while
712unstable configurations are supporting using snapshots of user macros and
713reference release candidates or source extracted directly from version
714control. The stable build sets are referenced as +<version>/rtems-<arch>+ and
715include `autoconf` and `automake`.
716
717If you are building a released version of RTEMS the release RTEMS tar file will
718be downloaded and built as part of the build process. If you are building a
719tool set for use with the development branch of RTEMS, the development branch
720will be cloned directly from the RTEMS GIT repository and built.
721
722When building RTEMS within the RTEMS Source Builder it needs a suitable working
723`autoconf` and `automake`. These packages need to built and installed in their
724prefix in order for them to work. The RTEMS Source Builder installs all
725packages only after they have been built so if you host does not have a
726recent enough version of `autoconf` and `automake` you first need to build them
727and install them then build your tool set. The commands are:
728
729-------------------------------------------------------------
730$ ../source-builder/sb-set-builder --log=l-4.11-at.txt \
731   --prefix=$HOME/development/rtems/4.11 4.11/rtems-autotools
732$ export PATH=~/development/rtems/4.11/bin:$PATH <1>
733$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
734   --prefix=$HOME/development/rtems/4.11 4.11/rtems-sparc
735-------------------------------------------------------------
736<1> Setting the path.
737
738TIP: If this is your first time building the tools and RTEMS it pays to add the
739`--dry-run` option. This will run through all the configuration files and if
740any checks fail you will see this quickly rather than waiting for until the
741build fails a check.
742
743To build snapshots for testing purposes you use the available macro maps
744passing them on the command line using the `--macros` option. For RTEMS these
745are held in `config/snapshots` directory. The following builds _newlib_ from
746CVS:
747
748-------------------------------------------------------------
749$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
750   --prefix=$HOME/development/rtems/4.11 \
751   --macros=snapshots/newlib-head.mc \
752   4.11/rtems-sparc
753-------------------------------------------------------------
754
755and the following uses the version control heads for _binutils_, _gcc_,
756_newlib_, _gdb_ and _RTEMS_:
757
758-------------------------------------------------------------
759$ ../source-builder/sb-set-builder --log=l-heads-sparc.txt \
760   --prefix=$HOME/development/rtems/4.11-head \
761   --macros=snapshots/binutils-gcc-newlib-gdb-head.mc \
762   4.11/rtems-sparc
763-------------------------------------------------------------
764
765Patches
766~~~~~~~
767
768Packages being built by the RSB need patches from time to time and the RSB
769supports patching upstream packages. The patches are held in a seperate
770directory called +patches+ relative to the configuration directory you are
771building. For example +%{\_topdir}/patches:%{\_sbdir}/patches+. Patches are
772declared in the configuraiton files in a similar manner to the package's source
773so please refer to the +%source+ documentation. Patches, like the source, are
774to be made publically available for configurations that live in the RSB package
775and are downloaded on demand.
776
777If a package has a patch management tool it is recommended you reference the
778package's patch management tools directly. If the RSB does not support the
779specific patch manage tool please contact the mailing list to see if support
780can be added.
781
782Patches for packages developed by the RTEMS project can be placed in the RTEMS
783Tools Git repository. The +tools+ directory in the repository has various
784places a patch can live. The tree is broken down in RTEMS releases and then
785tools within that release. If the package is not specific to any release the
786patch can be added closer to the top under the package's name. Patches to fix
787specific tool related issues for a specific architecture should be grouped
788under the specific architecture and only applied when building that
789architecture avoiding a patch breaking an uneffected architecture.
790
791Patches in the RTEMS Tools repository need to be submitted to the upstream
792project. It should not be a clearing house for patches that will not be
793accepted upstream.
794
795Patches are added to a component's name and in the +%prep:+ section the patches
796can be set up, meaning they are applied to source. The patches are applied in
797the order they are added. If there is a dependency make sure you order the
798patches correctly when you add them. You can add any number of patches and the
799RSB will handle them efficently.
800
801Patches can have options. These are added before the patch URL. If no options
802are provided the patch's setup default options are used.
803
804Patches can be declared in build set up files.
805
806This examples shows how to declare a patch for gdb in the +lm32+ architecture:
807
808-------------------------------------------------------------
809%patch add <1> gdb <2> %{rtems_gdb_patches}/lm32/gdb-sim-lm32uart.diff <3>
810-------------------------------------------------------------
811<1> The patch's +add+ command.
812<2> The group of patches this patch belongs too.
813<3> The patch's URL. It is downloaded from here.
814
815The patches are applied when a patch +setup+ command is issued in the +%prep:+
816section. All patches in the group are applied. To apply the GDB patch above
817use:
818
819-------------------------------------------------------------
820%patch setup <1> gdb <2> -p1 <3>
821-------------------------------------------------------------
822<1> The patch's +setup+ command.
823<2> The group of patches to apply.
824<3> The patch group's default options. If no option is given with the patch
825these options are used.
826
827Architecture specific patches live in the architecture build set file isolating
828the patch to that specific architecture. If a patch is common to a tool it
829resides in the RTEMS tools configuration file. Do not place patches for tools
830in the +source-builder/config+ template configuration files.
831
832To test a patch simply copy it to your local +patches+ directory. The RSB will
833see the patch is present and will not attempt to download it. Once you are
834happy with the patch submit it to the project and a core developer will review
835it and add it to the RTEMS Tools git repository.
836
837Cross Building
838--------------
839
840Cross building or Canadian cross compiling is the process of building a set of
841RTEMS tools for one type of host on another host. The RSB is a source builder
842so the need for this is not often called on because you can just build the
843tools for the host you wish to run on. There are however a few special cases
844where this is needed. The host may not have a stable reliable means of building
845the tools, such as Windows, or you wish to manage the configuration of tools on
846different hosts from a single environment because of auditing or verfication
847reasons.
848
849Canadian Cross Building
850~~~~~~~~~~~~~~~~~~~~~~~
851
852The process of building cross compilers on one host for another host is
853commonly referred to as a Canadian cross build. The process is controlled by
854setting the build triplet to the host you are building, the host triplet to the
855host the tools will run on and the target to the RTEMS architecture you
856require. The tools needed by the RSB are:
857
858. Build host C and C++ compiler
859. Host C and C++ cross compiler
860
861The RTEMS Source Builder requires you provide the build host C and C\++
862compiler and the final host C and C\++ cross-compiler. The RSB will build the
863build host RTEMS compiler and the final host RTEMS C and C++ compiler, the
864output of this process.
865
866The Host C and C++ compiler is a cross-compiler that builds executables for
867the host you want the tools for. You need to provide these tools. For Windows a
868number of Unix operating systems provide MinGW tool sets as packages.
869
870The RSB will build an RTEMS tool set for the build host. This is needed when
871building the final host's RTEMS compiler as it needs to build RTEMS runtime
872code such as _libc_ on the build host.
873
874TIP: Make sure the host's cross-compiler tools are in your path before run the
875RSB build command.
876
877TIP: Canadian Cross built tools will not run on the machine being used to build
878them so you should provide the +--bset-tar-files+ and +--no-install+
879options. The option to not install the files lets you provide a prefix that
880does not exist or you cannot access.
881
882Command Line
883~~~~~~~~~~~~
884
885To perform a cross build add +--host=+ to the command line. For example to
886build a MinGW tool set on FreeBSD for Windows add +--host=mingw32+ if the cross
887compiler is +mingw32-gcc+:
888
889-------------------------------------------------------------
890$ ../source-builder/sb-set-builder --host=mingw32 \
891   --log=l-mingw32-4.11-sparc.txt \
892   --prefix=$HOME/development/rtems/4.11 \
893   4.11/rtems-sparc
894-------------------------------------------------------------
895
896If you are on a Linux Fedora build host with the MinGW packages installed the
897command line is:
898
899-------------------------------------------------------------
900$ ../source-builder/sb-set-builder --host=i686-w64-mingw32 \
901   --log=l-mingw32-4.11-sparc.txt \
902   --prefix=$HOME/development/rtems/4.11 \
903   4.11/rtems-sparc
904-------------------------------------------------------------
905
906Configuration
907-------------
908
909The RTEMS Source Builder has two types of configuration data:
910
911. Build Sets
912. Package Build Configurations
913
914By default these files can be located in two separate directories and
915searched. The first directory is +config+ in your current working directory
916(+_topdir+) and the second is +config+ located in the base directory of the
917RTEMS Source Builder command you run (+_sbdir+). The RTEMS directory +rtems+
918located at the top of the RTEMS Source Builder source code is an example of a
919specific build configuration directory. You can create custom or private build
920configurations and if you run the RTEMS Source Builder command from that
921directory your configurations will be used.
922
923[[X1]] The configuration search path is a macro variable and is reference as
924+%\{_configdir\}+. It's default is defined as:
925
926-------------------------------------------------------------
927_configdir          : dir      optional   %{_topdir}/config:%{_sbdir}/config <1>
928-------------------------------------------------------------
929
930<1> The +_topdir+ is the directory you run the command from and +_sbdir+ is the
931location of the RTEMS Source Builder command.
932
933Build set files have the file extension +.bset+ and the package build
934configuration files have the file extension of +.cfg+. The +sb-set-builder+
935command will search for _build sets_ and the +sb-builder+ commands works with
936package build configuration files.
937
938Both types of configuration files use the \'#' character as a comment
939character. Anything after this character on the line is ignored. There is no
940block comment.
941
942Source and Patches
943~~~~~~~~~~~~~~~~~~
944
945The RTEMS Source Builder provides a flexible way to manage source. Source and
946patches are declare in configurations file using the +source+ and +patch+
947directives. These are a single line containing a Universal Resource Location or
948URL and can contain macros and shell expansions. The <<_prep,%prep>> section
949details the source and patch directives
950
951The URL can reference remote and local source and patch resources. The
952following schemes are provided:
953
954'http':: Remote access using the HTTP protocol.
955'https':: Remote access using the Secure HTTP protocol.
956'ftp':: Remote access using the FTP protocol.
957'git':: Remote access to a GIT repository.
958'cvs':: Remote access to a CVS repository.
959'pm':: Remote access to a patch management repository.
960'file':: Local access to an existing source directory.
961
962HTTP, HTTPS, and FTP
963^^^^^^^^^^^^^^^^^^^^
964
965Remote access to TAR or ZIP files is provided using HTTP, HTTPS and FTP
966protocols. The full URL provided is used to access the remote file including
967any query components. The URL is parsed to extract the file component and the
968local source directory is checked for that file. If the file is located locally
969the remote file is not downloaded. Currently no other checks are made. If a
970download fails you need to manually remove the file from the source directory
971and start the build process again.
972
973The URL can contain macros. These are expanded before issuing the request to
974download the file. The standard GNU GCC compiler source URL is:
975
976-------------------------------------------------------------
977%source set<1> gcc<2> ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
978-------------------------------------------------------------
979<1> The +%source+ command's set command sets the source. The first is set and
980following sets are ignored.
981<2> The source is part of the +gcc+ group.
982
983The type of compression is automatically detected from the file extension. The
984supported compression formats are:
985
986'gz':: GNU ZIP
987'bzip2':: BZIP2
988'zip':: ZIP
989'xy':: XY
990
991The output of the decompression tool is feed to the standard `tar` utility if
992not a ZIP file and unpacked into the build directory. ZIP files are unpacked by
993the decompression tool and all other files must be in the tar file format.
994
995The +%source+ directive typically supports a single source file tar or zip
996file. The +set+ command is used to set the URL for a specific source group. The
997first set command encoutner is registered and any further set commands are
998ignored. This allows you to define a base standard source location and override
999it in build and architecture specific files. You can also add extra source
1000files to a group. This is typically done when a collection of source is broken
1001down in a number of smaller files and you require the full package. The
1002source's +setup+ command must reide in the +%prep:+ section and it unpacks the
1003source code ready to be built.
1004
1005If the source URL references the GitHub API server 'https://api.github.com/' a
1006tarball of the specified version is download. For example the URL for the
1007STLINK project on GitHub and version is:
1008
1009-------------------------------------------------------------
1010%define stlink_version 3494c11
1011%source set stlink https://api.github.com/repos/texane/stlink/texane-stlink-%{stlink_version}.tar.gz
1012-------------------------------------------------------------
1013
1014GIT
1015^^^
1016
1017A GIT repository can be cloned and used as source. The GIT repository resides
1018in the 'source' directory under the `git` directory. You can edit, update and
1019use the repository as you normally do and the results will used to build the
1020tools. This allows you to prepare and test patches in the build environment the
1021tools are built in. The GIT URL only supports the GIT protocol. You can control
1022the repository via the URL by appending options and arguments to the GIT
1023path. The options are delimited by `?` and option arguments are delimited from
1024the options with `=`. The options are:
1025
1026`branch`:: Checkout the specified branch.
1027`pull`:: Perform a pull to update the repository.
1028`fetch`:: Perform a fetch to get any remote updates.
1029`reset`:: Reset the repository. Useful to remove any local changes. You can
1030pass the `hard` argument to force a hard reset.
1031
1032-------------------------------------------------------------
1033%source set gcc git://gcc.gnu.org/git/gcc.git?branch=gcc-4_7-branch?reset=hard
1034-------------------------------------------------------------
1035
1036This will clone the GCC git repository and checkout the 4.7-branch and perform
1037a hard reset. You can select specific branches and apply patches. The
1038repository is cleaned up before each build to avoid various version control
1039errors that can arise.
1040
1041CVS
1042^^^
1043
1044A CVS repository can be checked out. CVS is more complex than GIT to handle
1045because of the modules support. This can effect the paths the source ends up
1046in. The CVS URL only supports the CVS protocol. You can control the repository
1047via the URL by appending options and arguments to the CVS path. The options are
1048delimited by `?` and option arguments are delimited from the options with
1049`=`. The options are:
1050
1051`module`:: The module to checkout.
1052`src-prefix`:: The path into the source where the module starts.
1053`tag`:: The CVS tag to checkout.
1054`date`:: The CVS date to checkout.
1055
1056-------------------------------------------------------------
1057%source set newlib cvs://pserver:anoncvs@sourceware.org/cvs/src?module=newlib?src-prefix=src
1058-------------------------------------------------------------
1059
1060Macros and Defaults
1061~~~~~~~~~~~~~~~~~~~
1062
1063The RTEMS Source Builder uses tables of _macros_ read in when the tool
1064runs. The initial global set of macros is called the _defaults_. These values
1065are read from a file called `defaults.mc` and modified to suite your host. This
1066host specific adaption lets the Source Builder handle differences in the build
1067hosts.
1068
1069Build set and configuration files can define new values updating and extending
1070the global macro table. For example builds are given a release number. This is
1071typically a single number at the end of the package name. For example:
1072
1073-------------------------------------------------------------
1074%define release 1
1075-------------------------------------------------------------
1076
1077Once defined if can be accessed in a build set or package configuration file
1078with:
1079
1080-------------------------------------------------------------
1081%{release}
1082-------------------------------------------------------------
1083
1084The +sb-defaults+ command lists the defaults for your host. I will not include
1085the output of this command because of its size.
1086
1087-------------------------------------------------------------
1088$ ../source-builder/sb-defaults
1089-------------------------------------------------------------
1090
1091A nested build set is given a separate copy of the global macro maps. Changes
1092in one change set are not seen in other build sets. That same happens with
1093configuration files unless inline includes are used. Inline includes are seen
1094as part of the same build set and configuration and changes are global to that
1095build set and configuration.
1096
1097Macro Maps and Files
1098^^^^^^^^^^^^^^^^^^^^
1099
1100Macros are read in from files when the tool starts. The default settings are
1101read from the defaults macro file called `defaults.mc` located in the top level
1102RTEMS Source Builder command directory. User macros can be read in at start up
1103by using the `--macros` command line option.
1104
1105The format for a macro in macro files is:
1106
1107[options="header,compact",width="50%",cols="15%,15%,15%,55%"]
1108|=================================
1109| Name | Type | Attribute | String
1110|=================================
1111
1112where 'Name' is a case insensitive macro name, the 'Type' field is:
1113
1114[horizontal]
1115`none`:: Nothing, ignore.
1116`dir`:: A directory path.
1117`exe`:: An executable path.
1118`triplet`:: A GNU style architecture, platform, operating system string.
1119
1120the 'Attribute' field is:
1121
1122[horizontal]
1123`none`:: Nothing, ignore
1124`required`:: The host check must find the executable or path.
1125`optional`:: The host check generates a warning if not found.
1126`override`:: Only valid outside of the `global` map to indicate this macro
1127             overrides the same one in the `global` map when the map containing
1128             it is selected.
1129`undefine`:: Only valid outside of the `global` map to undefine the macro if it
1130             exists in the `global` map when the map containing it is
1131             selected. The `global` map's macro is not visible but still
1132             exists.
1133
1134and the 'String' field is a single or tripled multiline quoted string. The
1135'String' can contain references to other macros. Macro that loop are not
1136currently detected and will cause the tool to lock up.
1137
1138Maps are declared anywhere in the map using the map directive:
1139
1140-------------------------------------------------------------
1141# Comments
1142[my-special-map] <1>
1143_host:  none, override, 'abc-xyz'
1144multiline: none, override, '''First line,
1145second line,
1146and finally the last line'''
1147-------------------------------------------------------------
1148<1> The map is set to `my-special-map`.
1149
1150Any macro defintions following a map declaration are placed in that map and the
1151default map is `global` when loading a file. Maps are selected in configuration
1152files by using the `%select` directive.
1153
1154-------------------------------------------------------------
1155%select my-special-map
1156-------------------------------------------------------------
1157
1158Selecting a map means all requests for a macro first check the selected map and
1159if present return that value else the `global` map is used. Any new macros or
1160changes update only the `global` map. This may change in future releases so
1161please make sure you use the `override` attribute.
1162
1163The macro files specificed on the command line are looked for in the
1164`_configdir` paths. See <<X1,+_configdir+>> variable for details. Included
1165files need to add the `%{_configdir}` macro to the start of the file.
1166
1167Macro map files can include other macro map files using the `%include`
1168directive. The macro map to build _binutils_, _gcc_, _newlib_, _gdb_ and
1169_RTEMS_ from version control heads is:
1170
1171-------------------------------------------------------------
1172# <1>
1173# Build all tool parts from version control head.
1174#
1175%include %{_configdir}/snapshots/binutils-head.mc
1176%include %{_configdir}/snapshots/gcc-head.mc
1177%include %{_configdir}/snapshots/newlib-head.mc
1178%include %{_configdir}/snapshots/gdb-head.mc
1179-------------------------------------------------------------
1180<1> The file is `config/snapshots/binutils-gcc-newlib-gdb-head.mc`.
1181
1182The macro map defaults to `global` at the start of each included file and the
1183map setting of the macro file including the other macro files does not change.
1184
1185Personal Macros
1186^^^^^^^^^^^^^^^
1187
1188When the tools start to run they will load personal macros. Personal macros are
1189in the standard format for macros in a file. There are two places personal
1190macros can be configured. The first is the environment variable
1191`RSB_MACROS`. If present the macros from the file the environment variable
1192points to are loaded. The second is a file called `.rsb_macros` in your home
1193directory. You need to have the environment variable `HOME` defined for this
1194work.
1195
1196Report Mailing
1197~~~~~~~~~~~~~~
1198
1199The build reports can be mailed to a specific email address to logging and
1200monitoring. Mailing requires a number of parameters to function. These are:
1201
1202. To mail address
1203. From mail address
1204. SMTP host
1205
1206.To Mail Address
1207
1208The +to+ mail address is taken from the macro `%{_mail_tools_to}` and the
1209default is _rtems-tooltestresults at rtems.org_. You can override the default
1210with a personal or user macro file or via the command line option _--mail-to_.
1211
1212.From Mail Address
1213
1214The +from+ mail address is taken from:
1215
1216. GIT configuration
1217. User `.mailrc` file
1218. Command line
1219
1220If you have configured an email and name in git it will be used used. If you do
1221not a check is made for a `.mailrc` file. The environment variable _MAILRC_ is
1222used if present else your home directory is check. If found the file is scanned
1223for the `from` setting:
1224
1225  set from="Foo Bar <foo@bar>"
1226
1227You can also support a from address on the command line with the _--mail-from_
1228option.
1229
1230.SMTP Host
1231
1232The SMTP host is taken from the macro `%{_mail_smtp_host}` and the default is
1233`localhost`. You can override the default with a personal or user macro file or
1234via the command line option _--smtp-host_.
1235
1236Build Set Files
1237~~~~~~~~~~~~~~~
1238
1239Build set files lets you list the packages in the build set you are defining
1240and have a file extension of +.bset+. Build sets can define macro variables,
1241inline include other files and reference other build set or package
1242configuration files.
1243
1244Defining macros is performed with the +%define+ macro:
1245
1246-------------------------------------------------------------
1247%define _target m32r-rtems4.11
1248-------------------------------------------------------------
1249
1250Inline including another file with the +%include+ macro continues processing
1251with the specified file returning to carry on from just after the include
1252point.
1253
1254-------------------------------------------------------------
1255%include rtems-4.11-base.bset
1256-------------------------------------------------------------
1257
1258This includes the RTEMS 4.11 base set of defines and checks. The configuration
1259paths as defined by +_configdir+ are scanned. The file extension is optional.
1260
1261You reference build set or package configuration files by placing the file name
1262on a single line.
1263
1264-------------------------------------------------------------
1265tools/rtems-binutils-2.22-1
1266-------------------------------------------------------------
1267
1268The +_configdir+ path is scanned for +tools/rtems-binutils-2.22-1.bset+ or
1269+tools/rtems-binutils-2.22-1.cfg+. Build set files take precedent over package
1270configuration files. If +tools/rtems-binutils-2.22-1+ is a build set a new
1271instance of the build set processor is created and if the file is a package
1272configuration the package is built with the package builder. This all happens
1273once the build set file has finished being scanned.
1274
1275Configuration Control
1276~~~~~~~~~~~~~~~~~~~~~
1277
1278The RTEMS Souce Builder is designed to fit within most verification and
1279validation processes. All of the RTEMS Source Builder is source code. The
1280Python code is source and comes with a commercial friendly license. All
1281configuration data is text and can be read or parsed with standard text based
1282tools.
1283
1284File naming provides configuration management. A specific version of a package
1285is captured in a specific set of configuration files. The top level
1286configuration file referenced in a _build set_ or passed to the +sb-builder+
1287command relates to a specific configuration of the package being built. For
1288example the RTEMS configuration file +rtems-gcc-4.7.2-newlib-2.0.0-1.cfg+
1289creates an RTEMS GCC and Newlib package where the GCC version is 4.7.2, the
1290Newlib version is 2.0.0, plus any RTEMS specific patches that related to this
1291version. The configuration defines the version numbers of the various parts
1292that make up this package:
1293
1294-------------------------------------------------------------
1295%define gcc_version    4.7.2
1296%define newlib_version 2.0.0
1297%define mpfr_version   3.0.1
1298%define mpc_version    0.8.2
1299%define gmp_version    5.0.5
1300-------------------------------------------------------------
1301
1302The package build options, if there are any are also defined:
1303
1304-------------------------------------------------------------
1305%define with_threads 1
1306%define with_plugin  0
1307%define with_iconv   1
1308-------------------------------------------------------------
1309
1310The generic configuration may provide defaults in case options are not
1311specified. The patches this specific version of the package requires can be
1312included:
1313
1314-------------------------------------------------------------
1315Patch0: gcc-4.7.2-rtems4.11-20121026.diff
1316-------------------------------------------------------------
1317
1318Finally including the GCC 4.7 configuration script:
1319
1320-------------------------------------------------------------
1321%include %{_configdir}/gcc-4.7-1.cfg
1322-------------------------------------------------------------
1323
1324The +gcc-4.7-1.cfg+ file is a generic script to build a GCC 4.7 compiler with
1325Newlib. It is not specific to RTEMS. A bare no operating system tool set can be
1326built with this file.
1327
1328The +-1+ part of the file names is a revision. The GCC 4.7 script maybe revised
1329to fix a problem and if this fix effects an existing script the file is copied
1330and given a +-2+ revision number. Any dependent scripts referencing the earlier
1331revision number will not be effected by the change. This locks down a specific
1332configuration over time.
1333
1334Personal Configurations
1335~~~~~~~~~~~~~~~~~~~~~~~
1336
1337The RSB supports personal configurations. You can view the RTEMS support in the
1338+rtems+ directory as a private configuration tree that resides within the RSB
1339source. There is also the +bare+ set of configurations. You can create your own
1340configurations away from the RSB source tree yet use all that the RSB provides.
1341
1342To create a private configuration change to a suitable directory:
1343
1344-------------------------------------------------------------
1345$ cd ~/work
1346$ mkdir test
1347$ cd test
1348$ mkdir config
1349-------------------------------------------------------------
1350
1351and create a +config+ directory. Here you can add a new configuration or build
1352set file. The section 'Adding New Configurations' details how to add a new
1353confguration.
1354
1355New Configurations
1356~~~~~~~~~~~~~~~~~~
1357
1358This section describes how to add a new configuration to the RSB. We will add a
1359configuration to build the Device Tree Compiler. The Device Tree Compiler or
1360DTC is part of the Flattened Device Tree project and compiles Device Tree
1361Source (DTS) files into Device Tree Blobs (DTB). DTB files can be loaded by
1362operating systems and used to locate the various resources such as base
1363addresses of devices or interrupt numbers allocated to devices. The Device Tree
1364Compiler source code can be downloaded from http://www.jdl.com/software. The
1365DTC is supported in the RSB and you can find the configuration files under the
1366+bare/config+ tree. I suggest you have a brief look over these files.
1367
1368Layering by Including
1369^^^^^^^^^^^^^^^^^^^^^
1370
1371Configurations can be layered using the +%include+ directive. The user invokes
1372the outer layers which include inner layers until all the required
1373configuration is present and the package can be built. The outer layers can
1374provide high level details such as the version and the release and the inner
1375layers provide generic configuration details that do not change from one
1376release to another. Macro variables are used to provide the specific
1377configuration details.
1378
1379Configuration File Numbering
1380^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1381
1382Configuration files have a number at the end. This is a release number for that
1383configuration and it gives us the ability to track a specific configuration for
1384a specific version. For example lets say the developers of the DTC package
1385change the build system from a single makefile to autoconf and automake between
1386version 1.3.0 and version 1.4.0. The configuration file used to build the
1387package would change have to change. If we did not number the configuration
1388files the ability to build 1.1.0, 1.2.0 or 1.3.0 would be lost if we update a
1389common configuration file to build an autoconf and automake version. For
1390version 1.2.0 the same build script can be used so we can share the same
1391configuration file between version 1.1.0 and version 1.2.0. An update to any
1392previous release lets us still build the package.
1393
1394Common Configuration Scripts
1395^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1396
1397Common configuration scripts that are independent of version, platform and
1398architecture are useful to everyone. These live in the Source Builder's
1399configuration directory. Currently there are scripts to build binutils, expat,
1400DTC, GCC, GDB and libusb. These files contain the recipes to build these
1401package without the specific details of the versions or patches being
1402built. They expect to be wrapped by a configuration file that ties the package
1403to a specific version and optionally specific patches.
1404
1405DTC Example
1406^^^^^^^^^^^
1407
1408We will be building the DTC for your host rather than a package for RTEMS. We
1409will create a file called +source-builder/config/dtc-1-1.cfg+. This is a common
1410script that can be used to build a specific version using a general recipe. The
1411file name is 'dtc-1-1.cfg' where the 'cfg' extension indicates this is a
1412configuration file. The first *1* says this is for the major release 1 of the
1413package and the last *1* is the build configuration version.
1414
1415The file starts with some comments that detail the configuration. If there is
1416anything unusual about the configuration it is a good idea to add something in
1417the comments here. The comments are followed by a check for the release. In
1418this case if a release is not provided a default of 1 is used.
1419
1420-------------------------------------------------------------
1421#
1422# DTC 1.x.x Version 1.
1423#
1424# This configuration file configure's, make's and install's DTC.
1425#
1426
1427%if %{release} == %{nil}
1428%define release 1
1429%endif
1430-------------------------------------------------------------
1431
1432The next section defines some information about the package. It does not effect
1433the build and is used to annotate the reports. It is recommended this
1434information is kept updated and accurate.
1435
1436-------------------------------------------------------------
1437Name:      dtc-%{dtc_version}-%{_host}-%{release}
1438Summary:   Device Tree Compiler v%{dtc_version} for target %{_target} on host %{_host}
1439Version:   %{dtc_version}
1440Release:   %{release}
1441URL:       http://www.jdl.com/software/
1442BuildRoot: %{_tmppath}/%{name}-root-%(%{__id_u} -n)
1443-------------------------------------------------------------
1444
1445The next section defines the source and any patches. In this case there is a
1446single source package and it can be downloaded using the HTTP protocol. The RSB
1447knows this is GZip'ped tar file. If more than one package package is needed add
1448them increasing the index. The +gcc-4.8-1.cfg+ configuration contains examples
1449of more than one source package as well as conditionally including source
1450packages based on the outer configuration options.
1451
1452-------------------------------------------------------------
1453#
1454# Source
1455#
1456%source set dtc http://www.jdl.com/software/dtc-v%{dtc_version}.tgz
1457-------------------------------------------------------------
1458
1459The remainder of the script is broken in to the various phases of a build. They
1460are:
1461
1462. Preperation
1463. Bulding
1464. Installing, and
1465. Cleaning
1466
1467Preparation is the unpacking of the source, applying any patches as well as any
1468package specific set ups. This part of the script is a standard Unix shell
1469script. Be careful with the use of '%' and '$'. The RSB uses '%' while the
1470shell scripts use '$'.
1471
1472A standard pattern you will observe is the saving of the build's top
1473directory. This is used instead of changing into a subdirectory and then
1474changing to the parent when finished. Some hosts will change in a subdirectory
1475that is a link however changing to the parent does not change back to the
1476parent of the link rather it changes to the parent of the target of the link
1477and that is something the RSB nor you can track easily. The RSB configuration
1478script's are a collection of various subtle issues so please ask if you are
1479unsure why something is being done a particular way.
1480
1481The preparation phase will often include source and patch setup commands. Outer
1482layers can set the source package and add patches as needed while being able to
1483use a common recipe for the build. Users can override the standard build and
1484supply a custom patch for testing using the user macro command line interface.
1485
1486-------------------------------------------------------------
1487#
1488# Prepare the source code.
1489#
1490%prep
1491  build_top=$(pwd)
1492
1493  %source setup dtc -q -n dtc-v%{dtc_version}
1494  %patch setup dtc -p1
1495
1496  cd ${build_top}
1497-------------------------------------------------------------
1498
1499The configuration file 'gcc-common-1.cfg' is a complex example of source
1500preparation. It contains a number of source packages and patches and it
1501combines these into a single source tree for building. It uses links to map
1502source into the GCC source tree so GCC can be built using the _single source
1503tree_ method. It also shows how to fetch source code from version
1504control. Newlib is taken directly from its CVS repository.
1505
1506Next is the building phase and for the DTC example this is simply a matter of
1507running +make+. Note the use of the RSB macros for commands. In the case of
1508'%{\__make}' it maps to the correct make for your host. In the case of BSD
1509systems we need to use the GNU make and not the GNU make.
1510
1511If your package requires a configuration stage you need to run this before the
1512make stage. Again the GCC common configuration file provides a detailed example.
1513
1514-------------------------------------------------------------
1515%build
1516  build_top=$(pwd)
1517
1518  cd dtc-v%{dtc_version}
1519
1520  %{build_build_flags}
1521
1522  %{__make} PREFIX=%{_prefix}
1523
1524  cd ${build_top}
1525-------------------------------------------------------------
1526
1527You can invoke make with the macro '%{?_smp_flags}' as a command line
1528argument. This macro is controlled by the '--jobs' command line option and the
1529host CPU detection support in the RSB. If you are on a multicore host you can
1530increase the build speed using this macro. It also lets you disabled building on
1531multicores to aid debugging when testing.
1532
1533Next is the install phase. This phase is a little more complex because you may
1534be building a tar file and the end result of the build is never actually
1535installed into the prefix on the build host and you may not even have
1536permissions to perform a real install. Most packages install to the +prefix+
1537and the prefix is typically supplied via the command to the RSB or the
1538package's default is used. The default can vary depending on the host's
1539operating system. To install to a path that is not the prefix the +DESTDIR+
1540make variable is used. Most packages should honour the +DISTDIR+ make variables
1541and you can typically specify it on the command line to make when invoking the
1542install target. This results in the package being installed to a location that
1543is not the prefix but one you can control. The RSB provides a shell variable
1544called +SB_BUILD_ROOT+ you can use. In a build set where you are building a
1545number of packages you can collect all the built packages in a single tree that
1546is captured in the tar file.
1547
1548Also note the use of the macro +%{\__rmdir}+. The use of these macros allow the
1549RSB to vary specific commands based on the host. This can help on hosts like
1550Windows where bugs can effect the standard commands such as 'rm'. There are
1551many many macros to help you. You can find these listed in the +defaults.mc+
1552file and in the trace output. If you are new to creating and editing
1553configurations learning these can take a little time.
1554
1555-------------------------------------------------------------
1556%install
1557  build_top=$(pwd)
1558
1559  %{__rmdir} -rf $SB_BUILD_ROOT
1560
1561  cd dtc-v%{dtc_version}
1562  %{__make} DESTDIR=$SB_BUILD_ROOT PREFIX=%{_prefix} install
1563
1564  cd ${build_top}
1565-------------------------------------------------------------
1566
1567Finally there is an optional clean section. The RSB will run this section if
1568+--no-clean+ has not been provided on the command line. The RSB does clean up
1569for you.
1570
1571Once we have the configuration files we can execute the build using the
1572`sb-builder` command. The command will perform the build and create a tar file
1573in the +tar+ directory.
1574
1575-------------------------------------------------------------
1576$  ../source-builder/sb-builder --prefix=/usr/local \
1577     --log=log_dtc devel/dtc-1.2.0
1578RTEMS Source Builder, Package Builder v0.2.0
1579config: devel/dtc-1.2.0
1580package: dtc-1.2.0-x86_64-freebsd9.1-1
1581download: http://www.jdl.com/software/dtc-v1.2.0.tgz -> sources/dtc-v1.2.0.tgz
1582building: dtc-1.2.0-x86_64-freebsd9.1-1
1583$ ls tar
1584dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2
1585-------------------------------------------------------------
1586
1587If you want to have the package installed automatically you need to create a
1588build set. A build set can build one or more packages from their configurations
1589at once to create a single package. For example the GNU tools is typically seen
1590as binutils, GCC and GDB and a build set will build each of these packages and
1591create a single build set tar file or install the tools on the host into the
1592prefix path.
1593
1594The DTC build set file is called +dtc.bset+ and contains:
1595
1596-------------------------------------------------------------
1597#
1598# Build the DTC.
1599#
1600
1601%define release 1
1602
1603devel/dtc-1.2.0.cfg
1604-------------------------------------------------------------
1605
1606To build this you can use something similar to:
1607
1608-------------------------------------------------------------
1609$ ../source-builder/sb-set-builder --prefix=/usr/local --log=log_dtc \
1610   --trace --bset-tar-file --no-install dtc
1611RTEMS Source Builder - Set Builder, v0.2.0
1612Build Set: dtc
1613config: devel/dtc-1.2.0.cfg
1614package: dtc-1.2.0-x86_64-freebsd9.1-1
1615building: dtc-1.2.0-x86_64-freebsd9.1-1
1616tarball: tar/x86_64-freebsd9.1-dtc-set.tar.bz2
1617cleaning: dtc-1.2.0-x86_64-freebsd9.1-1
1618Build Set: Time 0:00:02.865758
1619$ ls tar
1620dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2   x86_64-freebsd9.1-dtc-set.tar.bz2
1621-------------------------------------------------------------
1622
1623The build is for a FreeBSD host and the prefix is for user installed
1624packages. In this example I cannot let the source builder perform the install
1625because I never run the RSB with root priviledges so a build set or bset tar
1626file is created. This can then be installed using root privildges.
1627
1628The command also supplies the --trace option. The output in the log file will
1629contian all the macros.
1630
1631Debugging
1632^^^^^^^^^
1633
1634New configuration files require debugging. There are two types of
1635debugging. The first is debugging RSB script bugs. The +--dry-run+ option is
1636used here. Suppling this option will result in most of the RSB processing to be
1637performed and suitable output placed in the log file. This with the +--trace+
1638option should help you resolve any issues.
1639
1640The second type of bug to fix are related to the execution of one of
1641phases. These are usually a mix of shell script bugs or package set up or
1642configuration bugs. Here you can use any normal shell script type debug
1643technique such as +set -x+ to output the commands or +echo+
1644statements. Debugging package related issues may require you start a build with
1645teh RSB and supply +--no-clean+ option and then locate the build directories
1646and change directory into them and manually run commands until to figure what
1647the package requires.
1648
1649Snapshot Testing
1650~~~~~~~~~~~~~~~~
1651
1652_This section needs to be updated once I sort out snapshot testing._
1653
1654Testing of release canidates and snapshots is important to those helping
1655maintain tool sets. The RTEMS Source Builder helps by providing a simple and
1656flexible way to use existing build sets and configuration without needing to
1657change them or creating new temporary build sets and configurations.
1658
1659The process uses snapshot macro files loaded via the command line option
1660`--macros`. These files provide macros that override the standard build set and
1661configuration file macros.
1662
1663Lets consider testing a GCC 4.7 snapshot for RTEMS 4.11. Lets assume the
1664current RTEMS 4.11 tools reference GCC 4.7.3 with a patch as the stable tool
1665set. We want to use a recent snapshot with no patches. In the
1666`rtems/config/snapshots` directoy create a file called `gcc-4.7-snapshot.mc`
1667containing:
1668
1669-------------------------------------------------------------
1670[gcc-4.7-snapshot]
1671GCC_Version: none, override, '4.7-20130413'
1672Source:     none, override, 'http://mirrors.kernel.org/sources.redhat.com/gcc/
1673snapshots/%{gcc_version}/gcc-%{gcc_version}.tar.bz2'
1674Patch0:      none, udefine,  ''
1675-------------------------------------------------------------
1676
1677In the standard configuration file `source-builder/config/gcc-4.7-1.cfg` the
1678map is selected with:
1679
1680-------------------------------------------------------------
1681#
1682# Select the GCC 4.7 Snapshot Macro Map
1683#
1684%select gcc-4.7-snapshot
1685-------------------------------------------------------------
1686
1687On the command line add `--macros=snapshots/gcc-4.7-snapshot.mc` and this
1688snapshot will be built. With careful use of the `--prefix` option you can
1689locate the tools in a specific directory and test them without needing to
1690effect your production environment.
1691
1692Scripting
1693~~~~~~~~~
1694
1695Configuration files specify how to build a package. Configuration files are
1696scripts and have a +.cfg+ file extension. The script format is based loosely on
1697the RPM spec file format however the use and purpose in this tool does not
1698compare with the functionality and therefore the important features of the spec
1699format RPM needs and uses.
1700
1701The script language is implemented in terms of macros. The built-in list is:
1702
1703[horizontal]
1704+%{}+:: Macro expansion with conditional logic.
1705+%()+:: Shell expansion.
1706+%prep+:: The source preparation shell commands.
1707+%build+:: The build shell commands.
1708+%install+:: The package install shell commands.
1709+%clean+:: The package clean shell commands.
1710+%include+:: Inline include another configuration file.
1711+%name+:: The name of the package.
1712+%summary+:: A brief package description. Useful when reporting about a build.
1713+%release+:: The package release. A number that is the release as built by this tool.
1714+%version+:: The package's version string.
1715+%buildarch+:: The build architecture.
1716+%source+:: Define a source code package. This macro has a number appended.
1717+%patch+:: Define a patch. This macro has a is number appended.
1718+%echo+:: Print the following string as a message.
1719+%warning+:: Print the following string as a warning and continue.
1720+%error+:: Print the following string as an error and exit.
1721+%select+:: Select the macro map. If there is no map nothing is reported.
1722+%define+:: Define a macro. Macros cannot be redefined, you must first undefine it.
1723+%undefine+:: Undefine a macro.
1724+%if+:: Start a conditional logic block that ends with a +%endif+.
1725+%ifn+:: Inverted start of a conditional logic block.
1726+%ifarch+:: Test the architecture against the following string.
1727+%ifnarch+:: Inverted test of the architecture
1728+%ifos+:: Test the host operating system.
1729+%else+:: Start the _else_ conditional logic block.
1730+%endfi+:: End the conditional logic block.
1731+%bconf_with+:: Test the build condition _with_ setting. This is the +--with-*+
1732command line option.
1733+%bconf_without+:: Test the build condition _without_ setting. This is the
1734+--without-*+ command line option.
1735
1736Expanding
1737^^^^^^^^^
1738
1739A macro can be `%{string}` or the equivalent of `%string`. The following macro
1740expansions supported are:
1741
1742`%{string}`;;
1743Expand the 'string' replacing the entire macro text with the text in the table
1744for the entry 'string . For example if 'var' is 'foo' then `${var}` would
1745become `foo`.
1746
1747`%{expand: string}`;;
1748Expand the 'string' and then use it as a ``string'' to the macro expanding the
1749macro. For example if _foo_ is set to 'bar' and 'bar' is set to 'foobar' then
1750`%{expand:foo}` would result in `foobar`. Shell expansion can also be used.
1751
1752`%{with string}`;;
1753Expand the macro to '1' if the macro `with_`'string' is defined else expand to
1754_0_. Macros with the name `with_`'string' can be define with command line
1755arguments to the RTEMS Source Builder commands.
1756
1757`%{defined string}`;;
1758Expand the macro to '1' if a macro of name 'string' is defined else expand to '0'.
1759
1760`%{?string: expression}`;;
1761Expand the macro to 'expression' if a macro of name 'string' is defined else expand to `%{nil}`.
1762
1763`%{!?string: expression}`;;
1764Expand the macro to 'expression' if a macro of name 'string' is not defined. If
1765the macro is define expand to `%{nil}`.
1766
1767`%(expression)`;;
1768Expand the macro to the result of running the 'expression' in a host shell. It
1769is assumed this is a Unix type shell. For example `%(whoami)` will return your
1770user name and `%(date)` will return the current date string.
1771
1772%prep
1773^^^^^
1774
1775The +%prep+ macro starts a block that continues until the next block macro. The
1776_prep_ or preparation block defines the setup of the package's source and is a
1777mix of RTEMS Source Builder macros and shell scripting. The sequence is
1778typically +%source+ macros for source, +%patch+ macros to patch the source
1779mixed with some shell commands to correct any source issues.
1780
1781-------------------------------------------------------------
1782              <1>                <2>      <3>
1783%source setup gcc -q -c -T -n %{name}-%{version}
1784-------------------------------------------------------------
1785
1786<1> The source group to set up.
1787<2> The source's name.
1788<3> The version of the source.
1789
1790The source set up are declared with the source +set+ and +add+ commands. For
1791example:
1792
1793-------------------------------------------------------------
1794%source set gdb http://ftp.gnu.org/gnu/gdb/gdb-%{gdb_version}.tar.bz2
1795-------------------------------------------------------------
1796
1797This URL is the primary location of the GNU GDB source code and the RTEMS
1798Source Builder can download the file from this location and by inspecting the
1799file extension use +bzip2+ decompression with +tar+. When the +%prep+ section
1800is processed a check of the local +source+ directory is made to see if the file
1801has already been downloaded. If not found in the source cache directory the
1802package is downloaded from the URL. You can append other base URLs via the
1803command line option +--url+. This option accepts a comma delimited list of
1804sites to try.
1805
1806You could optionally have a few source files that make up the package. For
1807example GNU's GCC was a few tar files for a while and it is now a single tar
1808file. Support for multiple source files can be conditionally implemented with
1809the following scripting:
1810
1811-------------------------------------------------------------
1812%source set gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-code-%{gcc_version}.tar.bz2
1813%source add gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-g++-%{gcc_version}.tar.bz2
1814%source setup gcc -q -T -D -n gcc-%{gcc_version}
1815-------------------------------------------------------------
1816
1817Separate modules use separate source groups. The GNU GCC compiler for RTEMS
1818uses Newlib, MPFR, MPC, and GMP source packages. You define the source with:
1819
1820-------------------------------------------------------------
1821%source set gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
1822%source set newlib ftp://sourceware.org/pub/newlib/newlib-%{newlib_version}.tar.gz
1823%source set mpfr http://www.mpfr.org/mpfr-%{mpfr_version}/mpfr-%{mpfr_version}.tar.bz2
1824%source set mpc http://www.multiprecision.org/mpc/download/mpc-%{mpc_version}.tar.gz
1825%source set gmp ftp://ftp.gnu.org/gnu/gmp/gmp-%{gmp_version}.tar.bz2
1826-------------------------------------------------------------
1827
1828and set up with:
1829
1830-------------------------------------------------------------
1831%source setup gcc -q -n gcc-%{gcc_version}
1832%source setup newlib -q -D -n newlib-%{newlib_version}
1833%source setup mpfr -q -D -n mpfr-%{mpfr_version}
1834%source setup mpc -q -D -n mpc-%{mpc_version}
1835%source setup gmp -q -D -n gmp-%{gmp_version}
1836-------------------------------------------------------------
1837
1838Patching also occurs during the preparation stage. Patches are handled in a
1839similar way to the source packages except you only +add+ patches. Patches are
1840applied using the +setup+ command. The +setup+ command takes the default patch
1841option. You can provide options with each patch by adding them as arguments
1842before the patch URL. Patches with no options uses the +setup+ default.
1843
1844-------------------------------------------------------------
1845%patch add gdb %{rtems_gdb_patches}/gdb-sim-arange-inline.diff
1846%patch add gdb -p0 <1> %{rtems_gdb_patches}/gdb-sim-cgen-inline.diff
1847-------------------------------------------------------------
1848<1> This patch has a custom option.
1849
1850To apply these patches:
1851
1852-------------------------------------------------------------
1853%patch setup gdb -p1 <1>
1854-------------------------------------------------------------
1855<1> The default options.
1856
1857%build
1858^^^^^^
1859
1860The +%build+ macro starts a block that continues until the next block
1861macro. The build block is a series of shell commands that execute to build the
1862package. It assumes all source code has been unpacked, patch and adjusted so
1863the build will succeed.
1864
1865The following is an example take from the GutHub STLink project:
1866
1867NOTE: STLink is a JTAG debugging device for the ST ARM family of processors.
1868
1869-------------------------------------------------------------
1870%build
1871  export PATH="%{_bindir}:${PATH}" <1>
1872
1873  cd texane-stlink-%{stlink_version} <2>
1874
1875  ./autogen.sh <3>
1876
1877%if "%{_build}" != "%{_host}"
1878  CFLAGS_FOR_BUILD="-g -O2 -Wall" \ <4>
1879%endif
1880  CPPFLAGS="-I $SB_TMPPREFIX/include/libusb-1.0" \ <5>
1881  CFLAGS="$SB_OPT_FLAGS" \
1882  LDFLAGS="-L $SB_TMPPREFIX/lib" \
1883  ./configure \ <6>
1884    --build=%{_build} --host=%{_host} \
1885    --verbose \
1886    --prefix=%{_prefix} --bindir=%{_bindir} \
1887    --exec-prefix=%{_exec_prefix} \
1888    --includedir=%{_includedir} --libdir=%{_libdir} \
1889    --mandir=%{_mandir} --infodir=%{_infodir}
1890
1891  %{__make} %{?_smp_mflags} all <7>
1892
1893  cd ..
1894-------------------------------------------------------------
1895
1896<1> Setup the PATH environment variable. This is not always needed.
1897<2> This package builds in the source tree so enter it.
1898<3> The package is actually checked directly out from the github project and so
1899    it needs its autoconf and automake files generated.
1900<4> Flags for a cross-compiled build.
1901<5> Various settings passed to configure to customise the build. In this
1902    example an include path is being set to the install point of _libusb_. This
1903    package requires _libusb_ is built before it.
1904<6> The +configure+ command. The RTEMS Source Builder provides all the needed
1905    paths as macro variables. You just need to provide them to +configure+.
1906<7> Running make. Do not use +make+ directly, use the RTEMS Source Builder's
1907    defined value. This value is specific to the host. A large number of
1908    packages need GNU make and on BSD systems this is +gmake+. You can
1909    optionally add the SMP flags if the packages build system can handle
1910    parallel building with multiple jobs. The +_smp_mflags+ value is
1911    automatically setup for SMP hosts to match the number of cores the host has.
1912
1913%install
1914^^^^^^^^
1915
1916The +%install+ macro starts a block that continues until the next block
1917macro. The install block is a series of shell commands that execute to install
1918the package. You can assume the package has build correctly when this block
1919starts executing.
1920
1921Never install the package to the actual _prefix_ the package was built
1922with. Always install to the RTEMS Source Builder's temporary path defined in
1923the macro variable +\__tmpdir+. The RTEMS Source Builder sets up a shell
1924environment variable called +SB_BUILD_ROOT+ as the standard install point. Most
1925packages support adding +DESTDIR=+ to the _make install_ command.
1926
1927Looking at the same example as in <<_build, %build>>:
1928
1929-------------------------------------------------------------
1930%install
1931  export PATH="%{_bindir}:${PATH}" <1>
1932  rm -rf $SB_BUILD_ROOT <2>
1933
1934  cd texane-stlink-%{stlink_version} <3>
1935  %{__make} DESTDIR=$SB_BUILD_ROOT install <4>
1936
1937  cd ..
1938-------------------------------------------------------------
1939
1940<1> Setup the PATH environment variable. This is not always needed.
1941<2> Clean any installed files. This make sure the install is just what
1942    the package installs and not any left over files from a broken build or
1943    install.
1944<3> Enter the build directory. In this example it just happens to be the source
1945    directory.
1946<4> Run +make install+ to install the package overriding the +DESTDIR+ make
1947    variable.
1948
1949%clean
1950^^^^^^
1951
1952The +%clean+ macro starts a block that continues until the next block
1953macro. The clean block is a series of shell commands that execute to clean up
1954after a package has been built and install. This macro is currenly not been
1955used because the RTEMS Source Builder automatically cleans up.
1956
1957%include
1958^^^^^^^^
1959
1960The +%include+ macro inline includes the specific file. The +\__confdir+
1961path is searched. Any relative path component of the include file is appended
1962to each part of the +\__configdir+. Adding an extension is optional as files
1963with +.bset+ and +.cfg+ are automatically searched for.
1964
1965Inline including means the file is processed as part of the configuration at
1966the point it is included. Parsing continues from the next line in the
1967configuration file that contains the +%include+ macro.
1968
1969Including files allow a kind of configuration file reuse. The outer
1970configuration files provide specific information such as package version
1971numbers and patches and then include a generic configuration script which
1972builds the package.
1973
1974-------------------------------------------------------------
1975%include %{_configdir}/gcc-4.7-1.cfg
1976-------------------------------------------------------------
1977
1978%name
1979^^^^^
1980
1981The name of the package being built. The name typically contains the components
1982of the package and their version number plus a revision number. For the GCC
1983with Newlib configuration the name is typically:
1984
1985-------------------------------------------------------------
1986Name: %{_target}-gcc-%{gcc_version}-newlib-%{newlib_version}-%{release}
1987-------------------------------------------------------------
1988
1989%summary
1990^^^^^^^^
1991
1992The +%summary+ is a brief description of the package. It is useful when
1993reporting. This information is not capture in the package anywhere. For the GCC
1994with Newlib configuration the summary is typically:
1995
1996-------------------------------------------------------------
1997Summary: GCC v%{gcc_version} and Newlib v%{newlib_version} for target %{_target} on host %{_host}
1998-------------------------------------------------------------
1999
2000%release
2001^^^^^^^^
2002
2003The +%release+ is packaging number that allows revisions of a package to happen
2004where none package versions change. This value typically increases when the
2005configuration building the package changes.
2006
2007-------------------------------------------------------------
2008%define release 1
2009-------------------------------------------------------------
2010
2011%version
2012^^^^^^^^
2013
2014The +%version% macro sets the version the package. If the package is a single
2015component it tracks that component's version number. For example in the
2016_libusb_ configuration the +%version+ is the same as +%libusb_version+, however
2017in a GCC with Newlib configuration there is no single version number. In this
2018case the GCC version is used.
2019
2020-------------------------------------------------------------
2021Version: %{gcc_version}
2022-------------------------------------------------------------
2023
2024%buildarch
2025^^^^^^^^^^
2026
2027The +%buildarch+ macro is set to the architecture the package contains. This is
2028currently not used in the RTEMS Source Builder and may go away. This macro is
2029more important in a real packaging system where the package could end up on the
2030wrong architecture.
2031
2032%source
2033^^^^^^^
2034
2035The +%source+ macro has 3 commands that controls what it does. You can +set+
2036the source files, +add+ source files to a source group, and +setup+ the source
2037file group getting it ready to be used.
2038
2039Source files are source code files in tar or zip files that are unpacked,
2040copied or symbolically linked into the package's build tree. Building a package
2041requires one or more dependent packages. These are typically the packages
2042source code plus dependent libraries or modules. You can create any number of
2043these source groups and set each of them up with a separe source group for each
2044needed library or module. Each source group normally has a single tar, zip or
2045repository and the +set+ defines this. Some projects split the source code into
2046separate tar or zip files and you install them by using the +add+ command.
2047
2048The first instance of a +set+ command creates the source group and sets the
2049source files to be set up. Subsequence +set+ commands for the same source group
2050are ignored. this lets you define the standard source files and override them
2051for specific releases or snapshots.. To set a source file group:
2052
2053-------------------------------------------------------------
2054%source set gcc <1> ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
2055-------------------------------------------------------------
2056<1> The source group is +gcc+.
2057
2058To add another source package to be installed into the same source tree you use
2059the +add+ command:
2060
2061-------------------------------------------------------------
2062%source add gcc ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/g++-%{gcc_version}.tar.bz2
2063-------------------------------------------------------------
2064
2065The source +setup+ command can only be issued in the +%prep:+ section. The
2066setup is:
2067
2068-------------------------------------------------------------
2069%source gcc setup -q -T -D -n %{name}-%{version}
2070-------------------------------------------------------------
2071
2072Accepted options are:
2073
2074[horizontal]
2075*Switch*:: *Description*
2076+-n+:: The -n option is used to set the name of the software's build
2077directory. This is necessary only when the source archive unpacks into a
2078directory named other than +<name>-<version>+.
2079+-c+:: The -c option is used to direct %setup to create the top-level build
2080directory before unpacking the sources.
2081+-D+:: The -D option is used to direct %setup to not delete the build directory
2082prior to unpacking the sources. This option is used when more than one source
2083archive is to be unpacked into the build directory, normally with the +-b+ or
2084+-a+ options.
2085+-T+:: The -T option is used to direct %setup to not perform the default
2086unpacking of the source archive specified by the first Source: macro. It is used
2087with the +-a+ or +-b+ options.
2088+-b <n>+:: The -b option is used to direct %setup to unpack the source archive
2089specified on the nth Source: macro line before changing directory into the build
2090directory.
2091
2092%patch
2093^^^^^^
2094
2095The +%patch+ macro has the same 3 command as the +%source+ command however the
2096+set+ commands is not really that useful with the with command. You add patches
2097with the +add+ command and +setup+ applies the patches. Patch options can be
2098added to each patch by placing them before the patch URL. If no patch option is
2099provided the default options passed to the +setup+ command are used. An option
2100starts with a '-'. The +setup+ command must reside inside the +%prep+ section.
2101
2102Patches are grouped in a similar way to the +%source+ macro so you can control
2103applying a group of patches to a specific source tree.
2104
2105The +__patchdir+ path is search.
2106
2107
2108To add a patch:
2109
2110-------------------------------------------------------------
2111%patch add gcc <1>  gcc-4.7.2-rtems4.11-20121026.diff
2112%patch add gcc -p0 <2>  gcc-4.7.2-rtems4.11-20121101.diff
2113-------------------------------------------------------------
2114<1> The patch group is +gcc+.
2115<2> Option for this specific patch.
2116
2117Placing +%patch setup+ in the +%prep+ section will apply the groups patches.
2118
2119-------------------------------------------------------------
2120%patch setup gcc <1>  -p1 <2>
2121-------------------------------------------------------------
2122<1> The patch group.
2123<2> The default option used to apply the patch.
2124
2125%echo
2126^^^^^
2127
2128The +%echo+ macro outputs the following string to stdout. This can also be used
2129as `%{echo: message}`.
2130
2131%warning
2132^^^^^^^^
2133
2134The +%warning+ macro outputs the following string as a warning. This can also
2135be used as `%{warning: message}`.
2136
2137%error
2138^^^^^^
2139
2140The +%error+ macro outputs the follow string as an error and exits the RTEMS
2141Source Builder. This can also be used as `%{error: message}`.
2142
2143%select
2144^^^^^^^
2145
2146The +%select+ macro selects the map specified. If there is no map no error or
2147warning is generated. Macro maps provide a simple way for a user to override
2148the settings is a configuration file without having to edit it. The changes are
2149recorded in the build report so can be traced.
2150
2151Configuration use different maps so macro overrides can target a specific
2152package.
2153
2154The default map is `global'.
2155
2156-------------------------------------------------------------
2157%select gcc-4.8-snapshot <1>
2158%define one_plus_one 2 <2>
2159-------------------------------------------------------------
2160
2161<1> The map switches to `gcc-4.8-snapshot`. Any overrides in this map will be
2162    used.
2163<2> Defining macros only updates the `global` map and not the selected map.
2164
2165%define
2166^^^^^^^
2167
2168The +%define+ macro defines a new macro or updates an existing one. If no value
2169is given it is assumed to be 1.
2170
2171-------------------------------------------------------------
2172%define foo bar
2173%define one_plus_one 2
2174%define one <1>
2175-------------------------------------------------------------
2176
2177<1> The macro _one_ is set to 1.
2178
2179%undefine
2180^^^^^^^^^
2181
2182The +%undefine+ macro removes a macro if it exists. Any further references to
2183it will result in an undefine macro error.
2184
2185%if
2186^^^
2187
2188The +%if+ macro starts a conditional logic block that can optionally have a
2189_else_ section. A test follows this macro and can have the following operators:
2190
2191[horizontal]
2192*Operator*:: *Description*
2193+%{}+:: Check the macro is set or _true_, ie non-zero.
2194+
2195-------------------------------------------------------------
2196%if ${foo}
2197 %warning The test passes, must not be empty or is non-zero
2198%else
2199 %error The test fails, must be empty or zero
2200%endif
2201-------------------------------------------------------------
2202+!+:: The _not_ operator inverts the test of the macro.
2203+
2204-------------------------------------------------------------
2205%if ! ${foo}
2206 %warning The test passes, must be empty or zero
2207%else
2208 %error The test fails, must not be empty or is non-zero
2209%endif
2210-------------------------------------------------------------
2211+==+:: The left hand size must equal the right hand side. For example:
2212+
2213-------------------------------------------------------------
2214%define one 1
2215%if ${one} == 1
2216 %warning The test passes
2217%else
2218 %error The test fails
2219%endif
2220-------------------------------------------------------------
2221+
2222You can also check to see if a macro is empty:
2223+
2224-------------------------------------------------------------
2225%if ${nothing} == %{nil}
2226 %warning The test passes
2227%else
2228 %error The test fails
2229%endif
2230-------------------------------------------------------------
2231+!=+:: The left hand size does not equal the right hand side. For example:
2232+
2233-------------------------------------------------------------
2234%define one 1
2235%if ${one} != 2
2236 %warning The test passes
2237%else
2238 %error The test fails
2239%endif
2240-------------------------------------------------------------
2241+
2242You can also check to see if something is set:
2243+
2244-------------------------------------------------------------
2245%if ${something} != %{nil}
2246 %warning The test passes
2247%else
2248 %error The test fails
2249%endif
2250-------------------------------------------------------------
2251+>+:: The left hand side is numerically greater than the right hand side.
2252+>=+:: The left hand side is numerically greater than or equal to the right
2253hand side.
2254+<+:: The left hand side is numerically less than the right hand side.
2255+\<=+:: The left hand side is numerically less than or equal to the right hand
2256side.
2257
2258%ifn
2259^^^^
2260
2261The +%ifn+ macro inverts the normal +%if+ logic. It avoids needing to provide
2262empty _if_ blocks followed by _else_ blocks. It is useful when checking if a
2263macro is defined:
2264
2265-------------------------------------------------------------
2266%ifn %{defined foo}
2267 %define foo bar
2268%endif
2269-------------------------------------------------------------
2270
2271%ifarch
2272^^^^^^^
2273
2274The +%ifarch+ is a short cut for "+%if %{\_arch} == i386+". Currently not used.
2275
2276%ifnarch
2277^^^^^^^^
2278
2279The +%ifnarch+ is a short cut for "+%if %{\_arch} != i386+". Currently not
2280used.
2281
2282%ifos
2283^^^^^
2284
2285The +%ifos+ is a short cut for "+%if %{\_os} != mingw32+". It allows
2286conditional support for various operating system differences when building
2287packages.
2288
2289%else
2290^^^^^
2291
2292The +%else+ macro starts the conditional _else_ block.
2293
2294%endfi
2295^^^^^^
2296
2297The +%endif+ macro ends a conditional logic block.
2298
2299%bconf_with
2300^^^^^^^^^^^
2301
2302The +%bconf_with+ macro provides a way to test if the user has passed a
2303specific option on the command line with the +--with-<label>+ option. This
2304option is only available with the +sb-builder+ command.
2305
2306%bconf_without
2307^^^^^^^^^^^^^^
2308
2309The +%bconf_without+ macro provides a way to test if the user has passed a
2310specific option on the command line with the +--without-<label>+ option. This
2311option is only available with the +sb-builder+ command.
2312
2313Commands
2314--------
2315
2316Checker (sb-check)
2317~~~~~~~~~~~~~~~~~~
2318
2319This commands checks your system is set up correctly. Most options are ignored.
2320
2321-------------------------------------------------------------
2322$ ../source-builder/sb-check --help
2323sb-check: [options] [args]
2324RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2325Options and arguments:
2326--force                : Force the build to proceed
2327--quiet                : Quiet output (not used)
2328--trace                : Trace the execution
2329--dry-run              : Do everything but actually run the build
2330--warn-all             : Generate warnings
2331--no-clean             : Do not clean up the build tree
2332--always-clean         : Always clean the build tree, even with an error
2333--jobs                 : Run with specified number of jobs, default: num CPUs.
2334--host                 : Set the host triplet
2335--build                : Set the build triplet
2336--target               : Set the target triplet
2337--prefix path          : Tools build prefix, ie where they are installed
2338--topdir path          : Top of the build tree, default is $PWD
2339--configdir path       : Path to the configuration directory, default: ./config
2340--builddir path        : Path to the build directory, default: ./build
2341--sourcedir path       : Path to the source directory, default: ./source
2342--tmppath path         : Path to the temp directory, default: ./tmp
2343--macros file[,[file]  : Macro format files to load after the defaults
2344--log file             : Log file where all build out is written too
2345--url url[,url]        : URL to look for source
2346--no-download          : Disable the source downloader
2347--targetcflags flags   : List of C flags for the target code
2348--targetcxxflags flags : List of C++ flags for the target code
2349--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2350--with-<label>         : Add the --with-<label> to the build
2351--without-<label>      : Add the --without-<label> to the build
2352--regression           : Set --no-install, --keep-going and --always-clean
2353$ ../source-builder/sb-check
2354RTEMS Source Builder - Check, v0.2.0
2355Environment is ok
2356-------------------------------------------------------------
2357
2358Defaults (sb-defaults)
2359~~~~~~~~~~~~~~~~~~~~~~
2360
2361This commands outputs and the default macros for your when given no
2362arguments. Most options are ignored.
2363
2364-------------------------------------------------------------
2365$ ../source-builder/sb-defaults --help
2366sb-defaults: [options] [args]
2367RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2368Options and arguments:
2369--force                : Force the build to proceed
2370--quiet                : Quiet output (not used)
2371--trace                : Trace the execution
2372--dry-run              : Do everything but actually run the build
2373--warn-all             : Generate warnings
2374--no-clean             : Do not clean up the build tree
2375--always-clean         : Always clean the build tree, even with an error
2376--jobs                 : Run with specified number of jobs, default: num CPUs.
2377--host                 : Set the host triplet
2378--build                : Set the build triplet
2379--target               : Set the target triplet
2380--prefix path          : Tools build prefix, ie where they are installed
2381--topdir path          : Top of the build tree, default is $PWD
2382--configdir path       : Path to the configuration directory, default: ./config
2383--builddir path        : Path to the build directory, default: ./build
2384--sourcedir path       : Path to the source directory, default: ./source
2385--tmppath path         : Path to the temp directory, default: ./tmp
2386--macros file[,[file]  : Macro format files to load after the defaults
2387--log file             : Log file where all build out is written too
2388--url url[,url]        : URL to look for source
2389--no-download          : Disable the source downloader
2390--targetcflags flags   : List of C flags for the target code
2391--targetcxxflags flags : List of C++ flags for the target code
2392--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2393--with-<label>         : Add the --with-<label> to the build
2394--without-<label>      : Add the --without-<label> to the build
2395--regression           : Set --no-install, --keep-going and --always-clean
2396-------------------------------------------------------------
2397
2398Set Builder (sb-set-builder)
2399~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2400
2401This command builds a set.
2402
2403-------------------------------------------------------------
2404$ ../source-builder/sb-set-builder --help
2405RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
2406Options and arguments:
2407--force                : Force the build to proceed
2408--quiet                : Quiet output (not used)
2409--trace                : Trace the execution
2410--dry-run              : Do everything but actually run the build
2411--warn-all             : Generate warnings
2412--no-clean             : Do not clean up the build tree
2413--always-clean         : Always clean the build tree, even with an error
2414--regression           : Set --no-install, --keep-going and --always-clean
2415---jobs                 : Run with specified number of jobs, default: num CPUs.
2416--host                 : Set the host triplet
2417--build                : Set the build triplet
2418--target               : Set the target triplet
2419--prefix path          : Tools build prefix, ie where they are installed
2420--topdir path          : Top of the build tree, default is $PWD
2421--configdir path       : Path to the configuration directory, default: ./config
2422--builddir path        : Path to the build directory, default: ./build
2423--sourcedir path       : Path to the source directory, default: ./source
2424--tmppath path         : Path to the temp directory, default: ./tmp
2425--macros file[,[file]  : Macro format files to load after the defaults
2426--log file             : Log file where all build out is written too
2427--url url[,url]        : URL to look for source
2428--no-download          : Disable the source downloader
2429--no-install           : Do not install the packages to the prefix
2430--targetcflags flags   : List of C flags for the target code
2431--targetcxxflags flags : List of C++ flags for the target code
2432--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2433--with-<label>         : Add the --with-<label> to the build
2434--without-<label>      : Add the --without-<label> to the build
2435--mail-from            : Email address the report is from.
2436--mail-to              : Email address to send the email too.
2437--mail                 : Send email report or results.
2438--smtp-host            : SMTP host to send via.
2439--no-report            : Do not create a package report.
2440--report-format        : The report format (text, html, asciidoc).
2441--bset-tar-file        : Create a build set tar file
2442--pkg-tar-files        : Create package tar files
2443--list-bsets           : List available build sets
2444--list-configs         : List available configurations
2445--list-deps            : List the dependent files.
2446-------------------------------------------------------------
2447
2448.Arguments
2449The +[args]+ are a list build sets to build.
2450
2451.Options
2452+--force+;;
2453Force the build to proceed even if the host check fails. Typically this happens
2454if executable files are found in the path at a different location to the host
2455defaults.
2456+--trace+;;
2457Trace enable printing of debug information to stdout. It is really only of use
2458to RTEMS Source Builder's developers.
2459+--dry-run+;;
2460Do everything but actually run the build commands. This is useful when checking
2461a new configuration parses cleanly.
2462+--warn-all+;;
2463Generate warnings.
2464+--no-clean+;;
2465Do not clean up the build tree during the cleaning phase of the build. This
2466leaves the source and the build output on disk so you can make changes, or
2467amend or generate new patches. It also allows you to review configure type
2468output such as +config.log+.
2469+--always-clean+;;
2470Clean away the results of a build even if the build fails. This is normally
2471used with `--keep-going` when regression testing to see which build sets
2472fail to build. It keeps the disk usage down.
2473+--jobs+;;
2474Control the number of jobs make is given. The jobs can be 'none' for only 1
2475job, 'half' so the number of jobs is half the number of detected cores, a
2476fraction such as '0.25' so the number of jobs is a quarter of the number of
2477detected cores and a number such as '25' which forces the number of jobs to
2478that number.
2479+--host+;;
2480Set the host triplet value. Be careful with this option.
2481+--build+;;
2482Set the build triplet. Be careful with this option.
2483+--target+;;
2484Set the target triplet. Be careful with this option. This is useful if you have
2485a generic configuration script that can work for a range of architectures.
2486+--prefix path+;;
2487Tools build prefix, ie where they are installed.
2488+--topdir path+;;
2489Top of the build tree, that is the current directory you are in.
2490+--configdir path+;;
2491Path to the configuration directory. This overrides the built in defaults.
2492+--builddir path+;;
2493Path to the build directory. This overrides the default of +build+.
2494+--sourcedir path+;;
2495Path to the source directory. This overrides the default of +source+.
2496+--tmppath path+;;
2497Path to the temporary directory. This overrides the default of +tmp+.
2498+--macros files+;;
2499Macro files to load. The configuration directory path is searched.
2500+--log file+;;
2501Log all the output from the build process. The output is directed to +stdout+
2502if no log file is provided.
2503+--url url+;;
2504URL to look for source when downloading. This is can be comma separate list.
2505+--no-download+;;
2506Disable downloading of source and patches. If the source is not found an error
2507is raised.
2508+--targetcflags flags+;;
2509List of C flags for the target code. This allows for specific local
2510customisation when testing new variations.
2511+--targetcxxflags flags+;;
2512List of C++ flags for the target code. This allows for specific local
2513customisation when testing new variations.
2514+--libstdcxxflags flags+;;
2515List of C\++ flags to build the target libstdc++ code. This allows for specific
2516local customisation when testing new variations.
2517+--with-<label>+;;
2518Add the --with-<label> to the build. This can be tested for in a script with
2519the +%bconf_with+ macro.
2520+--without-<label>+;;
2521Add the --without-<label> to the build. This can be tested for in a script with
2522the +%bconf_without+ macro.
2523+--mail-from+;;
2524Set the from mail address if report mailing is enabled.
2525+--mail-to+;;
2526Set the to mail address if report mailing is enabled. The report is mailed to
2527this address.
2528+--mail+;;
2529Mail the build report to the mail to address.
2530+--smtp-host+;;
2531The SMTP host to use to send the email. The default is +localhost+.
2532+--no-report+;;
2533Do not create a report format.
2534+--report-format format+;;
2535The report format can be 'text' or 'html'. The default is 'html'.
2536+--keep-going+;;
2537Do not stop on error. This is useful if your build sets performs a large number
2538of testing related builds and there are errors.
2539+--always-clean+.
2540Always clean the build tree even with a failure.
2541+--no-install+;;
2542Do not install the packages to the prefix. Use this if you are only after the
2543tar files.
2544+--regression+;;
2545A convenience option which is the same as +--no-install+, +--keep-going+ and
2546+--bset-tar-file+;;
2547Create a build set tar file. This is a single tar file of all the packages in
2548the build set.
2549+--pkg-tar-files+;;
2550Create package tar files. A tar file will be created for each package built in
2551a build set.
2552+--list-bsets+;;
2553List available build sets.
2554+--list-configs+;;
2555List available configurations.
2556+--list-deps+;;
2557Print a list of dependent files used by a build set. Dependent files have a
2558'dep[?]' prefix where '?' is a number. The files are listed alphabetically.
2559
2560Set Builder (sb-builder)
2561~~~~~~~~~~~~~~~~~~~~~~~~
2562
2563This command builds a configuration as described in a configuration
2564file. Configuration files have the extension of +.cfg+.
2565
2566-------------------------------------------------------------
2567$ ./source-builder/sb-builder --help
2568sb-builder: [options] [args]
2569RTEMS Source Builder, an RTEMS Tools Project (c) 2012 Chris Johns
2570Options and arguments:
2571--force                : Force the build to proceed
2572--quiet                : Quiet output (not used)
2573--trace                : Trace the execution
2574--dry-run              : Do everything but actually run the build
2575--warn-all             : Generate warnings
2576--no-clean             : Do not clean up the build tree
2577--always-clean         : Always clean the build tree, even with an error
2578--jobs                 : Run with specified number of jobs, default: num CPUs.
2579--host                 : Set the host triplet
2580--build                : Set the build triplet
2581--target               : Set the target triplet
2582--prefix path          : Tools build prefix, ie where they are installed
2583--topdir path          : Top of the build tree, default is $PWD
2584--configdir path       : Path to the configuration directory, default: ./config
2585--builddir path        : Path to the build directory, default: ./build
2586--sourcedir path       : Path to the source directory, default: ./source
2587--tmppath path         : Path to the temp directory, default: ./tmp
2588--macros file[,[file]  : Macro format files to load after the defaults
2589--log file             : Log file where all build out is written too
2590--url url[,url]        : URL to look for source
2591--targetcflags flags   : List of C flags for the target code
2592--targetcxxflags flags : List of C++ flags for the target code
2593--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
2594--with-<label>         : Add the --with-<label> to the build
2595--without-<label>      : Add the --without-<label> to the build
2596--list-configs         : List available configurations
2597-------------------------------------------------------------
2598
2599Host Setups
2600-----------
2601
2602The host versions are listed. If a later version of the host operating system
2603exists it should work unless listed.
2604
2605Please provide patches to update these sections if they are wrong or need
2606updating. I cannot install and test each one and rely on getting your feedback.
2607
2608Linux
2609~~~~~
2610
2611A number of different Linux distrubutions are known to work. The following have
2612been tested and report as working.
2613
2614Archlinux
2615^^^^^^^^^
2616
2617The following packages are required on a fresh Archlinux 64bit installation:
2618
2619--------------------------------------------------------------
2620# pacman -S base-devel gdb xz unzip ncurses git zlib
2621--------------------------------------------------------------
2622
2623Archlinux, by default installs `texinfo-5` which is incompatible for building
2624GCC 4.7 tree. You will have to obtain `texinfo-legacy` from `AUR` and provide
2625a manual override.
2626
2627--------------------------------------------------------------
2628# pacman -R texinfo
2629$ yaourt -S texinfo-legacy
2630# ln -s /usr/bin/makeinfo-4.13a /usr/bin/makeinfo
2631--------------------------------------------------------------
2632
2633CentOS
2634^^^^^^
2635
2636The following packages are required on a minimal CentOS 6.3 64bit installation:
2637
2638-------------------------------------------------------------
2639# yum install autoconf automake binutils gcc gcc-c++ gdb make patch \
2640bison flex xz unzip ncurses-devel texinfo zlib-devel python-devel git
2641-------------------------------------------------------------
2642
2643The minimal CentOS distribution is a specific DVD that installs a minimal
2644system. If you use a full system some of these packages may have been
2645installed.
2646
2647Fedora
2648^^^^^^
2649
2650The RTEMS Source Builder has been tested on Fedora 18 64bit.
2651
2652-------------------------------------------------------------
2653# yum install ncurses-devel python4.7-devel
2654-------------------------------------------------------------
2655
2656Raspbian
2657^^^^^^^^
2658
2659The is the Debian distribution for the Raspberry Pi. The following packages are
2660required.
2661
2662-------------------------------------------------------------
2663$ sudo apt-get install autoconf automake bison flex binutils gcc g++ gdb \
2664texinfo unzip ncurses-dev python-dev git
2665-------------------------------------------------------------
2666
2667It is recommended you get Model B of the Pi with 512M of memory and to mount a
2668remote disk over the network. The tools can be build with a prefix under your
2669home directory as recommended and end up on the SD card.
2670
2671Ubuntu
2672^^^^^^
2673
2674The latest testing was with Ubuntu 13.10 64bit. This section also includes Xubuntu. A
2675minimal installation was used and the following packages installed.
2676
2677-------------------------------------------------------------
2678$ sudo apt-get build-dep binutils gcc g++ gdb unzip git python2.7-dev
2679-------------------------------------------------------------
2680
2681FreeBSD
2682~~~~~~~
2683
2684The RTEMS Source Builder has been tested on FreeBSD 9.1 and 10.0 64bit. You
2685need to install some ports. They are:
2686
2687-------------------------------------------------------------
2688# cd /usr/ports
2689# portinstall --batch lang/python27
2690-------------------------------------------------------------
2691
2692If you wish to build Windows (mingw32) tools please install the following
2693ports:
2694
2695-------------------------------------------------------------
2696# cd /usr/ports
2697# portinstall --batch devel/mingw32-binutils devel/mingw32-gcc
2698# portinstall --batch devel/mingw32-zlib devel/mingw32-pthreads
2699-------------------------------------------------------------
2700
2701The +zlip+ and +pthreads+ ports for MinGW32 are used for builiding a Windows
2702QEMU.
2703
2704If you are on FreeBSD 10.0 and you have pkgng installed you can use 'pkg
2705install' rather than 'portinstall'.
2706
2707NetBSD
2708~~~~~~
2709
2710The RTEMS Source Builder has been tested on NetBSD 6.1 i386. Packages to add
2711are:
2712
2713-------------------------------------------------------------
2714# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/devel/gmake-3.82nb7.tgz
2715# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/devel/bison-2.7.1.tgz
2716# pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/6.1/archivers/xz-5.0.4.tgz
2717-------------------------------------------------------------
2718
2719MacOS X
2720~~~~~~~
2721
2722The RTEMS Source Builder has been tested on Mountain Lion. You will need to
2723install the Xcode app using the _App Store_ tool, run Xcode and install the
2724Developers Tools package within Xcode.
2725
2726Maverick
2727^^^^^^^^
2728
2729The RSB works on Maverick and the GNU tools can be built for RTEMS using the
2730Maverick clang LLVM tool chain. You will need to build and install a couple of
2731packages to make the RSB pass the +sb-check+. These are CVS and XZ. You can get
2732these tools from a packaging tool for MacOS such as _MacPorts_ or _HomeBrew_.
2733
2734I do not use 3rd party packaging on MacOS and prefer to build the packages from
2735source using a prefix of '/usr/local'. The XZ package's home page is
2736http://tukaani.org/xz/ and it builds without change. CVS can be found at
2737http://cvs.nongnu.org/ and it requires this patch
2738http://www.rtems.org/ftp/pub/rtems/people/chrisj/source-builder/cvs-1.11.23-osx-maverick.diff
2739to build.
2740
2741Windows
2742~~~~~~~
2743
2744Windows tool sets are supported creating native Windows executable. Native
2745Windows tools are built using a MinGW compiler and do not need any extra
2746libraries or emulation layer to run once built. The tools understand and use
2747standard Windows paths and integrate easly into Windows IDE environments. A
2748shell maybe needed to build other parts of your system however if your
2749development tools are all native Windows tool you can easly integrate these
2750tool sets.
2751
2752.Ready To Go Windows Tools
2753NOTE: I provide tools for Windows at
2754http://www.rtems.org/ftp/pub/rtems/people/chrisj/source-builder/4.11/mingw32/
2755
2756Building on Windows is a little more complicated because the Cygwin shell is
2757used rather than the MinGW MSYS shell. The MSYS shell is simpler because the
2758detected host triple is MinGW so the build is standard cross-compiler build.
2759The age of the MSYS code base, its stability and ability to to complete a build
2760with limitations such as the length of file names support make using MSYS
2761difficult therefore the more complex path of a Canadian cross-build using
2762Cygwin is supported.
2763
2764Install a recent Cygwin version using the Cygwin setup tool. Select and install
2765the groups and packages listed:
2766
2767.Cygwin Packages
2768[options="header,compact",width="50%",cols="20%,80%"]
2769|================================
2770|Group   |Package
2771|Archive |bsdtar
2772|        |unzip
2773|        |xz
2774|Devel   |autoconf
2775|        |autoconf2.1
2776|        |autoconf2.5
2777|        |automake
2778|        |binutils
2779|        |bison
2780|        |flex
2781|        |gcc4-core
2782|        |gcc4-g++
2783|        |git
2784|        |make
2785|        |mingw64-x86_64-binutils
2786|        |mingw64-x86_64-gcc-core
2787|        |mingw64-x86_64-g++
2788|        |mingw64-x86_64-runtime
2789|        |mingw64-x86_64-zlib
2790|        |patch
2791|        |zlib-devel
2792|MinGW   |mingw-zlib-devel
2793|Python  |python
2794|================================
2795
2796The setup tool will add a number of dependent package and it is ok to accept
2797them.
2798
2799I have found turning off Windows Defender improves performance if you have
2800another up to date virus detection tool installed and enabled. I used the
2801excellent `Process Hacker 2` tool to monitor the performance and I found the
2802Windows Defender service contributed a high load. In my case I had a 3rd party
2803virus tool installed so the Windows Defender service was not needed.
2804
2805A Canadian cross-compile (Cxc) is required on Cygwin because the host is Cygwin
2806therefore a traditional cross-compile will result in Cygiwn binaries. With a
2807Canadian cross-compile a Cygwin cross-compiler is built as well as the MinGW
2808RTEMS cross-compiler. The Cygwin cross-compiler is required to build the C
2809runtime for the RTEMS target because we are building under Cygiwn. The build
2810output for an RTEMS 4.10 ARM tool set is:
2811
2812-------------------------------------------------------------
2813chris@cygthing ~/development/rtems/src/rtems-source-builder/rtems
2814$ ../source-builder/sb-set-builder --log=l-arm.txt --prefix=$HOME/development/rtems/4.10 4.10/rtems-arm
2815RTEMS Source Builder - Set Builder, v0.2
2816Build Set: 4.10/rtems-arm
2817config: expat-2.1.0-1.cfg
2818package: expat-2.1.0-x86_64-w64-mingw32-1
2819building: expat-2.1.0-x86_64-w64-mingw32-1
2820reporting: expat-2.1.0-1.cfg -> expat-2.1.0-x86_64-w64-mingw32-1.html
2821config: tools/rtems-binutils-2.20.1-1.cfg
2822package: arm-rtems4.10-binutils-2.20.1-1 <1>
2823building: arm-rtems4.10-binutils-2.20.1-1
2824package: (Cxc) arm-rtems4.10-binutils-2.20.1-1 <2>
2825building: (Cxc) arm-rtems4.10-binutils-2.20.1-1
2826reporting: tools/rtems-binutils-2.20.1-1.cfg ->
2827arm-rtems4.10-binutils-2.20.1-1.html
2828config: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg
2829package: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2830building: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2831package: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2832building: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2833reporting: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg ->
2834arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1.html
2835config: tools/rtems-gdb-7.3.1-1.cfg
2836package: arm-rtems4.10-gdb-7.3.1-1
2837building: arm-rtems4.10-gdb-7.3.1-1
2838reporting: tools/rtems-gdb-7.3.1-1.cfg -> arm-rtems4.10-gdb-7.3.1-1.html
2839config: tools/rtems-kernel-4.10.2.cfg
2840package: arm-rtems4.10-kernel-4.10.2-1
2841building: arm-rtems4.10-kernel-4.10.2-1
2842reporting: tools/rtems-kernel-4.10.2.cfg -> arm-rtems4.10-kernel-4.10.2-1.html
2843installing: expat-2.1.0-x86_64-w64-mingw32-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2844installing: arm-rtems4.10-binutils-2.20.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10 <3>
2845installing: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2846installing: arm-rtems4.10-gdb-7.3.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2847installing: arm-rtems4.10-kernel-4.10.2-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2848cleaning: expat-2.1.0-x86_64-w64-mingw32-1
2849cleaning: arm-rtems4.10-binutils-2.20.1-1
2850cleaning: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2851cleaning: arm-rtems4.10-gdb-7.3.1-1
2852cleaning: arm-rtems4.10-kernel-4.10.2-1
2853Build Set: Time 10:09:42.810547 <4>
2854-------------------------------------------------------------
2855
2856<1> The Cygwin version of the ARM cross-binutils.
2857<2> The +(Cxc)+ indicates this is the MinGW build of the package.
2858<3> Only the MinGW version is installed.
2859<4> Cygwin is slow so please be patient. This time was on an AMD Athlon 64bit
2860    Dual Core 6000+ running at 3GHz with 4G RAM running Windows 7 64bit.
2861
2862CAUTION: Cygwin documents the 'Big List Of Dodgy Apps' or 'BLODA'. The link is
2863http://cygwin.com/faq/faq.html#faq.using.bloda and it is worth a
2864look. You will see a large number of common pieces of software found on Windows
2865systems that can cause problems. My testing has been performed with NOD32
2866running and I have seen some failures. The list is for all of Cygwin so I am
2867not sure which of the listed programs effect the RTEMS Source Biulder. The
2868following FAQ item talks about +fork+ failures and presents some technical
2869reasons they cannot be avoided in all cases. Cygwin and it's fork MSYS are
2870fantastic pieces of software in a difficult environment. I have found building
2871a single tool tends to work, building all at once is harder.
2872
2873
2874Build Status By Host
2875~~~~~~~~~~~~~~~~~~~~
2876
2877This table lists the current build and testing status for reported hosts:
2878
2879[grid="rows",format="csv"]
2880[options="header",cols="<,<,<,<,<,<"]
2881|===========================
2882OS,Uname,4.9,4.10,4.11,Comments
2883include::host-results.csv[]
2884|===========================
2885
2886[NOTE]
2887==================================================================
2888Report any unlisted hosts as a patch.
2889==================================================================
2890
2891History
2892-------
2893
2894The RTEMS Source Builder is a stand alone tool based on another tool called the
2895'SpecBuilder'. The SpecBuilder was written for the RTEMS project to give me a
2896way to build tools on hosts that did not support RPMs. At the time the RTEMS
2897tools maintainer only used spec files to create various packages. This meant I
2898had either spec files, RPM files or SRPM files. The RPM and SPRM files where
2899useless because you needed an 'rpm' type tool to extract and manage them. There
2900are versions of 'rpm' for a number of non-RPM hosts however these proved to be
2901in various broken states and randomly maintained. The solution I settled on was
2902to use spec files so I wrote a Python based tool that parsed the spec file
2903format and allowed me to create a shell script I could run to build the
2904package. This approach proved successful and I was able to track the RPM
2905version of the RTEMS tools on a non-RPM host over a number of years. however
2906the SpecBuilder tool did not help me build tools or other packages not related
2907to the RTEMS project where there was no spec file I could use so I needed
2908another tool. Rather than start again I decided to take the parsing code for
2909the spec file format and build a new tool called the RTEMS Source Builder.
Note: See TracBrowser for help on using the repository browser.