source: rtems-source-builder/doc/source-builder.txt @ 624954b

4.104.114.9
Last change on this file since 624954b was 624954b, checked in by Chris Johns <chrisj@…>, on Apr 26, 2013 at 2:05:53 AM

Add macro include support. Use it for building from head.

This change provides a simple way to build all parts of the tools from
version control.

  • Property mode set to 100644
File size: 91.8 KB
Line 
1RTEMS Source Builder
2====================
3:doctype: book
4:toc2:
5:toclevels: 5
6:icons:
7:numbered:
8
9image:images/rtemswhitebg.jpg["RTEMS",width="30%"]
10
11Chris Johns <chrisj@rtems.org>
121.2, April 2013
13
14RTEMS Tools From Source
15-----------------------
16
17The RTEMS Source Builder is a tool to aid building packages from source. It is
18not a package manager. It helps consolidate the details you need to build a
19package from source in a controlled and verifiable way. The tool is aimed at
20developers of software who use tool sets for embedded type
21development. Embedded development typically uses cross-compiling tool chains,
22debuggers, and debugging aids. Together we call these a 'tool set'. The RTEMS
23Source Builder is not limited to this role but designed to fit with-in this
24specific niche. It can be used outside of the RTEMS project and we welcome this
25happening in other open source or commercial projects.
26
27The RTEMS Source Builder is typically used to build a set of tools or a 'build
28set'. A 'build set' is a collection of packages and a package is a specific
29tool, for example gcc or gdb. The RTEMS Source Builder attempts to support any
30host environment that runs Python and you can build the package on. It is not
31some sort of magic that can take any piece of source code and make it
32build. Someone at some point in time has figured out how to build that package
33from source and taught this tool. The RTEMS Source Builder has been tested on:
34
35* <<_archlinux,Archlinux>>
36* <<_centos,Centos>>
37* <<_fedora,Fedora>>
38* <<_freebsd,FreeBSD>>
39* <<_macos,MacOS>>
40* <<_opensuse,openSUSE>>
41* <<_raspbian,Raspbian>>
42* <<_ubuntu,Ubuntu>>
43* <<_windows,Windows>>
44
45The RTEMS Source Builder has two types configuration data. The first is the
46'build set'. A _build set_ describes a collection of packages that define a set
47of tools you would use when developing software for RTEMS. For example the
48basic GNU tool set is binutils, gcc, and gdb and is the typical base suite of
49tools you need for an embedded cross-development type project. The second type
50of configuration data is the configuration files and they define how a package
51is built. Configuration files are scripts loosely based on the RPM spec file
52format and detail the steps needed to build a package. The steps are
53'preparation', 'building', and 'installing'. Scripts support macros, shell
54expansion, logic, includes plus many more features useful when build packages.
55
56The RTEMS Source Builder does not interact with any host package management
57systems. There is no automatic dependence checking between various packages you
58build or packages and software your host system you may have installed. We
59assume the build sets and configuration files you are using have been created
60by developers who do. If you have a problem please ask on the RTEMS Users
61mailing list.
62
63Quick Start
64-----------
65
66The quick start will show you how to build a set of RTEMS tools for a supported
67architecture.
68
69There is no need to become root or the administrator and we recommend you avoid
70doing this. You can build and install the tools anywhere on the host's file
71system you, as a standard user, have read and write access too. I recommend you
72use your home directory and work under the directory `~/development/rtems`. The
73examples shown here will do this.
74
75You can use the build _prefix_ to install and maintain different versions of
76the tools. Doing this lets you try a new set of tools while not touching your
77proven working production set of tools. Once you have proven the new tools are
78working rebuild with the 'production' prefix switching your development to them.
79
80I also suggest you keep your environment to the bare minimum, particularly the
81path variable. Using environment variables has been proven over the years to be
82difficult to manage in production systems.
83
84Setup
85~~~~~
86
87Setup a development work space:
88
89-------------------------------------------------------------
90$ cd
91$ mkdir -p development/rtems/src
92$ cd development/rtems/src
93-------------------------------------------------------------
94
95The RTEMS Source Builder is distributed as source. It is Python code so all you
96need to do is head over to the RTEMS GIT repository and clone the code directly
97from git:
98
99-------------------------------------------------------------
100$ git clone git://git.rtems.org/chrisj/rtems-source-builder.git
101$ cd rtems-source-builder
102-------------------------------------------------------------
103
104Checking
105~~~~~~~~
106
107The next step is to check if your host is set up correctly. The RTEMS Source
108Builder provides a tool to help:
109
110-------------------------------------------------------------
111$ source-builder/sb-check
112warning: exe: absolute exe found in path: (__objcopy) /usr/local/bin/objcopy <1>
113warning: exe: absolute exe found in path: (__objdump) /usr/local/bin/objdump
114error: exe: not found: (_xz) /usr/local/bin/xz <2>
115RTEMS Source Builder environment is not correctly set up
116$ source-builder/sb-check
117RTEMS Source Builder environment is ok <3>
118-------------------------------------------------------------
119
120<1> A tool is in the environment path but does not match the expected path.
121<2> The executable 'xz' is not found.
122<3> The host's environment is set up correct.
123
124The checking tool will output a list of executable files not found if problems
125are detected. Locate those executable files and install them. You may also be
126given a list of warnings about executable files not in the expected location
127however the executable was located somewhere in your environment's path. You
128will need to check each tool to determine if this is an issue. An executable
129not in the standard location may indicate it is not the host operating system's
130standard tool. It maybe ok or it could be buggy, only you can determine this.
131
132The <<_host_setups,Host Setups>> section lists packages you should install for
133common host operating systems. It maybe worth checking if you have those
134installed.
135
136Build Sets
137~~~~~~~~~~
138
139The RTEMS tools can be built within the RTEMS Source Builder's source tree. We
140recommend you do this so lets change into the RTEMS directory in the RTEMS
141Source Builder package:
142
143-------------------------------------------------------------
144$ cd rtems
145-------------------------------------------------------------
146
147If you are unsure how to specify the build set for the architecture you wish to
148build ask the tool:
149
150-------------------------------------------------------------
151$ ../source-builder/sb-set-builder --list-bsets <1>
152RTEMS Source Builder - Set Builder, v0.2.0
153Examining: config <2>
154Examining: ../source-builder/config <2>
155    4.10/rtems-all.bset <3>
156    4.10/rtems-arm.bset <4>
157    4.10/rtems-autotools.bset
158    4.10/rtems-avr.bset
159    4.10/rtems-bfin.bset
160    4.10/rtems-h8300.bset
161    4.10/rtems-i386.bset
162    4.10/rtems-lm32.bset
163    4.10/rtems-m32c.bset
164    4.10/rtems-m32r.bset
165    4.10/rtems-m68k.bset
166    4.10/rtems-mips.bset
167    4.10/rtems-nios2.bset
168    4.10/rtems-powerpc.bset
169    4.10/rtems-sh.bset
170    4.10/rtems-sparc.bset
171    4.11/rtems-all.bset
172    4.11/rtems-arm.bset
173    4.11/rtems-autotools.bset
174    4.11/rtems-avr.bset
175    4.11/rtems-bfin.bset
176    4.11/rtems-h8300.bset
177    4.11/rtems-i386.bset
178    4.11/rtems-lm32.bset
179    4.11/rtems-m32c.bset
180    4.11/rtems-m32r.bset
181    4.11/rtems-m68k.bset
182    4.11/rtems-microblaze.bset
183    4.11/rtems-mips.bset
184    4.11/rtems-moxie.bset
185    4.11/rtems-nios2.bset
186    4.11/rtems-powerpc.bset
187    4.11/rtems-sh.bset
188    4.11/rtems-sparc.bset
189    4.11/rtems-sparc64.bset
190    4.11/rtems-v850.bset
191    4.9/rtems-all.bset
192    4.9/rtems-arm.bset
193    4.9/rtems-autotools.bset
194    4.9/rtems-i386.bset
195    4.9/rtems-m68k.bset
196    4.9/rtems-mips.bset
197    4.9/rtems-powerpc.bset
198    4.9/rtems-sparc.bset
199    gnu-tools-4.6.bset
200    rtems-4.10-base.bset <5>
201    rtems-4.11-base.bset
202    rtems-4.9-base.bset
203    rtems-base.bset <5>
204    unstable/4.11/rtems-all.bset <6>
205    unstable/4.11/rtems-arm.bset
206    unstable/4.11/rtems-avr.bset
207    unstable/4.11/rtems-bfin.bset
208    unstable/4.11/rtems-h8300.bset
209    unstable/4.11/rtems-i386.bset
210    unstable/4.11/rtems-lm32.bset
211    unstable/4.11/rtems-m32c.bset
212    unstable/4.11/rtems-m32r.bset
213    unstable/4.11/rtems-m68k.bset
214    unstable/4.11/rtems-microblaze.bset
215    unstable/4.11/rtems-mips.bset
216    unstable/4.11/rtems-moxie.bset
217    unstable/4.11/rtems-powerpc.bset
218    unstable/4.11/rtems-sh.bset
219    unstable/4.11/rtems-sparc.bset
220    unstable/4.11/rtems-sparc64.bset
221    unstable/4.11/rtems-v850.bset
222-------------------------------------------------------------
223<1> Only option needed is +--list-bsets+
224<2> The paths inspected. See <<X1,+_configdir+>> variable.
225<3> Build all RTEMS 4.10 supported architectures.
226<4> The build set for the ARM architecture on RTEMS 4.10.
227<5> Support build set file with common functionality included by other build
228    set files.
229<6> Unstable tool sets are used by RTEMS developers to test new tool sets. You
230    are welcome to try them but you must remember they are unstable, can change
231    at any point in time and your application may not work. If you have an
232    issue with a production tool it may pay to try the unstable tool to see if
233    it has been resolved.
234
235Building
236~~~~~~~~
237
238In this quick start I will build a SPARC tool set.
239
240-------------------------------------------------------------
241$ ../source-builder/sb-set-builder --log=l-sparc.txt <1> \
242                --prefix=$HOME/development/rtems/4.11 <2> 4.11/rtems-sparc <3>
243Source Builder - Set Builder, v0.2.0
244Build Set: 4.11/rtems-sparc
245config: expat-2.1.0-1.cfg <4>
246package: expat-2.1.0-x86_64-freebsd9.1-1
247building: expat-2.1.0-x86_64-freebsd9.1-1
248config: tools/rtems-binutils-2.22-1.cfg <5>
249package: sparc-rtems4.11-binutils-2.22-1
250building: sparc-rtems4.11-binutils-2.22-1
251config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg <6>
252package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
253building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
254config: tools/rtems-gdb-7.5.1-1.cfg <7>
255package: sparc-rtems4.11-gdb-7.5.1-1
256building: sparc-rtems4.11-gdb-7.5.1-1
257installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <8>
258installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
259installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
260installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
261cleaning: expat-2.1.0-x86_64-freebsd9.1-1 <9>
262cleaning: sparc-rtems4.11-binutils-2.22-1
263cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
264cleaning: sparc-rtems4.11-gdb-7.5.1-1
265Build Set: Time 0:13:43.616383 <10>
266-------------------------------------------------------------
267
268<1> Providing a log file redirects the build output into a file. Logging the
269    build output provides a simple way to report problems.
270<2> The prefix is the location on your file system the tools are installed
271    into. Provide a prefix to a location you have read and write access. You
272    can use the prefix to install different versions or builds of tools. Just
273    use the path to the tools you want to use when building RTEMS.
274<3> The build set. This is the SPARC build set.
275<4> The SPARC build set first builds the expat library as is used in GDB. This
276    is the expat configuration used.
277<5> The binutils build configuration.
278<6> The GCC and Newlib build configuration.
279<7> The GDB build configuration.
280<8> Installing the built packages to the install prefix.
281<9> All the packages built are cleaned at the end. If the build fails all the
282    needed files are present. You may have to clean up by deleting the build
283    directory if the build fails.
284<10> The time to build the package. This lets you see how different host
285     hardware or configurations perform.
286
287Your SPARC RTEMS 4.11 tool set will be installed and ready to build RTEMS and
288RTEMS applications. When the build runs you will notice the tool fetch the
289source code from the internet. These files are cached in a directory called
290+source+. If you run the build again the cached files are used. This is what
291happened in the show example before.
292
293The installed tools:
294
295-------------------------------------------------------------
296$ ls $HOME/development/rtems/4.11
297bin         include     lib         libexec     share       sparc-rtems4.11
298$ ls $HOME/development/rtems/4.11/bin
299sparc-rtems4.11-addr2line       sparc-rtems4.11-cpp
300sparc-rtems4.11-gcc-ar          sparc-rtems4.11-gprof
301sparc-rtems4.11-objdump         sparc-rtems4.11-size
302sparc-rtems4.11-ar              sparc-rtems4.11-elfedit
303sparc-rtems4.11-gcc-nm          sparc-rtems4.11-ld
304sparc-rtems4.11-ranlib          sparc-rtems4.11-strings
305sparc-rtems4.11-as              sparc-rtems4.11-g++
306sparc-rtems4.11-gcc-ranlib      sparc-rtems4.11-ld.bfd
307sparc-rtems4.11-readelf         sparc-rtems4.11-strip
308sparc-rtems4.11-c++             sparc-rtems4.11-gcc
309sparc-rtems4.11-gcov            sparc-rtems4.11-nm
310sparc-rtems4.11-run             xmlwf
311sparc-rtems4.11-c++filt         sparc-rtems4.11-gcc-4.7.2
312sparc-rtems4.11-gdb             sparc-rtems4.11-objcopy
313sparc-rtems4.11-sis
314$ $HOME/development/rtems/4.11/bin/sparc-rtems4.11-gcc -v
315Using built-in specs.
316COLLECT_GCC=/home/chris/development/rtems/4.11/bin/sparc-rtems4.11-gcc
317COLLECT_LTO_WRAPPER=/usr/home/chris/development/rtems/4.11/bin/../ \
318libexec/gcc/sparc-rtems4.11/4.7.2/lto-wrapper
319Target: sparc-rtems4.11 <1>
320Configured with: ../gcc-4.7.2/configure <2>
321--prefix=/home/chris/development/rtems/4.11
322--bindir=/home/chris/development/rtems/4.11/bin
323--exec_prefix=/home/chris/development/rtems/4.11
324--includedir=/home/chris/development/rtems/4.11/include
325--libdir=/home/chris/development/rtems/4.11/lib
326--libexecdir=/home/chris/development/rtems/4.11/libexec
327--mandir=/home/chris/development/rtems/4.11/share/man
328--infodir=/home/chris/development/rtems/4.11/share/info
329--datadir=/home/chris/development/rtems/4.11/share
330--build=x86_64-freebsd9.1 --host=x86_64-freebsd9.1 --target=sparc-rtems4.11
331--disable-libstdcxx-pch --with-gnu-as --with-gnu-ld --verbose --with-newlib
332--with-system-zlib --disable-nls --without-included-gettext
333--disable-win32-registry --enable-version-specific-runtime-libs --disable-lto
334--enable-threads --enable-plugin --enable-newlib-io-c99-formats
335--enable-newlib-iconv --enable-languages=c,c++
336Thread model: rtems <3>
337gcc version 4.7.2 20120920 <4>
338 (RTEMS4.11-RSB(cb12e4875c203f794a5cd4b3de36101ff9a88403)-1,gcc-4.7.2/newlib-2.0.0) (GCC)
339-------------------------------------------------------------
340
341<1> The target the compiler is built for.
342<2> The configure options used to build GCC.
343<3> The threading model is always RTEMS. This makes the RTEMS tools difficult
344    for bare metal development more difficult.
345<4> The version string. It contains the Git hash of the RTEMS Source Builder
346    you are using. If your local clone has been modifed that state is also
347    recorded in the version string. The hash allows you to track from a GCC
348    executable back to the original source used to build it.
349
350NOTE: The RTEMS thread model enables specific hooks in GCC so applications
351built with RTEMS tools need the RTEMS runtime to operate correctly. You can use
352RTEMS tools to build bare metal component but it is more difficult than with a
353bare metal tool chain and you need to know what you are doing at a low
354level. The RTEMS Source Builder can build bare metal tool chains.
355
356Installing and Tar Files
357~~~~~~~~~~~~~~~~~~~~~~~~
358
359The RTEMS Source Builder can install the built packages directly and optionally
360it can create a build set tar file or a tar file per package built. The normal
361and default behaviour is to let the RTEMS Source Builder install the tools. The
362source will be downloaded, built, installed and cleaned up.
363
364The tar files are created with the full build prefix present and if you follow
365the examples given in this documentation the path is absolute. This can cause
366problems if you are installing on a host you do not have super user or
367administrator rights on because the prefix path may references part you do not
368have write access too and tar will not extract the files. You can use the
369+--strip-components+ option in tar if your host tar application supports it to
370remove the parts you do not have write access too or you may need to unpack the
371tar file somewhere and copy the file tree from the level you have write access
372from. Embedding the full prefix path in the tar files lets you know what the
373prefix is and is recommended. For example if
374`/home/chris/development/rtems/4.11` is the prefix used you cannot change
375directory to the root (`/`) and install because the `/home` is root access
376only. To install you would:
377
378-------------------------------------------------------------
379$ cd
380$ tar --strip-components=3 -xjf rtems-4.11-sparc-rtems4.11-1.tar.bz2
381-------------------------------------------------------------
382
383A build set tar file is created by adding `--bset-tar-file` option to the
384`sb-set-builder` command.
385
386-------------------------------------------------------------
387$ ../source-builder/sb-set-builder --log=l-sparc.txt \
388         --prefix=$HOME/development/rtems/4.11 \
389         --bset-tar-file <1> 4.11/rtems-sparc
390Source Builder - Set Builder, v0.2.0
391Build Set: 4.11/rtems-sparc
392config: expat-2.1.0-1.cfg
393package: expat-2.1.0-x86_64-freebsd9.1-1
394building: expat-2.1.0-x86_64-freebsd9.1-1
395config: tools/rtems-binutils-2.22-1.cfg
396package: sparc-rtems4.11-binutils-2.22-1
397building: sparc-rtems4.11-binutils-2.22-1
398config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
399package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
400building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
401config: tools/rtems-gdb-7.5.1-1.cfg
402package: sparc-rtems4.11-gdb-7.5.1-1
403building: sparc-rtems4.11-gdb-7.5.1-1
404installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <2>
405installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
406installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
407installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
408tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
409cleaning: expat-2.1.0-x86_64-freebsd9.1-1
410cleaning: sparc-rtems4.11-binutils-2.22-1
411cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
412cleaning: sparc-rtems4.11-gdb-7.5.1-1
413Build Set: Time 0:15:25.92873
414-------------------------------------------------------------
415
416<1> The option to create a build set tar file.
417<2> The installation still happens unless you specify `--no-install`.
418<3> Creating the build set tar file.
419
420You can also suppress installing the files using the `--no-install` option to
421the `sb-set-builder` command.
422
423-------------------------------------------------------------
424$ ../source-builder/sb-set-builder --log=l-sparc.txt \
425            --prefix=$HOME/development/rtems/4.11 \
426            --bset-tar-file --no-install <1> 4.11/rtems-sparc
427Source Builder - Set Builder, v0.2.0
428Build Set: 4.11/rtems-sparc
429config: expat-2.1.0-1.cfg
430package: expat-2.1.0-x86_64-freebsd9.1-1
431building: expat-2.1.0-x86_64-freebsd9.1-1
432config: tools/rtems-binutils-2.22-1.cfg
433package: sparc-rtems4.11-binutils-2.22-1
434building: sparc-rtems4.11-binutils-2.22-1
435config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
436package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
437building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
438config: tools/rtems-gdb-7.5.1-1.cfg
439package: sparc-rtems4.11-gdb-7.5.1-1
440building: sparc-rtems4.11-gdb-7.5.1-1
441tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <2>
442cleaning: expat-2.1.0-x86_64-freebsd9.1-1
443cleaning: sparc-rtems4.11-binutils-2.22-1
444cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
445cleaning: sparc-rtems4.11-gdb-7.5.1-1
446Build Set: Time 0:14:11.721274
447$ ls tar
448rtems-4.11-sparc-rtems4.11-1.tar.bz2
449-------------------------------------------------------------
450
451<1> The option to supressing installing the packages.
452<2> Create the build set tar.
453
454A package tar file can be created by adding the +--pkg-tar-files+ to the
455+sb-set-builder+ command. This creates a tar file per package built in the
456build set.
457
458-------------------------------------------------------------
459$ ../source-builder/sb-set-builder --log=l-sparc.txt \
460            --prefix=$HOME/development/rtems/4.11 \
461            --bset-tar-file --pkg-tar-files <1> --no-install 4.11/rtems-sparc
462Source Builder - Set Builder, v0.2.0
463Build Set: 4.11/rtems-sparc
464config: expat-2.1.0-1.cfg
465package: expat-2.1.0-x86_64-freebsd9.1-1
466building: expat-2.1.0-x86_64-freebsd9.1-1
467config: tools/rtems-binutils-2.22-1.cfg
468package: sparc-rtems4.11-binutils-2.22-1
469building: sparc-rtems4.11-binutils-2.22-1
470config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
471package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
472building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
473config: tools/rtems-gdb-7.5.1-1.cfg
474package: sparc-rtems4.11-gdb-7.5.1-1
475building: sparc-rtems4.11-gdb-7.5.1-1
476tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2
477cleaning: expat-2.1.0-x86_64-freebsd9.1-1
478cleaning: sparc-rtems4.11-binutils-2.22-1
479cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
480cleaning: sparc-rtems4.11-gdb-7.5.1-1
481Build Set: Time 0:14:37.658460
482$ ls tar
483expat-2.1.0-x86_64-freebsd9.1-1.tar.bz2           sparc-rtems4.11-binutils-2.22-1.tar.bz2
484sparc-rtems4.11-gdb-7.5.1-1.tar.bz2 <2>           rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
485sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1.tar.bz2
486-------------------------------------------------------------
487
488<1> The option to create packages tar files.
489<2> The GDB package tar file.
490<3> The build set tar file. All the others in a single tar file.
491
492Why Build from Source ?
493-----------------------
494
495The RTEMS Source Builder is not a replacement for the binary install systems
496you have with commercial operating systems or open source operating system
497distributions. Those products and distributions are critically important and
498are the base that allows the Source Builder to work. The RTEMS Source Builder
499sits somewhere between you manually entering the commands to build a tool set
500and a tool such as +yum+ or +apt-get+ to install binary packages made
501specifically for your host operating system. Building manually or installing a
502binary package from a remote repository are valid and real alternatives while
503the Source Builder is attempting to provide a specific service of repeatably
504being able to build tool sets from source code.
505
506If you are developing a system or product that has a long shelf life or is used
507in a critical piece of infrastructure that has a long life cycle being able to
508build from source is important. It insulates the project from the fast ever
509changing world of the host development machines. If your tool set is binary and
510you have lost the ability to build it you have lost a degree of control and
511flexibility open source gives you. Fast moving host environments are
512fantastic. We have powerful multi-core computers with huge amounts of memory
513and state of the art operating systems to run on them however the product or
514project you are part of may need to be maintained well past the life time of
515these host. Being able to build from source an important and critical part of
516this process because you can move to a newer host and create an equivalent tool
517set.
518
519Building from source provides you with control over the configuration of the
520package you are building. If all or the most important dependent parts are
521built from source you limit the exposure to host variations. For example the
522GNU C compiler (gcc) currently uses a number of 3rd party libraries internally
523(gmp, mpfr, etc). If your validated compiler generating code for your target
524processor is dynamically linked against the host's version of these libraries
525any change in the host's configuration may effect you. The changes the host's
526package management system makes may be perfectly reasonable in relation to the
527distribution being managed however this may not extend to you and your
528tools. Building your tools from source and controlling the specific version of
529these dependent parts means you are not exposing yourself to unexpected and
530often difficult to resolve problems. On the other side you need to make sure
531your tools build and work with newer versions of the host operating
532system. Given the stability of standards based libraries like 'libc' and ever
533improving support for standard header file locations this task is becoming
534easier.
535
536The RTEMS Source Builder is designed to be audited and incorporated into a
537project's verification and validation process. If your project is developing
538critical applications that needs to be traced from source to executable code in
539the target, you need to also consider the tools and how to track them.
540
541If your IT department maintains all your computers and you do not have suitable
542rights to install binary packages, building from source lets you create your
543own tool set that you install under your home directory. Avoiding installing
544any extra packages as a super user is always helpful in maintaining a secure
545computing environment.
546
547Bugs, Crashes, and Build Failures
548---------------------------------
549
550The RTEMS Source Builder is a Python program and every care is taken to test
551the code however bugs, crashes, and build failure can and do happen. If you
552find a bug please report it via the RTEMS Bug tracker tool Bugzilla:
553
554https://www.rtems.org/bugzilla/
555
556or via email on the RTEMS Users list.
557
558Please include the git hash you are running, the host details with the output
559of the +uname -a+ command. If there is a crash please cut and paste the Python
560backtrace into the bug report. If the tools fail to build please locate the
561first error in the log file. This can be difficult to find on hosts with many
562cores so it sometimes pays to run the command with the +--jobs=none+ option to
563get a log that is correctly sequenced.
564
565Configuration
566-------------
567
568The RTEMS Source Builder has two types of configuration data:
569
570. Build Sets
571. Package Build Configurations
572
573By default these files can be located in two separate directories and
574searched. The first directory is +config+ in your current working directory
575(+_topdir+) and the second is +config+ located in the base directory of the
576RTEMS Source Builder command you run (+_sbdir+). The RTEMS directory +rtems+
577located at the top of the RTEMS Source Builder source code is an example of a
578specific build configuration directory. You can create custom or private build
579configurations and if you run the RTEMS Source Builder command from that
580directory your configurations will be used.
581
582[[X1]] The configuration search path is a macro variable and is reference as
583+%\{_configdir\}+. It's default is defined as:
584
585-------------------------------------------------------------
586_configdir          : dir      optional   %{_topdir}/config:%{_sbdir}/config <1>
587-------------------------------------------------------------
588
589<1> The +_topdir+ is the directory you run the command from and +_sbdir+ is the
590location of the RTEMS Source Builder command.
591
592Build set files have the file extension +.bset+ and the package build
593configuration files have the file extension of +.cfg+. The +sb-set-builder+
594command will search for _build sets_ and the +sb-builder+ commands works with
595package build configuration files.
596
597Both types of configuration files use the \'#' character as a comment
598character. Anything after this character on the line is ignored. There is no
599block comment.
600
601Source and Patches
602~~~~~~~~~~~~~~~~~~
603
604The RTEMS Source Builder provides a flexible way to manage source. Source and
605patches are declare in configurations file using the +source+ and +patch+
606directives. There are a single line containing a Universal Resource Location or
607URL and can contain macros and shell expansions. The <<_prep,%prep>> section
608details the source and patch directives
609
610The URL can reference remote and local source and patch resources. The
611following schemes are provided:
612
613'http':: Remote access using the HTTP protocol.
614'https':: Remote access using the Secure HTTP protocol.
615'ftp:: Remote access using the FTP protocol.
616'git:: Remote access to a GIT repository.
617'cvs:: Remote access to a CVS repository.
618'file:: Local access to an existing source directory.
619
620HTTP, HTTPS, and FTP
621^^^^^^^^^^^^^^^^^^^^
622
623Remote access to TAR files is provided using HTTP, HTTPS and FTP protocols. The
624full URL provided is used to access the remote file including any query
625components. The URL is parsed to extract the file component and the local
626source directory is checked for that file. If the file is located the remote
627file is not downloaded. Currently no other checks are made. If a download fails
628you need to manually remove the file from the source directory and start the
629build process again.
630
631The URL can contain macros. These are expand before issuing the request to
632download the file. The GNU GCC compiler source URL is:
633
634-------------------------------------------------------------
635Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
636-------------------------------------------------------------
637
638The type of compression is automatically detected from the file extension. The
639supported compression formats are:
640
641`gz`:: GNU ZIP
642`bzip2`:: BZIP2 ??
643`zip`:: ??
644'xy':: XY ??
645
646The output of the decompression tool is feed to the standard `tar` utility and
647unpacked into the build directory.
648
649If the URL references the GitHub API server 'https://api.github.com/' a tarball
650of the specified version is download. For example the URL for the STLINK
651project on GitHub and version is:
652
653-------------------------------------------------------------
654%define stlink_version 3494c11
655Source0: https://api.github.com/repos/texane/stlink/texane-stlink-%{stlink_version}.tar.gz
656-------------------------------------------------------------
657
658the TAR file is downloaded and used.
659
660GIT
661^^^
662
663A GIT repository can be cloned and used as source. The GIT repository resides
664in the 'source' directory under the `git` directory. You can edit, update and
665use the repository as you normally do and the results will used to build the
666tools. This allows you to prepare and test patches in the build environment the
667tools are built in. The GIT URL only supports the GIT protocol. You can control
668the repository via the URL by appending options and arguments to the GIT
669path. The options are delimited by `?` and option arguments are delimited from
670the options with `=`. The options are:
671
672`branch`:: Checkout the specified branch.
673`pull`:: Perform a pull to update the repository.
674`fetch`:: Perform a fetch to get any remote updates.
675`reset`:: Reset the repository. Useful to remove any local changes. You can
676pass the `hard` argument to force a hard reset.
677
678-------------------------------------------------------------
679Source0: git://gcc.gnu.org/git/gcc.git?branch=gcc-4_7-branch?reset=hard
680-------------------------------------------------------------
681
682This will clone the GCC git repository and checkout the 4.7-branch and perform
683a hard reset.
684
685CVS
686^^^
687
688A CVS repository can be checked out. CVS is more complex than GIT to handle
689because of the modules support. This can effect the paths the source ends up
690in. The CVS URL only supports the CVS protocol. You can control the repository
691via the URL by appending options and arguments to the CVS path. The options are
692delimited by `?` and option arguments are delimited from the options with
693`=`. The options are:
694
695`module`:: The module to checkout.
696`src-prefix`:: The path into the source where the module starts.
697`tag`:: The CVS tag to checkout.
698`date`:: The CVS date to checkout.
699
700-------------------------------------------------------------
701Source0: cvs://pserver:anoncvs@sourceware.org/cvs/src?module=newlib?src-prefix=src
702-------------------------------------------------------------
703
704Macros and Defaults
705~~~~~~~~~~~~~~~~~~~
706
707The RTEMS Source Builder uses tables of _macros_ read in when the tool
708runs. The initial global set of macros is called the _defaults_. These values
709are read from a file called `defaults.mc` and modified to suite your host. This
710host specific adaption lets the Source Builder handle differences in the build
711hosts.
712
713Build set and configuration files can define new values updating and extending
714the global macro table. For example builds are given a release number. This is
715typically a single number at the end of the package name. For example:
716
717-------------------------------------------------------------
718%define release 1
719-------------------------------------------------------------
720
721Once defined if can be accessed in a build set or package configuration file
722with:
723
724-------------------------------------------------------------
725%{release}
726-------------------------------------------------------------
727
728The +sb-defaults+ command lists the defaults for your host. I will not include
729the output of this command becauses of its size.
730
731-------------------------------------------------------------
732$ ../source-builder/sb-defaults
733-------------------------------------------------------------
734
735A nested build set is given a separate copy of the global macro maps. Changes
736in one change set are not seen in other build sets. That same happens with
737configuration files unless inline includes are used. Inline includes are seen
738as part of the same build set and configuration and changes are global to that
739build set and configuration.
740
741Macro Maps and Files
742^^^^^^^^^^^^^^^^^^^^
743
744Macros are read in from files when the tool starts. The default settings are
745read from the defaults macro file called `defaults.mc` located in the top level
746RTEMS Source Builder command directory. User macros can be read in at start up
747by using the `--macros` command line option.
748
749The format for a macro in macro files is:
750
751[options="header,compact",width="50%",cols="15%,15%,15%,55%"]
752|=================================
753| Name | Type | Attribute | String
754|=================================
755
756where 'Name' is a case insensitive macro name, the 'Type' field is:
757
758[horizontal]
759`none`:: Nothing, ignore.
760`dir`:: A directory path.
761`exe`:: An executable path.
762`triplet`:: A GNU style architecture, platform, operating system string.
763
764the 'Attribute' field is:
765
766[horizontal]
767`none`:: Nothing, ignore
768`required`:: The host check must find the executable or path.
769`optional`:: The host check generates a warning if not found.
770`override`:: Only valid outside of the `global` map to indicate this macro
771             overrides the same one in the `global` map when the map containing
772             it is selected.
773`undefine`:: Only valid outside of the `global` map to undefine the macro if it
774             exists in the `global` map when the map containing it is
775             selected. The `global` map's macro is not visible but still
776             exists.
777
778and the 'String' field is a single or tripled multiline quoted string. The
779'String' can contain references to other macros. Macro that loop are not
780currently detected and will cause the tool to lock up.
781
782Maps are declared anywhere in the map using the map directive:
783
784-------------------------------------------------------------
785# Comments
786[my-special-map] <1>
787_host:  none, override, 'abc-xyz'
788multiline: none, override, '''First line,
789second line,
790and finally the last line'''
791-------------------------------------------------------------
792<1> The map is set to `my-special-map`.
793
794Any macro defintions following a map declaration are placed in that map and the
795default map is `global` when loading a file. Maps are selected in configuration
796files by using the `%select` directive.
797
798-------------------------------------------------------------
799%select my-special-map
800-------------------------------------------------------------
801
802Selecting a map means all requests for a macro first check the selected map and
803if present return that value else the `global` map is used. Any new macros or
804changes update only the `global` map. This may change in future releases so
805please make sure you use the `override` attibute.
806
807The macro files specificed on the command line are looked for in the
808`_configdir` paths. See <<X1,+_configdir+>> variable for details. Included
809files need to add the `%{_configdir}` macro to the start of the file.
810
811Macro map files can include other macro map files using the `%include`
812directive. The macro map to build _binutils_, _gcc_, _newlib_, _gdb_ and
813_RTEMS_ from version control heads is:
814
815-------------------------------------------------------------
816# <1>
817# Build all tool parts from version control head.
818#
819%include %{_configdir}/snapshots/binutils-head.mc
820%include %{_configdir}/snapshots/gcc-head.mc
821%include %{_configdir}/snapshots/newlib-head.mc
822%include %{_configdir}/snapshots/gdb-head.mc
823-------------------------------------------------------------
824<1> The file is `config/snapshots/binutils-gcc-newlib-gdb-head.mc`.
825
826The macro map defaults to `global` at the start of each included file and the
827map setting of the macro file including the other macro files does not change.
828
829Build Set Files
830~~~~~~~~~~~~~~~
831
832Build set files lets you list the packages in the build set you are defining
833and have a file extension of +.bset+. Build sets can define macro variables,
834inline include other files and reference other build set or package
835configuration files.
836
837Defining macros is performed with the +%define+ macro:
838
839-------------------------------------------------------------
840%define _target m32r-rtems4.11
841-------------------------------------------------------------
842
843Inline including another file with the +%include+ macro continues processing
844with the specified file returning to carry on from just after the include
845point.
846
847-------------------------------------------------------------
848%include rtems-4.11-base.bset
849-------------------------------------------------------------
850
851This includes the RTEMS 4.11 base set of defines and checks. The configuration
852paths as defined by +_configdir+ are scanned. The file extension is optional.
853
854You reference build set or package configuration files by placing the file name
855on a single line.
856
857-------------------------------------------------------------
858tools/rtems-binutils-2.22-1
859-------------------------------------------------------------
860
861The +_configdir+ path is scanned for +tools/rtems-binutils-2.22-1.bset+ or
862+tools/rtems-binutils-2.22-1.cfg+. Build set files take precedent over package
863configuration files. If +tools/rtems-binutils-2.22-1+ is a build set a new
864instance of the build set processor is created and if the file is a package
865configuration the package is built with the package builder. This all happens
866once the build set file has finished being scanned.
867
868Configuration Control
869~~~~~~~~~~~~~~~~~~~~~
870
871The RTEMS Souce Builder is designed to fit within most verification and
872validation processes. All of the RTEMS Source Builder is source code. The
873Python code is source and comes with a commercial friendly license. All
874configuration data is text and can be read or parsed with standard text based
875tools.
876
877File naming provides configuration management. A specific version of a package
878is captured in a specific set of configuration files. The top level
879configuration file referenced in a _build set_ or passed to the +sb-builder+
880command relates to a specific configuration of the package being built. For
881example the RTEMS configuration file +rtems-gcc-4.7.2-newlib-2.0.0-1.cfg+
882creates an RTEMS GCC and Newlib package where the GCC version is 4.7.2, the
883Newlib version is 2.0.0, plus any RTEMS specific patches that related to this
884version. The configuration defines the version numbers of the various parts
885that make up this package:
886
887-------------------------------------------------------------
888%define gcc_version    4.7.2
889%define newlib_version 2.0.0
890%define mpfr_version   3.0.1
891%define mpc_version    0.8.2
892%define gmp_version    5.0.5
893-------------------------------------------------------------
894
895The package build options, if there are any are also defined:
896
897-------------------------------------------------------------
898%define with_threads 1
899%define with_plugin  0
900%define with_iconv   1
901-------------------------------------------------------------
902
903The generic configuration may provide defaults in case options are not
904specified. The patches this specific version of the package requires can be
905included:
906
907-------------------------------------------------------------
908Patch0: gcc-4.7.2-rtems4.11-20121026.diff
909-------------------------------------------------------------
910
911Finally including the GCC 4.7 configuration script:
912
913-------------------------------------------------------------
914%include %{_configdir}/gcc-4.7-1.cfg
915-------------------------------------------------------------
916
917The +gcc-4.7-1.cfg+ file is a generic script to build a GCC 4.7 compiler with
918Newlib. It is not specific to RTEMS. A bare no operating system tool set can be
919built with this file.
920
921The +-1+ part of the file names is a revision. The GCC 4.7 script maybe revised
922to fix a problem and if this fix effects an existing script the file is copied
923and given a +-2+ revision number. Any dependent scripts referencing the earlier
924revision number will not be effected by the change. This locks down a specific
925configuration over time.
926
927Snapshot Testing
928~~~~~~~~~~~~~~~~
929
930Testing of release canidates and snapshots is important to those helping
931maintain tool sets. The RTEMS Source Builder helps by providing a simple and
932flexible way to use existing build sets and configuration without needing to
933change them or creating new temporary build sets and configurations.
934
935The process uses snapshot macro files loaded via the command line option
936`--macros`. These files provide macros that override the standard build set and
937configuration file macros.
938
939Lets consider testing a GCC 4.7 snapshot for RTEMS 4.11. Lets assume the
940current RTEMS 4.11 tools reference GCC 4.7.3 with a patch as the stable tool
941set. We want to use a recent snapshot with no patches. In the
942`rtems/config/snapshots` directoy create a file called `gcc-4.7-snapshot.mc`
943containing:
944
945-------------------------------------------------------------
946[gcc-4.7-snapshot]
947GCC_Version: none, override, '4.7-20130413'
948Source0:     none, override, 'http://mirrors.kernel.org/sources.redhat.com/gcc/
949snapshots/%{gcc_version}/gcc-%{gcc_version}.tar.bz2'
950Patch0:      none, udefine,  ''
951-------------------------------------------------------------
952
953In the standard configuration file `source-builder/config/gcc-4.7-1.cfg` the
954map is selected with:
955
956-------------------------------------------------------------
957#
958# Select the GCC 4.7 Snapshot Macro Map
959#
960%select gcc-4.7-snapshot
961-------------------------------------------------------------
962
963On the command line add `--macros=snapshots/gcc-4.7-snapshot.mc` and this
964snapshot will be built. With careful use of the `--prefix` option you can
965locate the tools in a specific directory and test them without needing to
966effect your production environment.
967
968Scripting
969~~~~~~~~~
970
971Configuration files specify how to build a package. Configuration files are
972scripts and have a +.cfg+ file extension. The script format is based loosely on
973the RPM spec file format however the use and purpose in this tool does not
974compare with the functionality and therefore the important features of the spec
975format RPM needs and uses.
976
977The script language is implemented in terms of macros. The built-in list is:
978
979[horizontal]
980+%{}+:: Macro expansion with conditional logic.
981+%()+:: Shell expansion.
982+%prep+:: The source preparation shell commands.
983+%build+:: The build shell commands.
984+%install+:: The package install shell commands.
985+%clean+:: The package clean shell commands.
986+%include+:: Inline include another configuration file.
987+%name+:: The name of the package.
988+%summary+:: A brief package description. Useful when reporting about a build.
989+%release+:: The package release. A number that is the release as built by this tool.
990+%version+:: The package's version string.
991+%buildarch+:: The build architecture.
992+%setup+:: Setup a source package.
993+%source+:: Define a source code package. This macro has a number appended.
994+%patch+:: Define a patch. This macro has a is number appended.
995+%echo+:: Print the following string as a message.
996+%warning+:: Print the following string as a warning and continue.
997+%error+:: Print the following string as an error and exit.
998+%select+:: Select the macro map. If there is no map nothing is reported.
999+%define+:: Define a macro. Macros cannot be redefined, you must first undefine it.
1000+%undefine+:: Undefine a macro.
1001+%if+:: Start a conditional logic block that ends with a +%endif+.
1002+%ifn+:: Inverted start of a conditional logic block.
1003+%ifarch+:: Test the architecture against the following string.
1004+%ifnarch+:: Inverted test of the architecture
1005+%ifos+:: Test the host operating system.
1006+%else+:: Start the _else_ conditional logic block.
1007+%endfi+:: End the conditional logic block.
1008+%bconf_with+:: Test the build condition _with_ setting. This is the +--with-*+
1009command line option.
1010+%bconf_without+:: Test the build condition _without_ setting. This is the
1011+--without-*+ command line option.
1012
1013Expanding
1014^^^^^^^^^
1015
1016A macro can be `%{string}` or the equivalent of `%string`. The following macro
1017expansions supported are:
1018
1019`%{string}`;;
1020Expand the 'string' replacing the entire macro text with the text in the table
1021for the entry 'string . For example if 'var' is 'foo' then `${var}` would
1022become `foo`.
1023
1024`%{expand: string}`;;
1025Expand the 'string' and then use it as a ``string'' to the macro expanding the
1026macro. For example if _foo_ is set to 'bar' and 'bar' is set to 'foobar' then
1027`%{expand:foo}` would result in `foobar`. Shell expansion can also be used.
1028
1029`%{with string}`;;
1030Expand the macro to '1' if the macro `with_`'string' is defined else expand to
1031_0_. Macros with the name `with_`'string' can be define with command line
1032arguments to the RTEMS Source Builder commands.
1033
1034`%{defined string}`;;
1035Expand the macro to '1' if a macro of name 'string' is defined else expand to '0'.
1036
1037`%{?string: expression}`;;
1038Expand the macro to 'expression' if a macro of name 'string' is defined else expand to `%{nil}`.
1039
1040`%{!?string: expression}`;;
1041Expand the macro to 'expression' if a macro of name 'string' is not defined. If
1042the macro is define expand to `%{nil}`.
1043
1044`%(expression)`;;
1045Expand the macro to the result of running the 'expression' in a host shell. It
1046is assumed this is a Unix type shell. For example `%(whoami)` will return your
1047user name and `%(date)` will return the current date string.
1048
1049%prep
1050^^^^^
1051
1052The +%prep+ macro starts a block that continues until the next block macro. The
1053_prep_ or preparation block defines the setup of the package's source and is a
1054mix of RTEMS Source Builder macros and shell scripting. The sequence is
1055typically +%setup+ macros for source, +%patch+ macros to patch the source
1056mixed with some shell commands to correct any source issues.  A +%prep+ section
1057starts with a +%setup+ command. This creates the package source top level
1058directory then is followed by the first source file.
1059
1060-------------------------------------------------------------
1061                     <1>       <2>
1062%setup -q -c -T -n %{name}-%{version}
1063%setup -q -T -D -n %{name}-%{version} -a0
1064-------------------------------------------------------------
1065
1066<1> The package's name.
1067<2> The version of the package.
1068
1069The source for a package is declared with the +%source*+ macro where +*+ is
1070a number. For example +%source0+ is the source 0 tar file and is defined with
1071something similar to this:
1072
1073-------------------------------------------------------------
1074Source0: http://ftp.gnu.org/gnu/gdb/gdb-%{gdb_version}.tar.bz2
1075-------------------------------------------------------------
1076
1077This URL is the primary location of the GNU GCC source code and the RTEMS
1078Source Builder can download the file from this location and by inspecting the
1079file extension use +bzip2+ decompression. When the +%prep+ section is processed
1080a check of the local +source+ directory is made to see if the file has already
1081been downloaded. If not found in the source cache directory the package is
1082downloaded from the URL. You can append other base URLs via the command line
1083option +--url+. This option accepts a comma delimited list of sites to try.
1084
1085You can combine the source macro with conditional logic to implement a default
1086that can be over-riden in the top level files. This lets you reuse a generic
1087build script with different sources. This happens if you have a private source
1088package with local modifications. The following example is taken from the
1089+gcc-4.8-1.cfg+ build script.
1090
1091-------------------------------------------------------------
1092%ifn %{defined Source0}
1093 Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
1094%endif
1095-------------------------------------------------------------
1096
1097You could optionally have a few source files that make up the package. For
1098example GNU's GCC was a few tar files for a while and it is now a single tar
1099file. Support for multiple source files can be conditionally implemented with
1100the following scripting:
1101
1102-------------------------------------------------------------
1103%{?source1:%setup -q -T -D -n %{name}-%{version} -a1}
1104-------------------------------------------------------------
1105
1106The +source1+ macro value is checked and if present the command string after
1107the +:+ is executed. It is common to see a number of these present allowing top
1108level configuration files including a base configuration the ability to define
1109a number of source packages.
1110
1111-------------------------------------------------------------
1112%{?source1:%setup -q -T -D -n %{name}-%{version} -a1}
1113%{?source2:%setup -q -T -D -n %{name}-%{version} -a2}
1114%{?source3:%setup -q -T -D -n %{name}-%{version} -a3}
1115-------------------------------------------------------------
1116
1117Patching also occurs during the preparation stage. Patches are handled in a
1118similar way to the source packages. Most patches are based around the top of
1119the source tree. This is an example of the patch scripting for the GCC 4.8
1120series of compilers:
1121
1122-------------------------------------------------------------
1123cd gcc-%{gcc_version} <1>
1124%{?patch0:%patch0 -p1} <2>
1125%{?patch1:%patch1 -p1}
1126%{?patch2:%patch2 -p1}
1127%{?patch3:%patch3 -p1}
1128%{?patch4:%patch4 -p1}
1129%{?patch5:%patch5 -p1}
1130%{?patch6:%patch6 -p1}
1131%{?patch7:%patch7 -p1}
1132%{?patch8:%patch8 -p1}
1133%{?patch9:%patch9 -p1}
1134cd .. <3>
1135-------------------------------------------------------------
1136
1137<1> Change from the top of the source tree into the package being patched's top
1138    directory.
1139<2> The conditional macro expansion checks if +%patch0+ is defined and if
1140    defined issues the +%patch0" macro giving +-p1+ to the patch command.
1141<3> Return back to the top of the source tree.
1142
1143%build
1144^^^^^^
1145
1146The +%build+ macro starts a block that continues until the next block
1147macro. The build block is a series of shell commands that execute to build the
1148package. It assumes all source code has been unpacked, patch and adjusted so
1149the build will succeed.
1150
1151The following is an example take from the GutHub STLink project:
1152
1153NOTE: STLink is a JTAG debugging device for the ST ARM family of processors.
1154
1155-------------------------------------------------------------
1156%build
1157  export PATH="%{_bindir}:${PATH}" <1>
1158
1159  cd texane-stlink-%{stlink_version} <2>
1160
1161  ./autogen.sh <3>
1162
1163%if "%{_build}" != "%{_host}"
1164  CFLAGS_FOR_BUILD="-g -O2 -Wall" \ <4>
1165%endif
1166  CPPFLAGS="-I $SB_TMPPREFIX/include/libusb-1.0" \ <5>
1167  CFLAGS="$SB_OPT_FLAGS" \
1168  LDFLAGS="-L $SB_TMPPREFIX/lib" \
1169  ./configure \ <6>
1170    --build=%{_build} --host=%{_host} \
1171    --verbose \
1172    --prefix=%{_prefix} --bindir=%{_bindir} \
1173    --exec-prefix=%{_exec_prefix} \
1174    --includedir=%{_includedir} --libdir=%{_libdir} \
1175    --mandir=%{_mandir} --infodir=%{_infodir}
1176
1177  %{__make} %{?_smp_mflags} all <7>
1178
1179  cd ..
1180-------------------------------------------------------------
1181
1182<1> Setup the PATH environment variable. This is not always needed.
1183<2> This package builds in the source tree so enter it.
1184<3> The package is actually checked directly out from the github project and so
1185    it needs its autoconf and automake files generated.
1186<4> Flags for a cross-compiled build.
1187<5> Various settings passed to configure to customise the build. In this
1188    example an include path is being set to the install point of _libusb_. This
1189    package requires _libusb_ is built before it.
1190<6> The +configure+ command. The RTEMS Source Builder provides all the needed
1191    paths as macro variables. You just need to provide them to +configure+.
1192<7> Running make. Do not use +make+ directly, use the RTEMS Source Builder's
1193    defined value. This value is specific to the host. A large number of
1194    packages need GNU make and on BSD systems this is +gmake+. You can
1195    optionally add the SMP flags if the packages build system can handle
1196    parallel building with multiple jobs. The +_smp_mflags+ value is
1197    automatically setup for SMP hosts to match the number of cores the host has.
1198
1199%install
1200^^^^^^^^
1201
1202The +%install+ macro starts a block that continues until the next block
1203macro. The install block is a series of shell commands that execute to install
1204the package. You can assume the package has build correctly when this block
1205starts executing.
1206
1207Never install the package to the actual _prefix_ the package was built
1208with. Always install to the RTEMS Source Builder's temporary path defined in
1209the macro variable +\__tmpdir+. The RTEMS Source Builder sets up a shell
1210environment variable called +SB_BUILD_ROOT+ as the standard install point. Most
1211packages support adding +DESTDIR=+ to the _make install_ command.
1212
1213Looking at the same example as in <<_build, %build>>:
1214
1215-------------------------------------------------------------
1216%install
1217  export PATH="%{_bindir}:${PATH}" <1>
1218  rm -rf $SB_BUILD_ROOT <2>
1219
1220  cd texane-stlink-%{stlink_version} <3>
1221  %{__make} DESTDIR=$SB_BUILD_ROOT install <4>
1222
1223  cd ..
1224-------------------------------------------------------------
1225
1226<1> Setup the PATH environment variable. This is not always needed.
1227<2> Clean any installed files. This make sure the install is just what
1228    the package installs and not any left over files from a broken build or
1229    install.
1230<3> Enter the build directory. In this example it just happens to be the source
1231    directory.
1232<4> Run +make install+ to install the package overriding the +DESTDIR+ make
1233    variable.
1234
1235%clean
1236^^^^^^
1237
1238The +%clean+ macro starts a block that continues until the next block
1239macro. The clean block is a series of shell commands that execute to clean up
1240after a package has been built and install. This macro is currenly not been
1241used because the RTEMS Source Builder automatically cleans up.
1242
1243%include
1244^^^^^^^^
1245
1246The +%include+ macro inline includes the specific file. The +\__confdir+
1247path is searched. Any relative path component of the include file is appended
1248to each part of the +\__configdir+. Adding an extension is optional as files
1249with +.bset+ and +.cfg+ are automatically searched for.
1250
1251Inline including means the file is processed as part of the configuration at
1252the point it is included. Parsing continues from the next line in the
1253configuration file that contains the +%include+ macro.
1254
1255Including files allow a kind of configuration file reuse. The outer
1256configuration files provide specific information such as package version
1257numbers and patches and then include a generic configuration script which
1258builds the package.
1259
1260-------------------------------------------------------------
1261%include %{_configdir}/gcc-4.7-1.cfg
1262-------------------------------------------------------------
1263
1264%name
1265^^^^^
1266
1267The name of the package being built. The name typically contains the components
1268of the package and their version number plus a revision number. For the GCC
1269with Newlib configuration the name is typically:
1270
1271-------------------------------------------------------------
1272Name: %{_target}-gcc-%{gcc_version}-newlib-%{newlib_version}-%{release}
1273-------------------------------------------------------------
1274
1275%summary
1276^^^^^^^^
1277
1278The +%summary+ is a brief description of the package. It is useful when
1279reporting. This information is not capture in the package anywhere. For the GCC
1280with Newlib configuration the summary is typically:
1281
1282-------------------------------------------------------------
1283Summary: GCC v%{gcc_version} and Newlib v%{newlib_version} for target %{_target} on host %{_host}
1284-------------------------------------------------------------
1285
1286%release
1287^^^^^^^^
1288
1289The +%release+ is packaging number that allows revisions of a package to happen
1290where none package versions change. This value typically increases when the
1291configuration building the package changes.
1292
1293-------------------------------------------------------------
1294%define release 1
1295-------------------------------------------------------------
1296
1297%version
1298^^^^^^^^
1299
1300The +%version% macro sets the version the package. If the package is a single
1301component it tracks that component's version number. For example in the
1302_libusb_ configuration the +%version+ is the same as +%libusb_version+, however
1303in a GCC with Newlib configuration there is no single version number. In this
1304case the GCC version is used.
1305
1306-------------------------------------------------------------
1307Version: %{gcc_version}
1308-------------------------------------------------------------
1309
1310%buildarch
1311^^^^^^^^^^
1312
1313The +%buildarch+ macro is set to the architecture the package contains. This is
1314currently not used in the RTEMS Source Builder and may go away. This macro is
1315more important in a real packaging system where the package could end up on the
1316wrong architecture.
1317
1318%setup
1319^^^^^^
1320
1321The +%setup+ macro sets up the source code tree and is used in the +%prep+
1322section of the script. The options are:
1323
1324[horizontal]
1325*Switch*:: *Description*
1326+-n+:: The -n option is used to set the name of the software's build
1327directory. This is necessary only when the source archive unpacks into a
1328directory named other than +<name>-<version>+.
1329+-c+:: The -c option is used to direct %setup to create the top-level build
1330directory before unpacking the sources.
1331+-D+:: The -D option is used to direct %setup to not delete the build directory
1332prior to unpacking the sources. This option is used when more than one source
1333archive is to be unpacked into the build directory, normally with the +-b+ or
1334+-a+ options.
1335+-T+:: The -T option is used to direct %setup to not perform the default
1336unpacking of the source archive specified by the first Source: macro. It is used
1337with the +-a+ or +-b+ options.
1338+-b <n>+:: The -b option is used to direct %setup to unpack the source archive
1339specified on the nth Source: macro line before changing directory into the build
1340directory.
1341+-a <n>+:: The -a option is used to direct %setup to unpack the source archive
1342specified on the nth Source: macro line after changing directory into the build
1343directory.
1344
1345%source
1346^^^^^^^
1347
1348The +%source+ macro is numbered and defines a source tar file used in the
1349package. The +%setup+ macro references the packages defined here. A macro is
1350defined as:
1351
1352-------------------------------------------------------------
1353Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
1354-------------------------------------------------------------
1355
1356The setup script is:
1357
1358-------------------------------------------------------------
1359%setup -q -T -D -n %{name}-%{version} -a0
1360-------------------------------------------------------------
1361
1362The +-a0+ means use +%source0+.
1363
1364%patch
1365^^^^^^
1366
1367The +%patch+ macro is numbered and can define a patch and in the +%prep+
1368section applies the patch. To define a patch append a +:+ followed by the patch
1369filename:
1370
1371-------------------------------------------------------------
1372Patch0: gcc-4.7.2-rtems4.11-20121026.diff
1373-------------------------------------------------------------
1374
1375The +__patchdir+ path is search.
1376
1377Placing +%patch+ in the +%prep+ section will apply it with any trailing options
1378passed to the +patch+ command. This allows the +-p+ option to be passed to
1379strip any leading path components from the patch contents.
1380
1381-------------------------------------------------------------
1382%patch0 -p1
1383-------------------------------------------------------------
1384
1385You will typically see the patch conditionally used as:
1386
1387-------------------------------------------------------------
1388%{patch0:%patch0 -p1}
1389-------------------------------------------------------------
1390
1391In this case the patch will only be applied if it is defined.
1392
1393%echo
1394^^^^^
1395
1396The +%echo+ macro outputs the following string to stdout. This can also be used
1397as `%{echo: message}`.
1398
1399%warning
1400^^^^^^^^
1401
1402The +%warning+ macro outputs the following string as a warning. This can also
1403be used as `%{warning: message}`.
1404
1405%error
1406^^^^^^
1407
1408The +%error+ macro outputs the follow string as an error and exits the RTEMS
1409Source Builder. This can also be used as `%{error: message}`.
1410
1411%select
1412^^^^^^^
1413
1414The +%select+ macro selects the map specified. If there is no map no error or
1415warning is generated. Macro maps provide a simple way for a user to override
1416the settings is a configuration file without having to edit it. The changes are
1417recorded in the build report so can be traced.
1418
1419Configuration use different maps so macro overrides can target a specific
1420package.
1421
1422The default map is `global'.
1423
1424-------------------------------------------------------------
1425%select gcc-4.8-snapshot <1>
1426%define one_plus_one 2 <2>
1427-------------------------------------------------------------
1428
1429<1> The map switches to `gcc-4.8-snapshot`. Any overrides in this map will be
1430    used.
1431<2> Defining macros only updates the `global` map and not the selected map.
1432
1433%define
1434^^^^^^^
1435
1436The +%define+ macro defines a new macro or updates an existing one. If no value
1437is given it is assumed to be 1.
1438
1439-------------------------------------------------------------
1440%define foo bar
1441%define one_plus_one 2
1442%define one <1>
1443-------------------------------------------------------------
1444
1445<1> The macro _one_ is set to 1.
1446
1447%undefine
1448^^^^^^^^^
1449
1450The +%undefine+ macro removes a macro if it exists. Any further references to
1451it will result in an undefine macro error.
1452
1453%if
1454^^^
1455
1456The +%if+ macro starts a conditional logic block that can optionally have a
1457_else_ section. A test follows this macro and can have the following operators:
1458
1459[horizontal]
1460*Operator*:: *Description*
1461+%{}+:: Check the macro is set or _true_, ie non-zero.
1462+
1463-------------------------------------------------------------
1464%if ${foo}
1465 %warning The test passes, must not be empty or is non-zero
1466%else
1467 %error The test fails, must be empty or zero
1468%endif
1469-------------------------------------------------------------
1470+!+:: The _not_ operator inverts the test of the macro.
1471+
1472-------------------------------------------------------------
1473%if ! ${foo}
1474 %warning The test passes, must be empty or zero
1475%else
1476 %error The test fails, must not be empty or is non-zero
1477%endif
1478-------------------------------------------------------------
1479+==+:: The left hand size must equal the right hand side. For example:
1480+
1481-------------------------------------------------------------
1482%define one 1
1483%if ${one} == 1
1484 %warning The test passes
1485%else
1486 %error The test fails
1487%endif
1488-------------------------------------------------------------
1489+
1490You can also check to see if a macro is empty:
1491+
1492-------------------------------------------------------------
1493%if ${nothing} == %{nil}
1494 %warning The test passes
1495%else
1496 %error The test fails
1497%endif
1498-------------------------------------------------------------
1499+!=+:: The left hand size does not equal the right hand side. For example:
1500+
1501-------------------------------------------------------------
1502%define one 1
1503%if ${one} != 2
1504 %warning The test passes
1505%else
1506 %error The test fails
1507%endif
1508-------------------------------------------------------------
1509+
1510You can also check to see if something is set:
1511+
1512-------------------------------------------------------------
1513%if ${something} != %{nil}
1514 %warning The test passes
1515%else
1516 %error The test fails
1517%endif
1518-------------------------------------------------------------
1519+>+:: The left hand side is numerically greater than the right hand side.
1520+>=+:: The left hand side is numerically greater than or equal to the right
1521hand side.
1522+<+:: The left hand side is numerically less than the right hand side.
1523+\<=+:: The left hand side is numerically less than or equal to the right hand
1524side.
1525
1526%ifn
1527^^^^
1528
1529The +%ifn+ macro inverts the normal +%if+ logic. It avoids needing to provide
1530empty _if_ blocks followed by _else_ blocks. It is useful when checking if a
1531macro is defined:
1532
1533-------------------------------------------------------------
1534%ifn %{defined foo}
1535 %define foo bar
1536%endif
1537-------------------------------------------------------------
1538
1539%ifarch
1540^^^^^^^
1541
1542The +%ifarch+ is a short cut for "+%if %{\_arch} == i386+". Currently not used.
1543
1544%ifnarch
1545^^^^^^^^
1546
1547The +%ifnarch+ is a short cut for "+%if %{\_arch} != i386+". Currently not
1548used.
1549
1550%ifos
1551^^^^^
1552
1553The +%ifos+ is a short cut for "+%if %{\_os} != mingw32+". It allows
1554conditional support for various operating system differences when building
1555packages.
1556
1557%else
1558^^^^^
1559
1560The +%else+ macro starts the conditional _else_ block.
1561
1562%endfi
1563^^^^^^
1564
1565The +%endif+ macro ends a conditional logic block.
1566
1567%bconf_with
1568^^^^^^^^^^^
1569
1570The +%bconf_with+ macro provides a way to test if the user has passed a
1571specific option on the command line with the +--with-<label>+ option. This
1572option is only available with the +sb-builder+ command.
1573
1574%bconf_without
1575^^^^^^^^^^^^^^
1576
1577The +%bconf_without+ macro provides a way to test if the user has passed a
1578specific option on the command line with the +--without-<label>+ option. This
1579option is only available with the +sb-builder+ command.
1580
1581
1582Project Sets
1583------------
1584
1585The RTEMS Source Builder supports project configurations. Project
1586configurations can be public or private and can be contained in the RTEMS
1587Source Builder project if suitable, other projects they use the RTEM Source
1588Builder or privately on your local file system.
1589
1590The configuration file loader searches the macro +_configdir+ and by default
1591this is set to +%{\_topdir}/config:%{\_sbdir}/config+ where +_topdir+ is the
1592your current working direct, in other words the directory you invoke the RTEMS
1593Source Builder command in, and +_sbdir+ is the directory where the RTEMS Source
1594Builder command resides. Therefore the +config+ directory under each of these
1595is searched so all you need to do is create a +config+ in your project and add
1596your configuration files. They do not need to be under the RTEMS Source Builder
1597source tree. Public projects are included in the main RTEMS Source Builder such
1598as RTEMS.
1599
1600You can also add your own +patches+ directory next to your +config+ directory
1601as the +%patch+ command searches the +_patchdir+ macro variable and it is
1602by default set to +%{\_topdir}/patches:%{\_sbdir}/patches+.
1603
1604The +source-builder/config+ directory provides generic scripts for building
1605various tools. You can specialise these in your private configurations to make
1606use of them. If you add new generic configurations please contribute them back
1607to the project
1608
1609RTEMS Configurations
1610~~~~~~~~~~~~~~~~~~~~
1611
1612The RTEMS Configurations are grouped by RTEMS version. In RTEMS the tools are
1613specific to a specific version because of variations between Newlib and
1614RTEMS. Restructuring in RTEMS and Newlib sometimes moves _libc_ functionality
1615between these two parts and this makes existing tools incompatible with RTEMS.
1616
1617RTEMS allows architectures to have different tool versions and patches. The
1618large number of architectures RTEMS supports can make it difficult to get a
1619common stable version of all the packages. An architecture may require a recent
1620GCC because an existing bug has been fixed, however the more recent version may
1621have a bug in other architecture. Architecture specific patches should be
1622limited to the architecture it relates to. The patch may fix a problem on the
1623effect architecture however it could introduce a problem in another
1624architecture. Limit exposure limits any possible crosstalk between
1625architectures.
1626
1627RTEMS supports _stable_ and _unstable_. Stable configurations are fixed while
1628unstable configurations are supporting using snapshots of user macros and
1629reference release candidates or source extracted directly from version
1630control. The stable build sets are referenced as +<version>/rtems-<arch>+ and
1631include `autoconf` and `automake`.
1632
1633If you are building a released version of RTEMS the release RTEMS tar file will
1634be downloaded and built as part of the build process. If you are building a
1635tool set for use with the development branch of RTEMS, the development branch
1636will be cloned directly from the RTEMS GIT repository and built.
1637
1638When building RTEMS within the RTEMS Source Builder it needs a suitable working
1639`autoconf` and `automake`. These packages need to built and installed in their
1640prefix in order for them to work. The RTEMS Source Builder installs all
1641packages only after they have been built so if you host does not have a
1642recent enough version of `autoconf` and `automake` you first need to build them
1643and install them then build your tool set. The commands are:
1644
1645-------------------------------------------------------------
1646$ ../source-builder/sb-set-builder --log=l-4.11-at.txt \
1647   --prefix=$HOME/development/rtems/4.11 4.11/rtems-autotools
1648$ export PATH=~/development/rtems/4.11/bin:$PATH <1>
1649$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
1650   --prefix=$HOME/development/rtems/4.11 4.11/rtems-sparc
1651-------------------------------------------------------------
1652<1> Setting the path.
1653
1654TIP: If this is your first time building the tools and RTEMS it pays to add the
1655`--dry-run` option. This will run through all the configuration files and if
1656any checks fail you will see this quickly rather than waiting for until the
1657build fails a check.
1658
1659To build snapshots for testing purposes you use the available macro maps
1660passing them on the command line using the `--macros` option. For RTEMS these
1661are held in `config/snapshots` directory. The following builds _newlib_ from
1662CVS:
1663
1664-------------------------------------------------------------
1665$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
1666   --prefix=$HOME/development/rtems/4.11 \
1667   --macros=snapshots/newlib-head.mc \
1668   4.11/rtems-sparc
1669-------------------------------------------------------------
1670
1671and the following uses the version control heads for _binutils_, _gcc_,
1672_newlib_, _gdb_ and _RTEMS_:
1673
1674-------------------------------------------------------------
1675$ ../source-builder/sb-set-builder --log=l-heads-sparc.txt \
1676   --prefix=$HOME/development/rtems/4.11-head \
1677   --macros=snapshots/binutils-gcc-newlib-gdb-head.mc \
1678   4.11/rtems-sparc
1679-------------------------------------------------------------
1680
1681Commands
1682--------
1683
1684Checker (sb-check)
1685~~~~~~~~~~~~~~~~~~
1686
1687This commands checks your system is set up correctly. Most options are ignored.
1688
1689-------------------------------------------------------------
1690$ ../source-builder/sb-check --help
1691sb-check: [options] [args]
1692RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
1693Options and arguments:
1694--force                : Force the build to proceed
1695--quiet                : Quiet output (not used)
1696--trace                : Trace the execution
1697--dry-run              : Do everything but actually run the build
1698--warn-all             : Generate warnings
1699--no-clean             : Do not clean up the build tree
1700--always-clean         : Always clean the build tree, even with an error
1701--jobs                 : Run with specified number of jobs, default: num CPUs.
1702--host                 : Set the host triplet
1703--build                : Set the build triplet
1704--target               : Set the target triplet
1705--prefix path          : Tools build prefix, ie where they are installed
1706--topdir path          : Top of the build tree, default is $PWD
1707--configdir path       : Path to the configuration directory, default: ./config
1708--builddir path        : Path to the build directory, default: ./build
1709--sourcedir path       : Path to the source directory, default: ./source
1710--tmppath path         : Path to the temp directory, default: ./tmp
1711--macros file[,[file]  : Macro format files to load after the defaults
1712--log file             : Log file where all build out is written too
1713--url url[,url]        : URL to look for source
1714--no-download          : Disable the source downloader
1715--targetcflags flags   : List of C flags for the target code
1716--targetcxxflags flags : List of C++ flags for the target code
1717--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
1718--with-<label>         : Add the --with-<label> to the build
1719--without-<label>      : Add the --without-<label> to the build
1720--regression           : Set --no-install, --keep-going and --always-clean
1721$ ../source-builder/sb-check
1722RTEMS Source Builder - Check, v0.2.0
1723Environment is ok
1724-------------------------------------------------------------
1725
1726Defaults (sb-defaults)
1727~~~~~~~~~~~~~~~~~~~~~~
1728
1729This commands outputs and the default macros for your when given no
1730arguments. Most options are ignored.
1731
1732-------------------------------------------------------------
1733$ ../source-builder/sb-defaults --help
1734sb-defaults: [options] [args]
1735RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
1736Options and arguments:
1737--force                : Force the build to proceed
1738--quiet                : Quiet output (not used)
1739--trace                : Trace the execution
1740--dry-run              : Do everything but actually run the build
1741--warn-all             : Generate warnings
1742--no-clean             : Do not clean up the build tree
1743--always-clean         : Always clean the build tree, even with an error
1744--jobs                 : Run with specified number of jobs, default: num CPUs.
1745--host                 : Set the host triplet
1746--build                : Set the build triplet
1747--target               : Set the target triplet
1748--prefix path          : Tools build prefix, ie where they are installed
1749--topdir path          : Top of the build tree, default is $PWD
1750--configdir path       : Path to the configuration directory, default: ./config
1751--builddir path        : Path to the build directory, default: ./build
1752--sourcedir path       : Path to the source directory, default: ./source
1753--tmppath path         : Path to the temp directory, default: ./tmp
1754--macros file[,[file]  : Macro format files to load after the defaults
1755--log file             : Log file where all build out is written too
1756--url url[,url]        : URL to look for source
1757--no-download          : Disable the source downloader
1758--targetcflags flags   : List of C flags for the target code
1759--targetcxxflags flags : List of C++ flags for the target code
1760--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
1761--with-<label>         : Add the --with-<label> to the build
1762--without-<label>      : Add the --without-<label> to the build
1763--regression           : Set --no-install, --keep-going and --always-clean
1764-------------------------------------------------------------
1765
1766Set Builder (sb-set-builder)
1767~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1768
1769This command builds a set.
1770
1771-------------------------------------------------------------
1772$ ../source-builder/sb-set-builder --help
1773RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
1774Options and arguments:
1775--force                : Force the build to proceed
1776--quiet                : Quiet output (not used)
1777--trace                : Trace the execution
1778--dry-run              : Do everything but actually run the build
1779--warn-all             : Generate warnings
1780--no-clean             : Do not clean up the build tree
1781--always-clean         : Always clean the build tree, even with an error
1782--jobs                 : Run with specified number of jobs, default: num CPUs.
1783--host                 : Set the host triplet
1784--build                : Set the build triplet
1785--target               : Set the target triplet
1786--prefix path          : Tools build prefix, ie where they are installed
1787--topdir path          : Top of the build tree, default is $PWD
1788--configdir path       : Path to the configuration directory, default: ./config
1789--builddir path        : Path to the build directory, default: ./build
1790--sourcedir path       : Path to the source directory, default: ./source
1791--tmppath path         : Path to the temp directory, default: ./tmp
1792--macros file[,[file]  : Macro format files to load after the defaults
1793--log file             : Log file where all build out is written too
1794--url url[,url]        : URL to look for source
1795--no-download          : Disable the source downloader
1796--targetcflags flags   : List of C flags for the target code
1797--targetcxxflags flags : List of C++ flags for the target code
1798--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
1799--with-<label>         : Add the --with-<label> to the build
1800--without-<label>      : Add the --without-<label> to the build
1801--bset-tar-file        : Create a build set tar file
1802--no-report            : Do not create a package report.
1803--list-configs         : List available configurations
1804--report-format        : The report format (text, html, asciidoc).
1805--list-bsets           : List available build sets
1806--no-install           : Do not install the packages to the prefix.
1807--list-deps            : List the dependent files.
1808--pkg-tar-files        : Create package tar files
1809--regression           : Set --no-install, --keep-going and --always-clean
1810-------------------------------------------------------------
1811
1812.Arguments
1813The +[args]+ are a list build sets to build.
1814
1815.Options
1816+--force+;;
1817Force the build to proceed even if the host check fails. Typically this happens
1818if executable files are found in the path at a different location to the host
1819defaults.
1820+--trace+;;
1821Trace enable printing of debug information to stdout. It is really only of use
1822to RTEMS Source Builder's developers.
1823+--dry-run+;;
1824Do everything but actually run the build commands. This is useful when checking
1825a new configuration parses cleanly.
1826+--warn-all+;;
1827Generate warnings.
1828+--no-clean+;;
1829Do not clean up the build tree during the cleaning phase of the build. This
1830leaves the source and the build output on disk so you can make changes, or
1831amend or generate new patches. It also allows you to review configure type
1832output such as +config.log+.
1833+--always-clean+;;
1834Clean away the results of a build even if the build fails. This is normally
1835used with `--keep-going` when regression testing to see which build sets
1836fail to build. It keeps the disk usage down.
1837+--jobs+;;
1838Control the number of jobs make is given. The jobs can be 'none' for only 1
1839job, 'half' so the number of jobs is half the number of detected cores, a
1840fraction such as '0.25' so the number of jobs is a quarter of the number of
1841detected cores and a number such as '25' which forces the number of jobs to
1842that number.
1843+--host+;;
1844Set the host triplet value. Be careful with this option.
1845+--build+;;
1846Set the build triplet. Be careful with this option.
1847+--target+;;
1848Set the target triplet. Be careful with this option. This is useful if you have
1849a generic configuration script that can work for a range of architectures.
1850+--prefix path+;;
1851Tools build prefix, ie where they are installed.
1852+--topdir path+;;
1853Top of the build tree, that is the current directory you are in.
1854+--configdir path+;;
1855Path to the configuration directory. This overrides the built in defaults.
1856+--builddir path+;;
1857Path to the build directory. This overrides the default of +build+.
1858+--sourcedir path+;;
1859Path to the source directory. This overrides the default of +source+.
1860+--tmppath path+;;
1861Path to the temporary directory. This overrides the default of +tmp+.
1862+--macros files+;;
1863Macro files to load. The configuration directory path is searched.
1864+--log file+;;
1865Log all the output from the build process. The output is directed to +stdout+
1866if no log file is provided.
1867+--url url+;;
1868URL to look for source when downloading. This is can be comma separate list.
1869+--no-download+;;
1870Disable downloading of source and patches. If the source is not found an error
1871is raised.
1872+--targetcflags flags+;;
1873List of C flags for the target code. This allows for specific local
1874customisation when testing new variations.
1875+--targetcxxflags flags+;;
1876List of C++ flags for the target code. This allows for specific local
1877customisation when testing new variations.
1878+--libstdcxxflags flags+;;
1879List of C\++ flags to build the target libstdc++ code. This allows for specific
1880local customisation when testing new variations.
1881+--with-<label>+;;
1882Add the --with-<label> to the build. This can be tested for in a script with
1883the +%bconf_with+ macro.
1884+--without-<label>+;;
1885Add the --without-<label> to the build. This can be tested for in a script with
1886the +%bconf_without+ macro.
1887+--list-bsets+;;
1888List available build sets.
1889+--list-configs+;;
1890List available configurations.
1891+--keep-going+;;
1892Do not stop on error. This is useful if your build sets performs a large number
1893of testing related builds and there are errors.
1894+--no-install+;;
1895Do not install the packages to the prefix. Use this if you are only after the
1896tar files.
1897+--bset-tar-file+;;
1898Create a build set tar file. This is a single tar file of all the packages in
1899the build set.
1900+--pkg-tar-files+;;
1901Create package tar files. A tar file will be created for each package built in
1902a build set.
1903+--list-deps+;;
1904Print a list of dependent files used by a build set. Dependent files have a
1905'dep[?]' prefix where '?' is a number. The files are listed alphabetically.
1906+--report-format format+;;
1907The report format can be 'text' or 'html'. The default is 'html'.
1908+--regression+;;
1909A convenience option which is the same as +--no-install+, +--keep-going+ and
1910+--always-clean+.
1911
1912Set Builder (sb-builder)
1913~~~~~~~~~~~~~~~~~~~~~~~~
1914
1915This command builds a configuration as described in a configuration
1916file. Configuration files have the extension of +.cfg+.
1917
1918-------------------------------------------------------------
1919$ ./source-builder/sb-builder --help
1920sb-builder: [options] [args]
1921RTEMS Source Builder, an RTEMS Tools Project (c) 2012 Chris Johns
1922Options and arguments:
1923--force                : Force the build to proceed
1924--quiet                : Quiet output (not used)
1925--trace                : Trace the execution
1926--dry-run              : Do everything but actually run the build
1927--warn-all             : Generate warnings
1928--no-clean             : Do not clean up the build tree
1929--always-clean         : Always clean the build tree, even with an error
1930--jobs                 : Run with specified number of jobs, default: num CPUs.
1931--host                 : Set the host triplet
1932--build                : Set the build triplet
1933--target               : Set the target triplet
1934--prefix path          : Tools build prefix, ie where they are installed
1935--topdir path          : Top of the build tree, default is $PWD
1936--configdir path       : Path to the configuration directory, default: ./config
1937--builddir path        : Path to the build directory, default: ./build
1938--sourcedir path       : Path to the source directory, default: ./source
1939--tmppath path         : Path to the temp directory, default: ./tmp
1940--macros file[,[file]  : Macro format files to load after the defaults
1941--log file             : Log file where all build out is written too
1942--url url[,url]        : URL to look for source
1943--targetcflags flags   : List of C flags for the target code
1944--targetcxxflags flags : List of C++ flags for the target code
1945--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
1946--with-<label>         : Add the --with-<label> to the build
1947--without-<label>      : Add the --without-<label> to the build
1948--list-configs         : List available configurations
1949-------------------------------------------------------------
1950
1951Host Setups
1952-----------
1953
1954The host versions are listed. If a later version of the host operating system
1955exists it should unless listed.
1956
1957Linux
1958~~~~~
1959
1960A number of different Linux distrubutions are known to work. The following have
1961been tested and report as working.
1962
1963Archlinux
1964^^^^^^^^^
1965
1966The following packages are required on a fresh Archlinux 64bit installation:
1967
1968--------------------------------------------------------------
1969# pacman -S base-devel gdb xz unzip ncurses git zlib
1970--------------------------------------------------------------
1971
1972Archlinux, by default installs `texinfo-5` which is incompatible for building
1973GCC 4.7 tree. You will have to obtain `texinfo-legacy` from `AUR` and provide
1974a manual override.
1975
1976--------------------------------------------------------------
1977# pacman -R texinfo
1978$ yaourt -S texinfo-legacy
1979# ln -s /usr/bin/makeinfo-4.13a /usr/bin/makeinfo
1980--------------------------------------------------------------
1981
1982CentOS
1983^^^^^^
1984
1985The following packages are required on a minimal CentOS 6.3 64bit installation:
1986
1987-------------------------------------------------------------
1988# yum install autoconf automake binutils gcc gcc-c++ gdb make patch \
1989bison flex xz unzip ncurses-devel texinfo zlib-devel python-devel git
1990-------------------------------------------------------------
1991
1992The minimal CentOS distribution is a specific DVD that installs a minimal
1993system. If you use a full system some of these packages may have been
1994installed.
1995
1996Fedora
1997^^^^^^
1998
1999The RTEMS Source Builder has been tested on Fedora 18 64bit.
2000
2001Raspbian
2002^^^^^^^^
2003
2004The is the Debian distribution for the Raspberry Pi. The following packages are
2005required.
2006
2007-------------------------------------------------------------
2008$ sudo apt-get install autoconf automake bison flex binutils gcc g++ gdb \
2009texinfo unzip ncurses-dev python-dev git
2010-------------------------------------------------------------
2011
2012It is recommended you get Model B of the Pi with 512M of memory and to mount a
2013remote disk over the network. The tools can be build with a prefix under your
2014home directory as recommended and end up on the SD card.
2015
2016Ubuntu
2017^^^^^^
2018
2019The latest testing was with Ubuntu 12.10 64bit. A minimal installation was used
2020and the following packages installed.
2021
2022-------------------------------------------------------------
2023$ sudo apt-get build-dep binutils gcc g++ gdb unzip git
2024-------------------------------------------------------------
2025
2026FreeBSD
2027~~~~~~~
2028
2029The RTEMS Source Builder has been tested on FreeBSD 9.1 64bit.
2030
2031MacOS X
2032~~~~~~~
2033
2034The RTEMS Source Builder has been tested on Mountain Lion. You will need to
2035install the Xcode app using the _App Store_ tool, run Xcode and install the
2036Developers Tools package within Xcode.
2037
2038Windows
2039~~~~~~~
2040
2041Windows tool sets are supported creating native Windows executable. Native
2042Windows tools are built using a MinGW compiler and do not need any extra
2043libraries or emulation layer to run once built. The tools understand and use
2044standard Windows paths and integrate easly into Windows IDE environments. A
2045shell maybe needed to build other parts of your system however if your
2046development tools are all native Windows tool you can easly integrate these
2047tool sets.
2048
2049Building on Windows is a little more complicated because the Cygwin shell is
2050used rather than the MinGW MSYS shell. The MSYS shell is simpler because the
2051detected host triple is MinGW so the build is standard cross-compiler build.
2052The age of the MSYS code base, its stability and ability to to complete a build
2053with limitations such as the length of file names support make using MSYS
2054difficult therefore the more complex path of a Canadian cross-build using
2055Cygwin is supported.
2056
2057Install a recent Cygwin version using the Cygwin setup tool. Select and install
2058the groups and packages listed:
2059
2060.Cygwin Packages
2061[options="header,compact",width="50%",cols="20%,80%"]
2062|================================
2063|Group   |Package
2064|Archive |bsdtar
2065|        |unzip
2066|        |xz
2067|Devel   |autoconf
2068|        |autoconf2.1
2069|        |autoconf2.5
2070|        |automake
2071|        |binutils
2072|        |bison
2073|        |flex
2074|        |gcc4-core
2075|        |gcc4-g++
2076|        |git
2077|        |make
2078|        |mingw64-x86_64-binutils
2079|        |mingw64-x86_64-gcc-core
2080|        |mingw64-x86_64-g++
2081|        |mingw64-x86_64-runtime
2082|        |mingw64-x86_64-zlib
2083|        |zlib-devel
2084|MinGW   |mingw-zlib-devel
2085|Python  |python
2086|================================
2087
2088The setup tool will add a number of dependent package and it is ok to accept
2089them.
2090
2091I have found turning off Windows Defender improves performance if you have
2092another up to date virus detection tool installed and enabled. I used the
2093excellent `Process Hacker 2` tool to monitor the performance and I found the
2094Windows Defender service contributed a high load. In my case I had a 3rd party
2095virus tool installed so the Windows Defender service was not needed.
2096
2097A Canadian cross-compile (Cxc) is required on Cygwin because the host is Cygwin
2098therefore a traditional cross-compile will result in Cygiwn binaries. With a
2099Canadian cross-compile a Cygwin cross-compiler is built as well as the MinGW
2100RTEMS cross-compiler. The Cygwin cross-compiler is required to build the C
2101runtime for the RTEMS target because we are building under Cygiwn. The build
2102output for an RTEMS 4.10 ARM tool set is:
2103
2104-------------------------------------------------------------
2105chris@cygthing ~/development/rtems/src/rtems-source-builder/rtems
2106$ ../source-builder/sb-set-builder --log=l-arm.txt --prefix=$HOME/development/rtems/4.10 4.10/rtems-arm
2107RTEMS Source Builder - Set Builder, v0.2
2108Build Set: 4.10/rtems-arm
2109config: expat-2.1.0-1.cfg
2110package: expat-2.1.0-x86_64-w64-mingw32-1
2111building: expat-2.1.0-x86_64-w64-mingw32-1
2112reporting: expat-2.1.0-1.cfg -> expat-2.1.0-x86_64-w64-mingw32-1.html
2113config: tools/rtems-binutils-2.20.1-1.cfg
2114package: arm-rtems4.10-binutils-2.20.1-1 <1>
2115building: arm-rtems4.10-binutils-2.20.1-1
2116package: (Cxc) arm-rtems4.10-binutils-2.20.1-1 <2>
2117building: (Cxc) arm-rtems4.10-binutils-2.20.1-1
2118reporting: tools/rtems-binutils-2.20.1-1.cfg ->
2119arm-rtems4.10-binutils-2.20.1-1.html
2120config: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg
2121package: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2122building: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2123package: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2124building: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2125reporting: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg ->
2126arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1.html
2127config: tools/rtems-gdb-7.3.1-1.cfg
2128package: arm-rtems4.10-gdb-7.3.1-1
2129building: arm-rtems4.10-gdb-7.3.1-1
2130reporting: tools/rtems-gdb-7.3.1-1.cfg -> arm-rtems4.10-gdb-7.3.1-1.html
2131config: tools/rtems-kernel-4.10.2.cfg
2132package: arm-rtems4.10-kernel-4.10.2-1
2133building: arm-rtems4.10-kernel-4.10.2-1
2134reporting: tools/rtems-kernel-4.10.2.cfg -> arm-rtems4.10-kernel-4.10.2-1.html
2135installing: expat-2.1.0-x86_64-w64-mingw32-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2136installing: arm-rtems4.10-binutils-2.20.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10 <3>
2137installing: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2138installing: arm-rtems4.10-gdb-7.3.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2139installing: arm-rtems4.10-kernel-4.10.2-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
2140cleaning: expat-2.1.0-x86_64-w64-mingw32-1
2141cleaning: arm-rtems4.10-binutils-2.20.1-1
2142cleaning: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
2143cleaning: arm-rtems4.10-gdb-7.3.1-1
2144cleaning: arm-rtems4.10-kernel-4.10.2-1
2145Build Set: Time 10:09:42.810547 <4>
2146-------------------------------------------------------------
2147
2148<1> The Cygwin version of the ARM cross-binutils.
2149<2> The +(Cxc)+ indicates this is the MinGW build of the package.
2150<3> Only the MinGW version is installed.
2151<4> Cygwin is slow so please be patient. This time was on an AMD Athlon 64bit
2152    Dual Core 6000+ running at 3GHz with 4G RAM running Windows 7 64bit.
2153
2154CAUTION: Cygwin documents the 'Big List Of Dodgy Apps' or 'BLODA'. The link is
2155http://cygwin.com/faq/faq.using.html#faq.using.bloda and it is worth a
2156look. You will see a large number of common pieces of software found on Windows
2157systems that can cause problems. My testing has been performed with NOD32
2158running and I have seen some failures. The list is for all of Cygwin so I am
2159not sure which of the listed programs effect the RTEMS Source Biulder. The
2160following FAQ item talks about +fork+ failures and presents some technical
2161reasons they cannot be avoided in all cases. Cygwin and it's fork MSYS are
2162fantastic pieces of software in a difficult environment. I have found building
2163a single tool tends to work, building all at once is harder.
2164
2165
2166
2167Build Status By Host
2168~~~~~~~~~~~~~~~~~~~~
2169
2170This table lists the current build and testing status for reported hosts:
2171
2172[grid="rows",format="csv"]
2173[options="header",cols="<,<,<,<,<,<"]
2174|===========================
2175OS,Uname,4.9,4.10,4.11,Comments
2176include::host-results.csv[]
2177|===========================
2178
2179[NOTE]
2180==================================================================
2181Report any unlisted hosts as a patch.
2182==================================================================
2183
2184History
2185-------
2186
2187The RTEMS Source Builder is a stand alone tool based on another tool called the
2188'SpecBuilder'. The SpecBuilder was written for the RTEMS project to give me a
2189way to build tools on hosts that did not support RPMs. At the time the RTEMS
2190tools maintainer only used spec files to create various packages. This meant I
2191had either spec files, RPM files or SRPM files. The RPM and SPRM files where
2192useless because you needed an 'rpm' type tool to extract and manage them. There
2193are versions of 'rpm' for a number of non-RPM hosts however these proved to be
2194in various broken states and randomly maintained. The solution I settled on was
2195to use spec files so I wrote a Python based tool that parsed the spec file
2196format and allowed me to create a shell script I could run to build the
2197package. This approach proved successful and I was able to track the RPM
2198version of the RTEMS tools on a non-RPM host over a number of years. however
2199the SpecBuilder tool did not help me build tools or other packages not related
2200to the RTEMS project where there was no spec file I could use so I needed
2201another tool. Rather than start again I decided to take the parsing code for
2202the spec file format and build a new tool called the RTEMS Source Builder.
Note: See TracBrowser for help on using the repository browser.