source: rtems-source-builder/doc/source-builder.txt @ b328e49

RTEMS Source Builder
====================
:doctype: book
:toc2:
:toclevels: 5
:icons:
:numbered:

image:images/rtemswhitebg.jpg["RTEMS",width="30%"]

Chris Johns <chrisj@rtems.org>
1.2, April 2013

RTEMS Tools From Source
-----------------------

The RTEMS Source Builder is a tool to aid building packages from source. It is
not a package manager. It helps consolidate the details you need to build a
package from source in a controlled and verifiable way. The tool is aimed at
developers of software who use tool sets for embedded type
development. Embedded development typically uses cross-compiling tool chains,
debuggers, and debugging aids. Together we call these a 'tool set'. The RTEMS
Source Builder is not limited to this role but designed to fit with-in this
specific niche. It can be used outside of the RTEMS project and we welcome this
happening in other open source or commercial projects.

The RTEMS Source Builder is typically used to build a set of tools or a 'build
set'. A 'build set' is a collection of packages and a package is a specific
tool, for example gcc or gdb. The RTEMS Source Builder attempts to support any
host environment that runs Python and you can build the package on. It is not
some sort of magic that can take any piece of source code and make it
build. Someone at some point in time has figured out how to build that package
from source and taught this tool. The RTEMS Source Builder has been tested on:

* <<_archlinux,Archlinux>>
* <<_centos,Centos>>
* <<_fedora,Fedora>>
* <<_freebsd,FreeBSD>>
* <<_macos,MacOS>>
* <<_opensuse,openSUSE>>
* <<_raspbian,Raspbian>>
* <<_ubuntu,Ubuntu>>
* <<_windows,Windows>>

The RTEMS Source Builder has two types configuration data. The first is the
'build set'. A _build set_ describes a collection of packages that define a set
of tools you would use when developing software for RTEMS. For example the
basic GNU tool set is binutils, gcc, and gdb and is the typical base suite of
tools you need for an embedded cross-development type project. The second type
of configuration data is the configuration files and they define how a package
is built. Configuration files are scripts loosely based on the RPM spec file
format and detail the steps needed to build a package. The steps are
'preparation', 'building', and 'installing'. Scripts support macros, shell
expansion, logic, includes plus many more features useful when build packages.

The RTEMS Source Builder does not interact with any host package management
systems. There is no automatic dependence checking between various packages you
build or packages and software your host system you may have installed. We
assume the build sets and configuration files you are using have been created
by developers who do. If you have a problem please ask on the RTEMS Users
mailing list.

Quick Start
-----------

The quick start will show you how to build a set of RTEMS tools for a supported
architecture.

There is no need to become root or the administrator and we recommend you avoid
doing this. You can build and install the tools anywhere on the host's file
system you, as a standard user, have read and write access too. I recommend you
use your home directory and work under the directory `~/development/rtems`. The
examples shown here will do this.

You can use the build _prefix_ to install and maintain different versions of
the tools. Doing this lets you try a new set of tools while not touching your
proven working production set of tools. Once you have proven the new tools are
working rebuild with the 'production' prefix switching your development to them.

I also suggest you keep your environment to the bare minimum, particularly the
path variable. Using environment variables has been proven over the years to be
difficult to manage in production systems.

Setup
~~~~~

Setup a development work space:

-------------------------------------------------------------
$ cd
$ mkdir -p development/rtems/src
$ cd development/rtems/src
-------------------------------------------------------------

The RTEMS Source Builder is distributed as source. It is Python code so all you
need to do is head over to the RTEMS GIT repository and clone the code directly
from git:

-------------------------------------------------------------
$ git clone git://git.rtems.org/chrisj/rtems-source-builder.git
$ cd rtems-source-builder
-------------------------------------------------------------

Checking
~~~~~~~~

The next step is to check if your host is set up correctly. The RTEMS Source
Builder provides a tool to help:

-------------------------------------------------------------
$ source-builder/sb-check
warning: exe: absolute exe found in path: (__objcopy) /usr/local/bin/objcopy <1>
warning: exe: absolute exe found in path: (__objdump) /usr/local/bin/objdump
error: exe: not found: (_xz) /usr/local/bin/xz <2>
RTEMS Source Builder environment is not correctly set up
$ source-builder/sb-check
RTEMS Source Builder environment is ok <3>
-------------------------------------------------------------

<1> A tool is in the environment path but does not match the expected path.
<2> The executable 'xz' is not found.
<3> The host's environment is set up correct.

The checking tool will output a list of executable files not found if problems
are detected. Locate those executable files and install them. You may also be
given a list of warnings about executable files not in the expected location
however the executable was located somewhere in your environment's path. You
will need to check each tool to determine if this is an issue. An executable
not in the standard location may indicate it is not the host operating system's
standard tool. It maybe ok or it could be buggy, only you can determine this.

The <<_host_setups,Host Setups>> section lists packages you should install for
common host operating systems. It maybe worth checking if you have those
installed.

Build Sets
~~~~~~~~~~

The RTEMS tools can be built within the RTEMS Source Builder's source tree. We
recommend you do this so lets change into the RTEMS directory in the RTEMS
Source Builder package:

-------------------------------------------------------------
$ cd rtems
-------------------------------------------------------------

If you are unsure how to specify the build set for the architecture you wish to
build ask the tool:

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --list-bsets <1>
RTEMS Source Builder - Set Builder, v0.2.0
Examining: config <2>
Examining: ../source-builder/config <2>
    4.10/rtems-all.bset <3>
    4.10/rtems-arm.bset <4>
    4.10/rtems-autotools.bset
    4.10/rtems-avr.bset
    4.10/rtems-bfin.bset
    4.10/rtems-h8300.bset
    4.10/rtems-i386.bset
    4.10/rtems-lm32.bset
    4.10/rtems-m32c.bset
    4.10/rtems-m32r.bset
    4.10/rtems-m68k.bset
    4.10/rtems-mips.bset
    4.10/rtems-nios2.bset
    4.10/rtems-powerpc.bset
    4.10/rtems-sh.bset
    4.10/rtems-sparc.bset
    4.11/rtems-all.bset
    4.11/rtems-arm.bset
    4.11/rtems-autotools.bset
    4.11/rtems-avr.bset
    4.11/rtems-bfin.bset
    4.11/rtems-h8300.bset
    4.11/rtems-i386.bset
    4.11/rtems-lm32.bset
    4.11/rtems-m32c.bset
    4.11/rtems-m32r.bset
    4.11/rtems-m68k.bset
    4.11/rtems-microblaze.bset
    4.11/rtems-mips.bset
    4.11/rtems-moxie.bset
    4.11/rtems-nios2.bset
    4.11/rtems-powerpc.bset
    4.11/rtems-sh.bset
    4.11/rtems-sparc.bset
    4.11/rtems-sparc64.bset
    4.11/rtems-v850.bset
    4.9/rtems-all.bset
    4.9/rtems-arm.bset
    4.9/rtems-autotools.bset
    4.9/rtems-i386.bset
    4.9/rtems-m68k.bset
    4.9/rtems-mips.bset
    4.9/rtems-powerpc.bset
    4.9/rtems-sparc.bset
    gnu-tools-4.6.bset
    rtems-4.10-base.bset <5>
    rtems-4.11-base.bset
    rtems-4.9-base.bset
    rtems-base.bset <5>
    unstable/4.11/rtems-all.bset <6>
    unstable/4.11/rtems-arm.bset
    unstable/4.11/rtems-avr.bset
    unstable/4.11/rtems-bfin.bset
    unstable/4.11/rtems-h8300.bset
    unstable/4.11/rtems-i386.bset
    unstable/4.11/rtems-lm32.bset
    unstable/4.11/rtems-m32c.bset
    unstable/4.11/rtems-m32r.bset
    unstable/4.11/rtems-m68k.bset
    unstable/4.11/rtems-microblaze.bset
    unstable/4.11/rtems-mips.bset
    unstable/4.11/rtems-moxie.bset
    unstable/4.11/rtems-powerpc.bset
    unstable/4.11/rtems-sh.bset
    unstable/4.11/rtems-sparc.bset
    unstable/4.11/rtems-sparc64.bset
    unstable/4.11/rtems-v850.bset
-------------------------------------------------------------
<1> Only option needed is +--list-bsets+
<2> The paths inspected. See <<X1,+_configdir+>> variable.
<3> Build all RTEMS 4.10 supported architectures.
<4> The build set for the ARM architecture on RTEMS 4.10.
<5> Support build set file with common functionality included by other build
    set files.
<6> Unstable tool sets are used by RTEMS developers to test new tool sets. You
    are welcome to try them but you must remember they are unstable, can change
    at any point in time and your application may not work. If you have an
    issue with a production tool it may pay to try the unstable tool to see if
    it has been resolved.

Building
~~~~~~~~

In this quick start I will build a SPARC tool set.

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-sparc.txt <1> \
                --prefix=$HOME/development/rtems/4.11 <2> 4.11/rtems-sparc <3>
Source Builder - Set Builder, v0.2.0
Build Set: 4.11/rtems-sparc
config: expat-2.1.0-1.cfg <4>
package: expat-2.1.0-x86_64-freebsd9.1-1
building: expat-2.1.0-x86_64-freebsd9.1-1
config: tools/rtems-binutils-2.22-1.cfg <5>
package: sparc-rtems4.11-binutils-2.22-1
building: sparc-rtems4.11-binutils-2.22-1
config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg <6>
package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
config: tools/rtems-gdb-7.5.1-1.cfg <7>
package: sparc-rtems4.11-gdb-7.5.1-1
building: sparc-rtems4.11-gdb-7.5.1-1
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <8>
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
cleaning: expat-2.1.0-x86_64-freebsd9.1-1 <9>
cleaning: sparc-rtems4.11-binutils-2.22-1
cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
cleaning: sparc-rtems4.11-gdb-7.5.1-1
Build Set: Time 0:13:43.616383 <10>
-------------------------------------------------------------

<1> Providing a log file redirects the build output into a file. Logging the
    build output provides a simple way to report problems.
<2> The prefix is the location on your file system the tools are installed
    into. Provide a prefix to a location you have read and write access. You
    can use the prefix to install different versions or builds of tools. Just
    use the path to the tools you want to use when building RTEMS.
<3> The build set. This is the SPARC build set.
<4> The SPARC build set first builds the expat library as is used in GDB. This
    is the expat configuration used.
<5> The binutils build configuration.
<6> The GCC and Newlib build configuration.
<7> The GDB build configuration.
<8> Installing the built packages to the install prefix.
<9> All the packages built are cleaned at the end. If the build fails all the
    needed files are present. You may have to clean up by deleting the build
    directory if the build fails.
<10> The time to build the package. This lets you see how different host
     hardware or configurations perform.

Your SPARC RTEMS 4.11 tool set will be installed and ready to build RTEMS and
RTEMS applications. When the build runs you will notice the tool fetch the
source code from the internet. These files are cached in a directory called
+source+. If you run the build again the cached files are used. This is what
happened in the show example before.

The installed tools:

-------------------------------------------------------------
$ ls $HOME/development/rtems/4.11
bin         include     lib         libexec     share       sparc-rtems4.11
$ ls $HOME/development/rtems/4.11/bin
sparc-rtems4.11-addr2line       sparc-rtems4.11-cpp
sparc-rtems4.11-gcc-ar          sparc-rtems4.11-gprof
sparc-rtems4.11-objdump         sparc-rtems4.11-size
sparc-rtems4.11-ar              sparc-rtems4.11-elfedit
sparc-rtems4.11-gcc-nm          sparc-rtems4.11-ld
sparc-rtems4.11-ranlib          sparc-rtems4.11-strings
sparc-rtems4.11-as              sparc-rtems4.11-g++
sparc-rtems4.11-gcc-ranlib      sparc-rtems4.11-ld.bfd
sparc-rtems4.11-readelf         sparc-rtems4.11-strip
sparc-rtems4.11-c++             sparc-rtems4.11-gcc
sparc-rtems4.11-gcov            sparc-rtems4.11-nm
sparc-rtems4.11-run             xmlwf
sparc-rtems4.11-c++filt         sparc-rtems4.11-gcc-4.7.2
sparc-rtems4.11-gdb             sparc-rtems4.11-objcopy
sparc-rtems4.11-sis
$ $HOME/development/rtems/4.11/bin/sparc-rtems4.11-gcc -v
Using built-in specs.
COLLECT_GCC=/home/chris/development/rtems/4.11/bin/sparc-rtems4.11-gcc
COLLECT_LTO_WRAPPER=/usr/home/chris/development/rtems/4.11/bin/../ \
libexec/gcc/sparc-rtems4.11/4.7.2/lto-wrapper
Target: sparc-rtems4.11 <1>
Configured with: ../gcc-4.7.2/configure <2>
--prefix=/home/chris/development/rtems/4.11
--bindir=/home/chris/development/rtems/4.11/bin
--exec_prefix=/home/chris/development/rtems/4.11
--includedir=/home/chris/development/rtems/4.11/include
--libdir=/home/chris/development/rtems/4.11/lib
--libexecdir=/home/chris/development/rtems/4.11/libexec
--mandir=/home/chris/development/rtems/4.11/share/man
--infodir=/home/chris/development/rtems/4.11/share/info
--datadir=/home/chris/development/rtems/4.11/share
--build=x86_64-freebsd9.1 --host=x86_64-freebsd9.1 --target=sparc-rtems4.11
--disable-libstdcxx-pch --with-gnu-as --with-gnu-ld --verbose --with-newlib
--with-system-zlib --disable-nls --without-included-gettext
--disable-win32-registry --enable-version-specific-runtime-libs --disable-lto
--enable-threads --enable-plugin --enable-newlib-io-c99-formats
--enable-newlib-iconv --enable-languages=c,c++
Thread model: rtems <3>
gcc version 4.7.2 20120920 <4>
 (RTEMS4.11-RSB(cb12e4875c203f794a5cd4b3de36101ff9a88403)-1,gcc-4.7.2/newlib-2.0.0) (GCC)
-------------------------------------------------------------

<1> The target the compiler is built for.
<2> The configure options used to build GCC.
<3> The threading model is always RTEMS. This makes the RTEMS tools difficult
    for bare metal development more difficult.
<4> The version string. It contains the Git hash of the RTEMS Source Builder
    you are using. If your local clone has been modifed that state is also
    recorded in the version string. The hash allows you to track from a GCC
    executable back to the original source used to build it.

NOTE: The RTEMS thread model enables specific hooks in GCC so applications
built with RTEMS tools need the RTEMS runtime to operate correctly. You can use
RTEMS tools to build bare metal component but it is more difficult than with a
bare metal tool chain and you need to know what you are doing at a low
level. The RTEMS Source Builder can build bare metal tool chains.

Installing and Tar Files
~~~~~~~~~~~~~~~~~~~~~~~~

The RTEMS Source Builder can install the built packages directly and optionally
it can create a build set tar file or a tar file per package built. The normal
and default behaviour is to let the RTEMS Source Builder install the tools. The
source will be downloaded, built, installed and cleaned up.

The tar files are created with the full build prefix present and if you follow
the examples given in this documentation the path is absolute. This can cause
problems if you are installing on a host you do not have super user or
administrator rights on because the prefix path may references part you do not
have write access too and tar will not extract the files. You can use the
+--strip-components+ option in tar if your host tar application supports it to
remove the parts you do not have write access too or you may need to unpack the
tar file somewhere and copy the file tree from the level you have write access
from. Embedding the full prefix path in the tar files lets you know what the
prefix is and is recommended. For example if
`/home/chris/development/rtems/4.11` is the prefix used you cannot change
directory to the root (`/`) and install because the `/home` is root access
only. To install you would:

-------------------------------------------------------------
$ cd
$ tar --strip-components=3 -xjf rtems-4.11-sparc-rtems4.11-1.tar.bz2
-------------------------------------------------------------

A build set tar file is created by adding `--bset-tar-file` option to the
`sb-set-builder` command.

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-sparc.txt \
         --prefix=$HOME/development/rtems/4.11 \
         --bset-tar-file <1> 4.11/rtems-sparc
Source Builder - Set Builder, v0.2.0
Build Set: 4.11/rtems-sparc
config: expat-2.1.0-1.cfg
package: expat-2.1.0-x86_64-freebsd9.1-1
building: expat-2.1.0-x86_64-freebsd9.1-1
config: tools/rtems-binutils-2.22-1.cfg
package: sparc-rtems4.11-binutils-2.22-1
building: sparc-rtems4.11-binutils-2.22-1
config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
config: tools/rtems-gdb-7.5.1-1.cfg
package: sparc-rtems4.11-gdb-7.5.1-1
building: sparc-rtems4.11-gdb-7.5.1-1
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11 <2>
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
installing: rtems-4.11-sparc-rtems4.11-1 -> /home/chris/development/rtems/4.11
tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
cleaning: expat-2.1.0-x86_64-freebsd9.1-1
cleaning: sparc-rtems4.11-binutils-2.22-1
cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
cleaning: sparc-rtems4.11-gdb-7.5.1-1
Build Set: Time 0:15:25.92873
-------------------------------------------------------------

<1> The option to create a build set tar file.
<2> The installation still happens unless you specify `--no-install`.
<3> Creating the build set tar file.

You can also suppress installing the files using the `--no-install` option to
the `sb-set-builder` command.

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-sparc.txt \
            --prefix=$HOME/development/rtems/4.11 \
            --bset-tar-file --no-install <1> 4.11/rtems-sparc
Source Builder - Set Builder, v0.2.0
Build Set: 4.11/rtems-sparc
config: expat-2.1.0-1.cfg
package: expat-2.1.0-x86_64-freebsd9.1-1
building: expat-2.1.0-x86_64-freebsd9.1-1
config: tools/rtems-binutils-2.22-1.cfg
package: sparc-rtems4.11-binutils-2.22-1
building: sparc-rtems4.11-binutils-2.22-1
config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
config: tools/rtems-gdb-7.5.1-1.cfg
package: sparc-rtems4.11-gdb-7.5.1-1
building: sparc-rtems4.11-gdb-7.5.1-1
tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2 <2>
cleaning: expat-2.1.0-x86_64-freebsd9.1-1
cleaning: sparc-rtems4.11-binutils-2.22-1
cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
cleaning: sparc-rtems4.11-gdb-7.5.1-1
Build Set: Time 0:14:11.721274
$ ls tar
rtems-4.11-sparc-rtems4.11-1.tar.bz2
-------------------------------------------------------------

<1> The option to supressing installing the packages.
<2> Create the build set tar.

A package tar file can be created by adding the +--pkg-tar-files+ to the
+sb-set-builder+ command. This creates a tar file per package built in the
build set.

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-sparc.txt \
            --prefix=$HOME/development/rtems/4.11 \
            --bset-tar-file --pkg-tar-files <1> --no-install 4.11/rtems-sparc
Source Builder - Set Builder, v0.2.0
Build Set: 4.11/rtems-sparc
config: expat-2.1.0-1.cfg
package: expat-2.1.0-x86_64-freebsd9.1-1
building: expat-2.1.0-x86_64-freebsd9.1-1
config: tools/rtems-binutils-2.22-1.cfg
package: sparc-rtems4.11-binutils-2.22-1
building: sparc-rtems4.11-binutils-2.22-1
config: tools/rtems-gcc-4.7.2-newlib-1.20.0-1.cfg
package: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
building: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
config: tools/rtems-gdb-7.5.1-1.cfg
package: sparc-rtems4.11-gdb-7.5.1-1
building: sparc-rtems4.11-gdb-7.5.1-1
tarball: tar/rtems-4.11-sparc-rtems4.11-1.tar.bz2
cleaning: expat-2.1.0-x86_64-freebsd9.1-1
cleaning: sparc-rtems4.11-binutils-2.22-1
cleaning: sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1
cleaning: sparc-rtems4.11-gdb-7.5.1-1
Build Set: Time 0:14:37.658460
$ ls tar
expat-2.1.0-x86_64-freebsd9.1-1.tar.bz2           sparc-rtems4.11-binutils-2.22-1.tar.bz2
sparc-rtems4.11-gdb-7.5.1-1.tar.bz2 <2>           rtems-4.11-sparc-rtems4.11-1.tar.bz2 <3>
sparc-rtems4.11-gcc-4.7.2-newlib-1.20.0-1.tar.bz2
-------------------------------------------------------------

<1> The option to create packages tar files.
<2> The GDB package tar file.
<3> The build set tar file. All the others in a single tar file.

Why Build from Source ?
-----------------------

The RTEMS Source Builder is not a replacement for the binary install systems
you have with commercial operating systems or open source operating system
distributions. Those products and distributions are critically important and
are the base that allows the Source Builder to work. The RTEMS Source Builder
sits somewhere between you manually entering the commands to build a tool set
and a tool such as +yum+ or +apt-get+ to install binary packages made
specifically for your host operating system. Building manually or installing a
binary package from a remote repository are valid and real alternatives while
the Source Builder is attempting to provide a specific service of repeatably
being able to build tool sets from source code.

If you are developing a system or product that has a long shelf life or is used
in a critical piece of infrastructure that has a long life cycle being able to
build from source is important. It insulates the project from the fast ever
changing world of the host development machines. If your tool set is binary and
you have lost the ability to build it you have lost a degree of control and
flexibility open source gives you. Fast moving host environments are
fantastic. We have powerful multi-core computers with huge amounts of memory
and state of the art operating systems to run on them however the product or
project you are part of may need to be maintained well past the life time of
these host. Being able to build from source an important and critical part of
this process because you can move to a newer host and create an equivalent tool
set.

Building from source provides you with control over the configuration of the
package you are building. If all or the most important dependent parts are
built from source you limit the exposure to host variations. For example the
GNU C compiler (gcc) currently uses a number of 3rd party libraries internally
(gmp, mpfr, etc). If your validated compiler generating code for your target
processor is dynamically linked against the host's version of these libraries
any change in the host's configuration may effect you. The changes the host's
package management system makes may be perfectly reasonable in relation to the
distribution being managed however this may not extend to you and your
tools. Building your tools from source and controlling the specific version of
these dependent parts means you are not exposing yourself to unexpected and
often difficult to resolve problems. On the other side you need to make sure
your tools build and work with newer versions of the host operating
system. Given the stability of standards based libraries like 'libc' and ever
improving support for standard header file locations this task is becoming
easier.

The RTEMS Source Builder is designed to be audited and incorporated into a
project's verification and validation process. If your project is developing
critical applications that needs to be traced from source to executable code in
the target, you need to also consider the tools and how to track them.

If your IT department maintains all your computers and you do not have suitable
rights to install binary packages, building from source lets you create your
own tool set that you install under your home directory. Avoiding installing
any extra packages as a super user is always helpful in maintaining a secure
computing environment.

Bugs, Crashes, and Build Failures
---------------------------------

The RTEMS Source Builder is a Python program and every care is taken to test
the code however bugs, crashes, and build failure can and do happen. If you
find a bug please report it via the RTEMS Bug tracker tool Bugzilla:

https://www.rtems.org/bugzilla/

or via email on the RTEMS Users list.

Please include the git hash you are running, the host details with the output
of the +uname -a+ command. If there is a crash please cut and paste the Python
backtrace into the bug report. If the tools fail to build please locate the
first error in the log file. This can be difficult to find on hosts with many
cores so it sometimes pays to run the command with the +--jobs=none+ option to
get a log that is correctly sequenced.

Configuration
-------------

The RTEMS Source Builder has two types of configuration data:

. Build Sets
. Package Build Configurations

By default these files can be located in two separate directories and
searched. The first directory is +config+ in your current working directory
(+_topdir+) and the second is +config+ located in the base directory of the
RTEMS Source Builder command you run (+_sbdir+). The RTEMS directory +rtems+
located at the top of the RTEMS Source Builder source code is an example of a
specific build configuration directory. You can create custom or private build
configurations and if you run the RTEMS Source Builder command from that
directory your configurations will be used.

[[X1]] The configuration search path is a macro variable and is reference as
+%\{_configdir\}+. It's default is defined as:

-------------------------------------------------------------
_configdir          : dir      optional   %{_topdir}/config:%{_sbdir}/config <1>
-------------------------------------------------------------

<1> The +_topdir+ is the directory you run the command from and +_sbdir+ is the
location of the RTEMS Source Builder command.

Build set files have the file extension +.bset+ and the package build
configuration files have the file extension of +.cfg+. The +sb-set-builder+
command will search for _build sets_ and the +sb-builder+ commands works with
package build configuration files.

Both types of configuration files use the \'#' character as a comment
character. Anything after this character on the line is ignored. There is no
block comment.

Source and Patches
~~~~~~~~~~~~~~~~~~

The RTEMS Source Builder provides a flexible way to manage source. Source and
patches are declare in configurations file using the +source+ and +patch+
directives. There are a single line containing a Universal Resource Location or
URL and can contain macros and shell expansions. The <<_prep,%prep>> section
details the source and patch directives

The URL can reference remote and local source and patch resources. The
following schemes are provided:

'http':: Remote access using the HTTP protocol.
'https':: Remote access using the Secure HTTP protocol.
'ftp:: Remote access using the FTP protocol.
'git:: Remote access to a GIT repository.
'cvs:: Remote access to a CVS repository.
'file:: Local access to an existing source directory.

HTTP, HTTPS, and FTP
^^^^^^^^^^^^^^^^^^^^

Remote access to TAR files is provided using HTTP, HTTPS and FTP protocols. The
full URL provided is used to access the remote file including any query
components. The URL is parsed to extract the file component and the local
source directory is checked for that file. If the file is located the remote
file is not downloaded. Currently no other checks are made. If a download fails
you need to manually remove the file from the source directory and start the
build process again.

The URL can contain macros. These are expand before issuing the request to
download the file. The GNU GCC compiler source URL is:

-------------------------------------------------------------
Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
-------------------------------------------------------------

The type of compression is automatically detected from the file extension. The
supported compression formats are:

`gz`:: GNU ZIP
`bzip2`:: BZIP2 ??
`zip`:: ??
'xy':: XY ??

The output of the decompression tool is feed to the standard `tar` utility and
unpacked into the build directory.

If the URL references the GitHub API server 'https://api.github.com/' a tarball
of the specified version is download. For example the URL for the STLINK
project on GitHub and version is:

-------------------------------------------------------------
%define stlink_version 3494c11
Source0: https://api.github.com/repos/texane/stlink/texane-stlink-%{stlink_version}.tar.gz
-------------------------------------------------------------

the TAR file is downloaded and used.

GIT
^^^

A GIT repository can be cloned and used as source. The GIT repository resides
in the 'source' directory under the `git` directory. You can edit, update and
use the repository as you normally do and the results will used to build the
tools. This allows you to prepare and test patches in the build environment the
tools are built in. The GIT URL only supports the GIT protocol. You can control
the repository via the URL by appending options and arguments to the GIT
path. The options are delimited by `?` and option arguments are delimited from
the options with `=`. The options are:

`branch`:: Checkout the specified branch.
`pull`:: Perform a pull to update the repository.
`fetch`:: Perform a fetch to get any remote updates.
`reset`:: Reset the repository. Useful to remove any local changes. You can
pass the `hard` argument to force a hard reset.

-------------------------------------------------------------
Source0: git://gcc.gnu.org/git/gcc.git?branch=gcc-4_7-branch?reset=hard
-------------------------------------------------------------

This will clone the GCC git repository and checkout the 4.7-branch and perform
a hard reset.

CVS
^^^

A CVS repository can be checked out. CVS is more complex than GIT to handle
because of the modules support. This can effect the paths the source ends up
in. The CVS URL only supports the CVS protocol. You can control the repository
via the URL by appending options and arguments to the CVS path. The options are
delimited by `?` and option arguments are delimited from the options with
`=`. The options are:

`module`:: The module to checkout.
`src-prefix`:: The path into the source where the module starts.
`tag`:: The CVS tag to checkout.
`date`:: The CVS date to checkout.

-------------------------------------------------------------
Source0: cvs://pserver:anoncvs@sourceware.org/cvs/src?module=newlib?src-prefix=src
-------------------------------------------------------------

Macros and Defaults
~~~~~~~~~~~~~~~~~~~

The RTEMS Source Builder uses tables of _macros_ read in when the tool
runs. The initial global set of macros is called the _defaults_. These values
are read from a file called `defaults.mc` and modified to suite your host. This
host specific adaption lets the Source Builder handle differences in the build
hosts.

Build set and configuration files can define new values updating and extending
the global macro table. For example builds are given a release number. This is
typically a single number at the end of the package name. For example:

-------------------------------------------------------------
%define release 1
-------------------------------------------------------------

Once defined if can be accessed in a build set or package configuration file
with:

-------------------------------------------------------------
%{release}
-------------------------------------------------------------

The +sb-defaults+ command lists the defaults for your host. I will not include
the output of this command becauses of its size.

-------------------------------------------------------------
$ ../source-builder/sb-defaults
-------------------------------------------------------------

A nested build set is given a separate copy of the global macro maps. Changes
in one change set are not seen in other build sets. That same happens with
configuration files unless inline includes are used. Inline includes are seen
as part of the same build set and configuration and changes are global to that
build set and configuration.

Macro Maps and Files
^^^^^^^^^^^^^^^^^^^^

Macros are read in from files when the tool starts. The default settings are
read from the defaults macro file called `defaults.mc` located in the top level
RTEMS Source Builder command directory. User macros can be read in at start up
by using the `--macros` command line option.

The format for a macro in macro files is:

[options="header,compact",width="50%",cols="15%,15%,15%,55%"]
|=================================
| Name | Type | Attribute | String
|=================================

where 'Name' is a case insensitive macro name, the 'Type' field is:

[horizontal]
`none`:: Nothing, ignore.
`dir`:: A directory path.
`exe`:: An executable path.
`triplet`:: A GNU style architecture, platform, operating system string.

the 'Attribute' field is:

[horizontal]
`none`:: Nothing, ignore
`required`:: The host check must find the executable or path.
`optional`:: The host check generates a warning if not found.
`override`:: Only valid outside of the `global` map to indicate this macro
             overrides the same one in the `global` map when the map containing
             it is selected.
`undefine`:: Only valid outside of the `global` map to undefine the macro if it
             exists in the `global` map when the map containing it is
             selected. The `global` map's macro is not visible but still
             exists.

and the 'String' field is a single or tripled multiline quoted string. The
'String' can contain references to other macros. Macro that loop are not
currently detected and will cause the tool to lock up.

Maps are declared anywhere in the map using:

-------------------------------------------------------------
# Comments
[my-special-map]
_host:  none, override, 'abc-xyz'
multiline: none, override, '''First line,
second line,
and finally the last line'''
-------------------------------------------------------------

Any macro defintions following a map declaration are placed in that map and the
default map is `global` when loading a file. Maps are selected in configuration
files by using the `%select` directive.

-------------------------------------------------------------
%select my-special-map
-------------------------------------------------------------

Selecting a map means all requests for a macro first check the selected map and
if present return that value else the `global` map is used. Any new macros or
changes update only the `global` map. This may change in future releases so
please make sure you use the `override` attibute.

The macro files are looked for in the `_configdir` paths. See
<<X1,+_configdir+>> variable for details.

Build Set Files
~~~~~~~~~~~~~~~

Build set files lets you list the packages in the build set you are defining
and have a file extension of +.bset+. Build sets can define macro variables,
inline include other files and reference other build set or package
configuration files.

Defining macros is performed with the +%define+ macro:

-------------------------------------------------------------
%define _target m32r-rtems4.11
-------------------------------------------------------------

Inline including another file with the +%include+ macro continues processing
with the specified file returning to carry on from just after the include
point.

-------------------------------------------------------------
%include rtems-4.11-base.bset
-------------------------------------------------------------

This includes the RTEMS 4.11 base set of defines and checks. The configuration
paths as defined by +_configdir+ are scanned. The file extension is optional.

You reference build set or package configuration files by placing the file name
on a single line.

-------------------------------------------------------------
tools/rtems-binutils-2.22-1
-------------------------------------------------------------

The +_configdir+ path is scanned for +tools/rtems-binutils-2.22-1.bset+ or
+tools/rtems-binutils-2.22-1.cfg+. Build set files take precedent over package
configuration files. If +tools/rtems-binutils-2.22-1+ is a build set a new
instance of the build set processor is created and if the file is a package
configuration the package is built with the package builder. This all happens
once the build set file has finished being scanned.

Configuration Control
~~~~~~~~~~~~~~~~~~~~~

The RTEMS Souce Builder is designed to fit within most verification and
validation processes. All of the RTEMS Source Builder is source code. The
Python code is source and comes with a commercial friendly license. All
configuration data is text and can be read or parsed with standard text based
tools.

File naming provides configuration management. A specific version of a package
is captured in a specific set of configuration files. The top level
configuration file referenced in a _build set_ or passed to the +sb-builder+
command relates to a specific configuration of the package being built. For
example the RTEMS configuration file +rtems-gcc-4.7.2-newlib-2.0.0-1.cfg+
creates an RTEMS GCC and Newlib package where the GCC version is 4.7.2, the
Newlib version is 2.0.0, plus any RTEMS specific patches that related to this
version. The configuration defines the version numbers of the various parts
that make up this package:

-------------------------------------------------------------
%define gcc_version    4.7.2
%define newlib_version 2.0.0
%define mpfr_version   3.0.1
%define mpc_version    0.8.2
%define gmp_version    5.0.5
-------------------------------------------------------------

The package build options, if there are any are also defined:

-------------------------------------------------------------
%define with_threads 1
%define with_plugin  0
%define with_iconv   1
-------------------------------------------------------------

The generic configuration may provide defaults in case options are not
specified. The patches this specific version of the package requires can be
included:

-------------------------------------------------------------
Patch0: gcc-4.7.2-rtems4.11-20121026.diff
-------------------------------------------------------------

Finally including the GCC 4.7 configuration script:

-------------------------------------------------------------
%include %{_configdir}/gcc-4.7-1.cfg
-------------------------------------------------------------

The +gcc-4.7-1.cfg+ file is a generic script to build a GCC 4.7 compiler with
Newlib. It is not specific to RTEMS. A bare no operating system tool set can be
built with this file.

The +-1+ part of the file names is a revision. The GCC 4.7 script maybe revised
to fix a problem and if this fix effects an existing script the file is copied
and given a +-2+ revision number. Any dependent scripts referencing the earlier
revision number will not be effected by the change. This locks down a specific
configuration over time.

Snapshot Testing
~~~~~~~~~~~~~~~~

Testing of release canidates and snapshots is important to those helping
maintain tool sets. The RTEMS Source Builder helps by providing a simple and
flexible way to use existing build sets and configuration without needing to
change them or creating new temporary build sets and configurations.

The process uses snapshot macro files loaded via the command line option
`--macros`. These files provide macros that override the standard build set and
configuration file macros.

Lets consider testing a GCC 4.7 snapshot for RTEMS 4.11. Lets assume the
current RTEMS 4.11 tools reference GCC 4.7.3 with a patch as the stable tool
set. We want to use a recent snapshot with no patches. In the
`rtems/config/snapshots` directoy create a file called `gcc-4.7-snapshot.mc`
containing:

-------------------------------------------------------------
[gcc-4.7-snapshot]
GCC_Version: none, override, '4.7-20130413'
Source0:     none, override, 'http://mirrors.kernel.org/sources.redhat.com/gcc/
snapshots/%{gcc_version}/gcc-%{gcc_version}.tar.bz2'
Patch0:      none, udefine,  ''
-------------------------------------------------------------

In the standard configuration file `source-builder/config/gcc-4.7-1.cfg` the
map is selected with:

-------------------------------------------------------------
#
# Select the GCC 4.7 Snapshot Macro Map
#
%select gcc-4.7-snapshot
-------------------------------------------------------------

On the command line add `--macros=snapshots/gcc-4.7-snapshot.mc` and this
snapshot will be built. With careful use of the `--prefix` option you can
locate the tools in a specific directory and test them without needing to
effect your production environment.

Scripting
~~~~~~~~~

Configuration files specify how to build a package. Configuration files are
scripts and have a +.cfg+ file extension. The script format is based loosely on
the RPM spec file format however the use and purpose in this tool does not
compare with the functionality and therefore the important features of the spec
format RPM needs and uses.

The script language is implemented in terms of macros. The built-in list is:

[horizontal]
+%{}+:: Macro expansion with conditional logic.
+%()+:: Shell expansion.
+%prep+:: The source preparation shell commands.
+%build+:: The build shell commands.
+%install+:: The package install shell commands.
+%clean+:: The package clean shell commands.
+%include+:: Inline include another configuration file.
+%name+:: The name of the package.
+%summary+:: A brief package description. Useful when reporting about a build.
+%release+:: The package release. A number that is the release as built by this tool.
+%version+:: The package's version string.
+%buildarch+:: The build architecture.
+%setup+:: Setup a source package.
+%source+:: Define a source code package. This macro has a number appended.
+%patch+:: Define a patch. This macro has a is number appended.
+%echo+:: Print the following string as a message.
+%warning+:: Print the following string as a warning and continue.
+%error+:: Print the following string as an error and exit.
+%select+:: Select the macro map. If there is no map nothing is reported.
+%define+:: Define a macro. Macros cannot be redefined, you must first undefine it.
+%undefine+:: Undefine a macro.
+%if+:: Start a conditional logic block that ends with a +%endif+.
+%ifn+:: Inverted start of a conditional logic block.
+%ifarch+:: Test the architecture against the following string.
+%ifnarch+:: Inverted test of the architecture
+%ifos+:: Test the host operating system.
+%else+:: Start the _else_ conditional logic block.
+%endfi+:: End the conditional logic block.
+%bconf_with+:: Test the build condition _with_ setting. This is the +--with-*+
command line option.
+%bconf_without+:: Test the build condition _without_ setting. This is the
+--without-*+ command line option.

Expanding
^^^^^^^^^

A macro can be `%{string}` or the equivalent of `%string`. The following macro
expansions supported are:

`%{string}`;;
Expand the 'string' replacing the entire macro text with the text in the table
for the entry 'string . For example if 'var' is 'foo' then `${var}` would
become `foo`.

`%{expand: string}`;;
Expand the 'string' and then use it as a ``string'' to the macro expanding the
macro. For example if _foo_ is set to 'bar' and 'bar' is set to 'foobar' then
`%{expand:foo}` would result in `foobar`. Shell expansion can also be used.

`%{with string}`;;
Expand the macro to '1' if the macro `with_`'string' is defined else expand to
_0_. Macros with the name `with_`'string' can be define with command line
arguments to the RTEMS Source Builder commands.

`%{defined string}`;;
Expand the macro to '1' if a macro of name 'string' is defined else expand to '0'.

`%{?string: expression}`;;
Expand the macro to 'expression' if a macro of name 'string' is defined else expand to `%{nil}`.

`%{!?string: expression}`;;
Expand the macro to 'expression' if a macro of name 'string' is not defined. If
the macro is define expand to `%{nil}`.

`%(expression)`;;
Expand the macro to the result of running the 'expression' in a host shell. It
is assumed this is a Unix type shell. For example `%(whoami)` will return your
user name and `%(date)` will return the current date string.

%prep
^^^^^

The +%prep+ macro starts a block that continues until the next block macro. The
_prep_ or preparation block defines the setup of the package's source and is a
mix of RTEMS Source Builder macros and shell scripting. The sequence is
typically +%setup+ macros for source, +%patch+ macros to patch the source
mixed with some shell commands to correct any source issues.  A +%prep+ section
starts with a +%setup+ command. This creates the package source top level
directory then is followed by the first source file.

-------------------------------------------------------------
                     <1>       <2>
%setup -q -c -T -n %{name}-%{version}
%setup -q -T -D -n %{name}-%{version} -a0
-------------------------------------------------------------

<1> The package's name.
<2> The version of the package.

The source for a package is declared with the +%source*+ macro where +*+ is
a number. For example +%source0+ is the source 0 tar file and is defined with
something similar to this:

-------------------------------------------------------------
Source0: http://ftp.gnu.org/gnu/gdb/gdb-%{gdb_version}.tar.bz2
-------------------------------------------------------------

This URL is the primary location of the GNU GCC source code and the RTEMS
Source Builder can download the file from this location and by inspecting the
file extension use +bzip2+ decompression. When the +%prep+ section is processed
a check of the local +source+ directory is made to see if the file has already
been downloaded. If not found in the source cache directory the package is
downloaded from the URL. You can append other base URLs via the command line
option +--url+. This option accepts a comma delimited list of sites to try.

You can combine the source macro with conditional logic to implement a default
that can be over-riden in the top level files. This lets you reuse a generic
build script with different sources. This happens if you have a private source
package with local modifications. The following example is taken from the
+gcc-4.8-1.cfg+ build script.

-------------------------------------------------------------
%ifn %{defined Source0}
 Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
%endif
-------------------------------------------------------------

You could optionally have a few source files that make up the package. For
example GNU's GCC was a few tar files for a while and it is now a single tar
file. Support for multiple source files can be conditionally implemented with
the following scripting:

-------------------------------------------------------------
%{?source1:%setup -q -T -D -n %{name}-%{version} -a1}
-------------------------------------------------------------

The +source1+ macro value is checked and if present the command string after
the +:+ is executed. It is common to see a number of these present allowing top
level configuration files including a base configuration the ability to define
a number of source packages.

-------------------------------------------------------------
%{?source1:%setup -q -T -D -n %{name}-%{version} -a1}
%{?source2:%setup -q -T -D -n %{name}-%{version} -a2}
%{?source3:%setup -q -T -D -n %{name}-%{version} -a3}
-------------------------------------------------------------

Patching also occurs during the preparation stage. Patches are handled in a
similar way to the source packages. Most patches are based around the top of
the source tree. This is an example of the patch scripting for the GCC 4.8
series of compilers:

-------------------------------------------------------------
cd gcc-%{gcc_version} <1>
%{?patch0:%patch0 -p1} <2>
%{?patch1:%patch1 -p1}
%{?patch2:%patch2 -p1}
%{?patch3:%patch3 -p1}
%{?patch4:%patch4 -p1}
%{?patch5:%patch5 -p1}
%{?patch6:%patch6 -p1}
%{?patch7:%patch7 -p1}
%{?patch8:%patch8 -p1}
%{?patch9:%patch9 -p1}
cd .. <3>
-------------------------------------------------------------

<1> Change from the top of the source tree into the package being patched's top
    directory.
<2> The conditional macro expansion checks if +%patch0+ is defined and if
    defined issues the +%patch0" macro giving +-p1+ to the patch command.
<3> Return back to the top of the source tree.

%build
^^^^^^

The +%build+ macro starts a block that continues until the next block
macro. The build block is a series of shell commands that execute to build the
package. It assumes all source code has been unpacked, patch and adjusted so
the build will succeed.

The following is an example take from the GutHub STLink project:

NOTE: STLink is a JTAG debugging device for the ST ARM family of processors.

-------------------------------------------------------------
%build
  export PATH="%{_bindir}:${PATH}" <1>

  cd texane-stlink-%{stlink_version} <2>

  ./autogen.sh <3>

%if "%{_build}" != "%{_host}"
  CFLAGS_FOR_BUILD="-g -O2 -Wall" \ <4>
%endif
  CPPFLAGS="-I $SB_TMPPREFIX/include/libusb-1.0" \ <5>
  CFLAGS="$SB_OPT_FLAGS" \
  LDFLAGS="-L $SB_TMPPREFIX/lib" \
  ./configure \ <6>
    --build=%{_build} --host=%{_host} \
    --verbose \
    --prefix=%{_prefix} --bindir=%{_bindir} \
    --exec-prefix=%{_exec_prefix} \
    --includedir=%{_includedir} --libdir=%{_libdir} \
    --mandir=%{_mandir} --infodir=%{_infodir}

  %{__make} %{?_smp_mflags} all <7>

  cd ..
-------------------------------------------------------------

<1> Setup the PATH environment variable. This is not always needed.
<2> This package builds in the source tree so enter it.
<3> The package is actually checked directly out from the github project and so
    it needs its autoconf and automake files generated.
<4> Flags for a cross-compiled build.
<5> Various settings passed to configure to customise the build. In this
    example an include path is being set to the install point of _libusb_. This
    package requires _libusb_ is built before it.
<6> The +configure+ command. The RTEMS Source Builder provides all the needed
    paths as macro variables. You just need to provide them to +configure+.
<7> Running make. Do not use +make+ directly, use the RTEMS Source Builder's
    defined value. This value is specific to the host. A large number of
    packages need GNU make and on BSD systems this is +gmake+. You can
    optionally add the SMP flags if the packages build system can handle
    parallel building with multiple jobs. The +_smp_mflags+ value is
    automatically setup for SMP hosts to match the number of cores the host has.

%install
^^^^^^^^

The +%install+ macro starts a block that continues until the next block
macro. The install block is a series of shell commands that execute to install
the package. You can assume the package has build correctly when this block
starts executing.

Never install the package to the actual _prefix_ the package was built
with. Always install to the RTEMS Source Builder's temporary path defined in
the macro variable +\__tmpdir+. The RTEMS Source Builder sets up a shell
environment variable called +SB_BUILD_ROOT+ as the standard install point. Most
packages support adding +DESTDIR=+ to the _make install_ command.

Looking at the same example as in <<_build, %build>>:

-------------------------------------------------------------
%install
  export PATH="%{_bindir}:${PATH}" <1>
  rm -rf $SB_BUILD_ROOT <2>

  cd texane-stlink-%{stlink_version} <3>
  %{__make} DESTDIR=$SB_BUILD_ROOT install <4>

  cd ..
-------------------------------------------------------------

<1> Setup the PATH environment variable. This is not always needed.
<2> Clean any installed files. This make sure the install is just what
    the package installs and not any left over files from a broken build or
    install.
<3> Enter the build directory. In this example it just happens to be the source
    directory.
<4> Run +make install+ to install the package overriding the +DESTDIR+ make
    variable.

%clean
^^^^^^

The +%clean+ macro starts a block that continues until the next block
macro. The clean block is a series of shell commands that execute to clean up
after a package has been built and install. This macro is currenly not been
used because the RTEMS Source Builder automatically cleans up.

%include
^^^^^^^^

The +%include+ macro inline includes the specific file. The +\__confdir+
path is searched. Any relative path component of the include file is appended
to each part of the +\__configdir+. Adding an extension is optional as files
with +.bset+ and +.cfg+ are automatically searched for.

Inline including means the file is processed as part of the configuration at
the point it is included. Parsing continues from the next line in the
configuration file that contains the +%include+ macro.

Including files allow a kind of configuration file reuse. The outer
configuration files provide specific information such as package version
numbers and patches and then include a generic configuration script which
builds the package.

-------------------------------------------------------------
%include %{_configdir}/gcc-4.7-1.cfg
-------------------------------------------------------------

%name
^^^^^

The name of the package being built. The name typically contains the components
of the package and their version number plus a revision number. For the GCC
with Newlib configuration the name is typically:

-------------------------------------------------------------
Name: %{_target}-gcc-%{gcc_version}-newlib-%{newlib_version}-%{release}
-------------------------------------------------------------

%summary
^^^^^^^^

The +%summary+ is a brief description of the package. It is useful when
reporting. This information is not capture in the package anywhere. For the GCC
with Newlib configuration the summary is typically:

-------------------------------------------------------------
Summary: GCC v%{gcc_version} and Newlib v%{newlib_version} for target %{_target} on host %{_host}
-------------------------------------------------------------

%release
^^^^^^^^

The +%release+ is packaging number that allows revisions of a package to happen
where none package versions change. This value typically increases when the
configuration building the package changes.

-------------------------------------------------------------
%define release 1
-------------------------------------------------------------

%version
^^^^^^^^

The +%version% macro sets the version the package. If the package is a single
component it tracks that component's version number. For example in the
_libusb_ configuration the +%version+ is the same as +%libusb_version+, however
in a GCC with Newlib configuration there is no single version number. In this
case the GCC version is used.

-------------------------------------------------------------
Version: %{gcc_version}
-------------------------------------------------------------

%buildarch
^^^^^^^^^^

The +%buildarch+ macro is set to the architecture the package contains. This is
currently not used in the RTEMS Source Builder and may go away. This macro is
more important in a real packaging system where the package could end up on the
wrong architecture.

%setup
^^^^^^

The +%setup+ macro sets up the source code tree and is used in the +%prep+
section of the script. The options are:

[horizontal]
*Switch*:: *Description*
+-n+:: The -n option is used to set the name of the software's build
directory. This is necessary only when the source archive unpacks into a
directory named other than +<name>-<version>+.
+-c+:: The -c option is used to direct %setup to create the top-level build
directory before unpacking the sources.
+-D+:: The -D option is used to direct %setup to not delete the build directory
prior to unpacking the sources. This option is used when more than one source
archive is to be unpacked into the build directory, normally with the +-b+ or
+-a+ options.
+-T+:: The -T option is used to direct %setup to not perform the default
unpacking of the source archive specified by the first Source: macro. It is used
with the +-a+ or +-b+ options.
+-b <n>+:: The -b option is used to direct %setup to unpack the source archive
specified on the nth Source: macro line before changing directory into the build
directory.
+-a <n>+:: The -a option is used to direct %setup to unpack the source archive
specified on the nth Source: macro line after changing directory into the build
directory.

%source
^^^^^^^

The +%source+ macro is numbered and defines a source tar file used in the
package. The +%setup+ macro references the packages defined here. A macro is
defined as:

-------------------------------------------------------------
Source0: ftp://ftp.gnu.org/gnu/gcc/gcc-%{gcc_version}/gcc-%{gcc_version}.tar.bz2
-------------------------------------------------------------

The setup script is:

-------------------------------------------------------------
%setup -q -T -D -n %{name}-%{version} -a0
-------------------------------------------------------------

The +-a0+ means use +%source0+.

%patch
^^^^^^

The +%patch+ macro is numbered and can define a patch and in the +%prep+
section applies the patch. To define a patch append a +:+ followed by the patch
filename:

-------------------------------------------------------------
Patch0: gcc-4.7.2-rtems4.11-20121026.diff
-------------------------------------------------------------

The +__patchdir+ path is search.

Placing +%patch+ in the +%prep+ section will apply it with any trailing options
passed to the +patch+ command. This allows the +-p+ option to be passed to
strip any leading path components from the patch contents.

-------------------------------------------------------------
%patch0 -p1
-------------------------------------------------------------

You will typically see the patch conditionally used as:

-------------------------------------------------------------
%{patch0:%patch0 -p1}
-------------------------------------------------------------

In this case the patch will only be applied if it is defined.

%echo
^^^^^

The +%echo+ macro outputs the following string to stdout. This can also be used
as `%{echo: message}`.

%warning
^^^^^^^^

The +%warning+ macro outputs the following string as a warning. This can also
be used as `%{warning: message}`.

%error
^^^^^^

The +%error+ macro outputs the follow string as an error and exits the RTEMS
Source Builder. This can also be used as `%{error: message}`.

%select
^^^^^^^

The +%select+ macro selects the map specified. If there is no map no error or
warning is generated. Macro maps provide a simple way for a user to override
the settings is a configuration file without having to edit it. The changes are
recorded in the build report so can be traced.

Configuration use different maps so macro overrides can target a specific
package.

The default map is `global'.

-------------------------------------------------------------
%select gcc-4.8-snapshot <1>
%define one_plus_one 2 <2>
-------------------------------------------------------------

<1> The map switches to `gcc-4.8-snapshot`. Any overrides in this map will be
    used.
<2> Defining macros only updates the `global` map and not the selected map.

%define
^^^^^^^

The +%define+ macro defines a new macro or updates an existing one. If no value
is given it is assumed to be 1.

-------------------------------------------------------------
%define foo bar
%define one_plus_one 2
%define one <1>
-------------------------------------------------------------

<1> The macro _one_ is set to 1.

%undefine
^^^^^^^^^

The +%undefine+ macro removes a macro if it exists. Any further references to
it will result in an undefine macro error.

%if
^^^

The +%if+ macro starts a conditional logic block that can optionally have a
_else_ section. A test follows this macro and can have the following operators:

[horizontal]
*Operator*:: *Description*
+%{}+:: Check the macro is set or _true_, ie non-zero.
+
-------------------------------------------------------------
%if ${foo}
 %warning The test passes, must not be empty or is non-zero
%else
 %error The test fails, must be empty or zero
%endif
-------------------------------------------------------------
+!+:: The _not_ operator inverts the test of the macro.
+
-------------------------------------------------------------
%if ! ${foo}
 %warning The test passes, must be empty or zero
%else
 %error The test fails, must not be empty or is non-zero
%endif
-------------------------------------------------------------
+==+:: The left hand size must equal the right hand side. For example:
+
-------------------------------------------------------------
%define one 1
%if ${one} == 1
 %warning The test passes
%else
 %error The test fails
%endif
-------------------------------------------------------------
+
You can also check to see if a macro is empty:
+
-------------------------------------------------------------
%if ${nothing} == %{nil}
 %warning The test passes
%else
 %error The test fails
%endif
-------------------------------------------------------------
+!=+:: The left hand size does not equal the right hand side. For example:
+
-------------------------------------------------------------
%define one 1
%if ${one} != 2
 %warning The test passes
%else
 %error The test fails
%endif
-------------------------------------------------------------
+
You can also check to see if something is set:
+
-------------------------------------------------------------
%if ${something} != %{nil}
 %warning The test passes
%else
 %error The test fails
%endif
-------------------------------------------------------------
+>+:: The left hand side is numerically greater than the right hand side.
+>=+:: The left hand side is numerically greater than or equal to the right
hand side.
+<+:: The left hand side is numerically less than the right hand side.
+\<=+:: The left hand side is numerically less than or equal to the right hand
side.

%ifn
^^^^

The +%ifn+ macro inverts the normal +%if+ logic. It avoids needing to provide
empty _if_ blocks followed by _else_ blocks. It is useful when checking if a
macro is defined:

-------------------------------------------------------------
%ifn %{defined foo}
 %define foo bar
%endif
-------------------------------------------------------------

%ifarch
^^^^^^^

The +%ifarch+ is a short cut for "+%if %{\_arch} == i386+". Currently not used.

%ifnarch
^^^^^^^^

The +%ifnarch+ is a short cut for "+%if %{\_arch} != i386+". Currently not
used.

%ifos
^^^^^

The +%ifos+ is a short cut for "+%if %{\_os} != mingw32+". It allows
conditional support for various operating system differences when building
packages.

%else
^^^^^

The +%else+ macro starts the conditional _else_ block.

%endfi
^^^^^^

The +%endif+ macro ends a conditional logic block.

%bconf_with
^^^^^^^^^^^

The +%bconf_with+ macro provides a way to test if the user has passed a
specific option on the command line with the +--with-<label>+ option. This
option is only available with the +sb-builder+ command.

%bconf_without
^^^^^^^^^^^^^^

The +%bconf_without+ macro provides a way to test if the user has passed a
specific option on the command line with the +--without-<label>+ option. This
option is only available with the +sb-builder+ command.


Project Sets
------------

The RTEMS Source Builder supports project configurations. Project
configurations can be public or private and can be contained in the RTEMS
Source Builder project if suitable, other projects they use the RTEM Source
Builder or privately on your local file system.

The configuration file loader searches the macro +_configdir+ and by default
this is set to +%{\_topdir}/config:%{\_sbdir}/config+ where +_topdir+ is the
your current working direct, in other words the directory you invoke the RTEMS
Source Builder command in, and +_sbdir+ is the directory where the RTEMS Source
Builder command resides. Therefore the +config+ directory under each of these
is searched so all you need to do is create a +config+ in your project and add
your configuration files. They do not need to be under the RTEMS Source Builder
source tree. Public projects are included in the main RTEMS Source Builder such
as RTEMS.

You can also add your own +patches+ directory next to your +config+ directory
as the +%patch+ command searches the +_patchdir+ macro variable and it is
by default set to +%{\_topdir}/patches:%{\_sbdir}/patches+.

The +source-builder/config+ directory provides generic scripts for building
various tools. You can specialise these in your private configurations to make
use of them. If you add new generic configurations please contribute them back
to the project

RTEMS Configurations
~~~~~~~~~~~~~~~~~~~~

The RTEMS Configurations are grouped by RTEMS version. In RTEMS the tools are
specific to a specific version because of variations between Newlib and
RTEMS. Restructuring in RTEMS and Newlib sometimes moves _libc_ functionality
between these two parts and this makes existing tools incompatible with RTEMS.

RTEMS allows architectures to have different tool versions and patches. The
large number of architectures RTEMS supports can make it difficult to get a
common stable version of all the packages. An architecture may require a recent
GCC because an existing bug has been fixed, however the more recent version may
have a bug in other architecture. Architecture specific patches should be
limited to the architecture it relates to. The patch may fix a problem on the
effect architecture however it could introduce a problem in another
architecture. Limit exposure limits any possible crosstalk between
architectures.

RTEMS supports _stable_ and _unstable_. Stable configurations are fixed while
unstable configurations are supporting using snapshots of user macros and
reference release candidates or source extracted directly from version
control. The stable build sets are referenced as +<version>/rtems-<arch>+ and
include `autoconf` and `automake`.

If you are building a released version of RTEMS the release RTEMS tar file will
be downloaded and built as part of the build process. If you are building a
tool set for use with the development branch of RTEMS, the development branch
will be cloned directly from the RTEMS GIT repository and built.

When building RTEMS within the RTEMS Source Builder it needs a suitable working
`autoconf` and `automake`. These packages need to built and installed in their
prefix in order for them to work. The RTEMS Source Builder installs all
packages only after they have been built so if you host does not have a
recent enough version of `autoconf` and `automake` you first need to build them
and install them then build your tool set. The commands are:

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-4.11-at.txt \
   --prefix=$HOME/development/rtems/4.11-at 4.11/rtems-autotools
$ export PATH=~/development/rtems/4.11/bin:$PATH
$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
   --prefix=$HOME/development/rtems/4.11 4.11/rtems-sparc
-------------------------------------------------------------

To build snapshots for testing purposes you use the available macro maps
passing them on the command line using the `--macros` option. For RTEMS these
are held in `config/snapshots` directory. The following build _newlib_ from
CVS:

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --log=l-4.11-sparc.txt \
   --prefix=$HOME/development/rtems/4.11 --macros=snapshots/newlib-head.mc
   4.11/rtems-sparc
-------------------------------------------------------------

Commands
--------

Checker (sb-check)
~~~~~~~~~~~~~~~~~~

This commands checks your system is set up correctly. Most options are ignored.

-------------------------------------------------------------
$ ../source-builder/sb-check --help
sb-check: [options] [args]
RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
Options and arguments:
--force                : Force the build to proceed
--quiet                : Quiet output (not used)
--trace                : Trace the execution
--dry-run              : Do everything but actually run the build
--warn-all             : Generate warnings
--no-clean             : Do not clean up the build tree
--always-clean         : Always clean the build tree, even with an error
--jobs                 : Run with specified number of jobs, default: num CPUs.
--host                 : Set the host triplet
--build                : Set the build triplet
--target               : Set the target triplet
--prefix path          : Tools build prefix, ie where they are installed
--topdir path          : Top of the build tree, default is $PWD
--configdir path       : Path to the configuration directory, default: ./config
--builddir path        : Path to the build directory, default: ./build
--sourcedir path       : Path to the source directory, default: ./source
--tmppath path         : Path to the temp directory, default: ./tmp
--macros file[,[file]  : Macro format files to load after the defaults
--log file             : Log file where all build out is written too
--url url[,url]        : URL to look for source
--no-download          : Disable the source downloader
--targetcflags flags   : List of C flags for the target code
--targetcxxflags flags : List of C++ flags for the target code
--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
--with-<label>         : Add the --with-<label> to the build
--without-<label>      : Add the --without-<label> to the build
--regression           : Set --no-install, --keep-going and --always-clean
$ ../source-builder/sb-check
RTEMS Source Builder - Check, v0.2.0
Environment is ok
-------------------------------------------------------------

Defaults (sb-defaults)
~~~~~~~~~~~~~~~~~~~~~~

This commands outputs and the default macros for your when given no
arguments. Most options are ignored.

-------------------------------------------------------------
$ ../source-builder/sb-defaults --help
sb-defaults: [options] [args]
RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
Options and arguments:
--force                : Force the build to proceed
--quiet                : Quiet output (not used)
--trace                : Trace the execution
--dry-run              : Do everything but actually run the build
--warn-all             : Generate warnings
--no-clean             : Do not clean up the build tree
--always-clean         : Always clean the build tree, even with an error
--jobs                 : Run with specified number of jobs, default: num CPUs.
--host                 : Set the host triplet
--build                : Set the build triplet
--target               : Set the target triplet
--prefix path          : Tools build prefix, ie where they are installed
--topdir path          : Top of the build tree, default is $PWD
--configdir path       : Path to the configuration directory, default: ./config
--builddir path        : Path to the build directory, default: ./build
--sourcedir path       : Path to the source directory, default: ./source
--tmppath path         : Path to the temp directory, default: ./tmp
--macros file[,[file]  : Macro format files to load after the defaults
--log file             : Log file where all build out is written too
--url url[,url]        : URL to look for source
--no-download          : Disable the source downloader
--targetcflags flags   : List of C flags for the target code
--targetcxxflags flags : List of C++ flags for the target code
--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
--with-<label>         : Add the --with-<label> to the build
--without-<label>      : Add the --without-<label> to the build
--regression           : Set --no-install, --keep-going and --always-clean
-------------------------------------------------------------

Set Builder (sb-set-builder)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This command builds a set.

-------------------------------------------------------------
$ ../source-builder/sb-set-builder --help
RTEMS Source Builder, an RTEMS Tools Project (c) 2012-2013 Chris Johns
Options and arguments:
--force                : Force the build to proceed
--quiet                : Quiet output (not used)
--trace                : Trace the execution
--dry-run              : Do everything but actually run the build
--warn-all             : Generate warnings
--no-clean             : Do not clean up the build tree
--always-clean         : Always clean the build tree, even with an error
--jobs                 : Run with specified number of jobs, default: num CPUs.
--host                 : Set the host triplet
--build                : Set the build triplet
--target               : Set the target triplet
--prefix path          : Tools build prefix, ie where they are installed
--topdir path          : Top of the build tree, default is $PWD
--configdir path       : Path to the configuration directory, default: ./config
--builddir path        : Path to the build directory, default: ./build
--sourcedir path       : Path to the source directory, default: ./source
--tmppath path         : Path to the temp directory, default: ./tmp
--macros file[,[file]  : Macro format files to load after the defaults
--log file             : Log file where all build out is written too
--url url[,url]        : URL to look for source
--no-download          : Disable the source downloader
--targetcflags flags   : List of C flags for the target code
--targetcxxflags flags : List of C++ flags for the target code
--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
--with-<label>         : Add the --with-<label> to the build
--without-<label>      : Add the --without-<label> to the build
--bset-tar-file        : Create a build set tar file
--no-report            : Do not create a package report.
--list-configs         : List available configurations
--report-format        : The report format (text, html, asciidoc).
--list-bsets           : List available build sets
--no-install           : Do not install the packages to the prefix.
--list-deps            : List the dependent files.
--pkg-tar-files        : Create package tar files
--regression           : Set --no-install, --keep-going and --always-clean
-------------------------------------------------------------

.Arguments
The +[args]+ are a list build sets to build.

.Options
+--force+;;
Force the build to proceed even if the host check fails. Typically this happens
if executable files are found in the path at a different location to the host
defaults.
+--trace+;;
Trace enable printing of debug information to stdout. It is really only of use
to RTEMS Source Builder's developers.
+--dry-run+;;
Do everything but actually run the build commands. This is useful when checking
a new configuration parses cleanly.
+--warn-all+;;
Generate warnings.
+--no-clean+;;
Do not clean up the build tree during the cleaning phase of the build. This
leaves the source and the build output on disk so you can make changes, or
amend or generate new patches. It also allows you to review configure type
output such as +config.log+.
+--always-clean+;;
Clean away the results of a build even if the build fails. This is normally
used with `--keep-going` when regression testing to see which build sets
fail to build. It keeps the disk usage down.
+--jobs+;;
Control the number of jobs make is given. The jobs can be 'none' for only 1
job, 'half' so the number of jobs is half the number of detected cores, a
fraction such as '0.25' so the number of jobs is a quarter of the number of
detected cores and a number such as '25' which forces the number of jobs to
that number.
+--host+;;
Set the host triplet value. Be careful with this option.
+--build+;;
Set the build triplet. Be careful with this option.
+--target+;;
Set the target triplet. Be careful with this option. This is useful if you have
a generic configuration script that can work for a range of architectures.
+--prefix path+;;
Tools build prefix, ie where they are installed.
+--topdir path+;;
Top of the build tree, that is the current directory you are in.
+--configdir path+;;
Path to the configuration directory. This overrides the built in defaults.
+--builddir path+;;
Path to the build directory. This overrides the default of +build+.
+--sourcedir path+;;
Path to the source directory. This overrides the default of +source+.
+--tmppath path+;;
Path to the temporary directory. This overrides the default of +tmp+.
+--macros files+;;
Macro files to load. The configuration directory path is searched.
+--log file+;;
Log all the output from the build process. The output is directed to +stdout+
if no log file is provided.
+--url url+;;
URL to look for source when downloading. This is can be comma separate list.
+--no-download+;;
Disable downloading of source and patches. If the source is not found an error
is raised.
+--targetcflags flags+;;
List of C flags for the target code. This allows for specific local
customisation when testing new variations.
+--targetcxxflags flags+;;
List of C++ flags for the target code. This allows for specific local
customisation when testing new variations.
+--libstdcxxflags flags+;;
List of C\++ flags to build the target libstdc++ code. This allows for specific
local customisation when testing new variations.
+--with-<label>+;;
Add the --with-<label> to the build. This can be tested for in a script with
the +%bconf_with+ macro.
+--without-<label>+;;
Add the --without-<label> to the build. This can be tested for in a script with
the +%bconf_without+ macro.
+--list-bsets+;;
List available build sets.
+--list-configs+;;
List available configurations.
+--keep-going+;;
Do not stop on error. This is useful if your build sets performs a large number
of testing related builds and there are errors.
+--no-install+;;
Do not install the packages to the prefix. Use this if you are only after the
tar files.
+--bset-tar-file+;;
Create a build set tar file. This is a single tar file of all the packages in
the build set.
+--pkg-tar-files+;;
Create package tar files. A tar file will be created for each package built in
a build set.
+--list-deps+;;
Print a list of dependent files used by a build set. Dependent files have a
'dep[?]' prefix where '?' is a number. The files are listed alphabetically.
+--report-format format+;;
The report format can be 'text' or 'html'. The default is 'html'.
+--regression+;;
A convenience option which is the same as +--no-install+, +--keep-going+ and
+--always-clean+.

Set Builder (sb-builder)
~~~~~~~~~~~~~~~~~~~~~~~~

This command builds a configuration as described in a configuration
file. Configuration files have the extension of +.cfg+.

-------------------------------------------------------------
$ ./source-builder/sb-builder --help
sb-builder: [options] [args]
RTEMS Source Builder, an RTEMS Tools Project (c) 2012 Chris Johns
Options and arguments:
--force                : Force the build to proceed
--quiet                : Quiet output (not used)
--trace                : Trace the execution
--dry-run              : Do everything but actually run the build
--warn-all             : Generate warnings
--no-clean             : Do not clean up the build tree
--always-clean         : Always clean the build tree, even with an error
--jobs                 : Run with specified number of jobs, default: num CPUs.
--host                 : Set the host triplet
--build                : Set the build triplet
--target               : Set the target triplet
--prefix path          : Tools build prefix, ie where they are installed
--topdir path          : Top of the build tree, default is $PWD
--configdir path       : Path to the configuration directory, default: ./config
--builddir path        : Path to the build directory, default: ./build
--sourcedir path       : Path to the source directory, default: ./source
--tmppath path         : Path to the temp directory, default: ./tmp
--macros file[,[file]  : Macro format files to load after the defaults
--log file             : Log file where all build out is written too
--url url[,url]        : URL to look for source
--targetcflags flags   : List of C flags for the target code
--targetcxxflags flags : List of C++ flags for the target code
--libstdcxxflags flags : List of C++ flags to build the target libstdc++ code
--with-<label>         : Add the --with-<label> to the build
--without-<label>      : Add the --without-<label> to the build
--list-configs         : List available configurations
-------------------------------------------------------------

Host Setups
-----------

The host versions are listed. If a later version of the host operating system
exists it should unless listed.

Linux
~~~~~

A number of different Linux distrubutions are known to work. The following have
been tested and report as working.

Archlinux
^^^^^^^^^

The following packages are required on a fresh Archlinux 64bit installation:

--------------------------------------------------------------
# pacman -S base-devel gdb xz unzip ncurses git zlib
--------------------------------------------------------------

Archlinux, by default installs `texinfo-5` which is incompatible for building
GCC 4.7 tree. You will have to obtain `texinfo-legacy` from `AUR` and provide
a manual override.

--------------------------------------------------------------
# pacman -R texinfo
$ yaourt -S texinfo-legacy
# ln -s /usr/bin/makeinfo-4.13a /usr/bin/makeinfo
--------------------------------------------------------------

CentOS
^^^^^^

The following packages are required on a minimal CentOS 6.3 64bit installation:

-------------------------------------------------------------
# yum install autoconf automake binutils gcc gcc-c++ gdb make patch \
bison flex xz unzip ncurses-devel texinfo zlib-devel python-devel git
-------------------------------------------------------------

The minimal CentOS distribution is a specific DVD that installs a minimal
system. If you use a full system some of these packages may have been
installed.

Fedora
^^^^^^

The RTEMS Source Builder has been tested on Fedora 18 64bit.

Raspbian
^^^^^^^^

The is the Debian distribution for the Raspberry Pi. The following packages are
required.

-------------------------------------------------------------
$ sudo apt-get install autoconf automake bison flex binutils gcc g++ gdb \
texinfo unzip ncurses-dev python-dev git
-------------------------------------------------------------

It is recommended you get Model B of the Pi with 512M of memory and to mount a
remote disk over the network. The tools can be build with a prefix under your
home directory as recommended and end up on the SD card.

Ubuntu
^^^^^^

The latest testing was with Ubuntu 12.10 64bit. A minimal installation was used
and the following packages installed.

-------------------------------------------------------------
$ sudo apt-get build-dep binutils gcc g++ gdb unzip git
-------------------------------------------------------------

FreeBSD
~~~~~~~

The RTEMS Source Builder has been tested on FreeBSD 9.1 64bit.

MacOS X
~~~~~~~

The RTEMS Source Builder has been tested on Mountain Lion. You will need to
install the Xcode app using the _App Store_ tool, run Xcode and install the
Developers Tools package within Xcode.

Windows
~~~~~~~

Windows tool sets are supported creating native Windows executable. Native
Windows tools are built using a MinGW compiler and do not need any extra
libraries or emulation layer to run once built. The tools understand and use
standard Windows paths and integrate easly into Windows IDE environments. A
shell maybe needed to build other parts of your system however if your
development tools are all native Windows tool you can easly integrate these
tool sets.

Building on Windows is a little more complicated because the Cygwin shell is
used rather than the MinGW MSYS shell. The MSYS shell is simpler because the
detected host triple is MinGW so the build is standard cross-compiler build.
The age of the MSYS code base, its stability and ability to to complete a build
with limitations such as the length of file names support make using MSYS
difficult therefore the more complex path of a Canadian cross-build using
Cygwin is supported.

Install a recent Cygwin version using the Cygwin setup tool. Select and install
the groups and packages listed:

.Cygwin Packages
[options="header,compact",width="50%",cols="20%,80%"]
|================================
|Group   |Package
|Archive |bsdtar
|        |unzip
|        |xz
|Devel   |autoconf
|        |autoconf2.1
|        |autoconf2.5
|        |automake
|        |binutils
|        |bison
|        |flex
|        |gcc4-core
|        |gcc4-g++
|        |git
|        |make
|        |mingw64-x86_64-binutils
|        |mingw64-x86_64-gcc-core
|        |mingw64-x86_64-g++
|        |mingw64-x86_64-runtime
|        |mingw64-x86_64-zlib
|        |zlib-devel
|MinGW   |mingw-zlib-devel
|Python  |python
|================================

The setup tool will add a number of dependent package and it is ok to accept
them.

I have found turning off Windows Defender improves performance if you have
another up to date virus detection tool installed and enabled. I used the
excellent `Process Hacker 2` tool to monitor the performance and I found the
Windows Defender service contributed a high load. In my case I had a 3rd party
virus tool installed so the Windows Defender service was not needed.

A Canadian cross-compile (Cxc) is required on Cygwin because the host is Cygwin
therefore a traditional cross-compile will result in Cygiwn binaries. With a
Canadian cross-compile a Cygwin cross-compiler is built as well as the MinGW
RTEMS cross-compiler. The Cygwin cross-compiler is required to build the C
runtime for the RTEMS target because we are building under Cygiwn. The build
output for an RTEMS 4.10 ARM tool set is:

-------------------------------------------------------------
chris@cygthing ~/development/rtems/src/rtems-source-builder/rtems
$ ../source-builder/sb-set-builder --log=l-arm.txt --prefix=$HOME/development/rtems/4.10 4.10/rtems-arm
RTEMS Source Builder - Set Builder, v0.2
Build Set: 4.10/rtems-arm
config: expat-2.1.0-1.cfg
package: expat-2.1.0-x86_64-w64-mingw32-1
building: expat-2.1.0-x86_64-w64-mingw32-1
reporting: expat-2.1.0-1.cfg -> expat-2.1.0-x86_64-w64-mingw32-1.html
config: tools/rtems-binutils-2.20.1-1.cfg
package: arm-rtems4.10-binutils-2.20.1-1 <1>
building: arm-rtems4.10-binutils-2.20.1-1
package: (Cxc) arm-rtems4.10-binutils-2.20.1-1 <2>
building: (Cxc) arm-rtems4.10-binutils-2.20.1-1
reporting: tools/rtems-binutils-2.20.1-1.cfg ->
arm-rtems4.10-binutils-2.20.1-1.html
config: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg
package: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
building: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
package: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
building: (Cxc) arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
reporting: tools/rtems-gcc-4.4.7-newlib-1.18.0-1.cfg ->
arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1.html
config: tools/rtems-gdb-7.3.1-1.cfg
package: arm-rtems4.10-gdb-7.3.1-1
building: arm-rtems4.10-gdb-7.3.1-1
reporting: tools/rtems-gdb-7.3.1-1.cfg -> arm-rtems4.10-gdb-7.3.1-1.html
config: tools/rtems-kernel-4.10.2.cfg
package: arm-rtems4.10-kernel-4.10.2-1
building: arm-rtems4.10-kernel-4.10.2-1
reporting: tools/rtems-kernel-4.10.2.cfg -> arm-rtems4.10-kernel-4.10.2-1.html
installing: expat-2.1.0-x86_64-w64-mingw32-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
installing: arm-rtems4.10-binutils-2.20.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10 <3>
installing: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
installing: arm-rtems4.10-gdb-7.3.1-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
installing: arm-rtems4.10-kernel-4.10.2-1 -> /cygdrive/c/Users/chris/development/rtems/4.10
cleaning: expat-2.1.0-x86_64-w64-mingw32-1
cleaning: arm-rtems4.10-binutils-2.20.1-1
cleaning: arm-rtems4.10-gcc-4.4.7-newlib-1.18.0-1
cleaning: arm-rtems4.10-gdb-7.3.1-1
cleaning: arm-rtems4.10-kernel-4.10.2-1
Build Set: Time 10:09:42.810547 <4>
-------------------------------------------------------------

<1> The Cygwin version of the ARM cross-binutils.
<2> The +(Cxc)+ indicates this is the MinGW build of the package.
<3> Only the MinGW version is installed.
<4> Cygwin is slow so please be patient. This time was on an AMD Athlon 64bit
    Dual Core 6000+ running at 3GHz with 4G RAM running Windows 7 64bit.

CAUTION: Cygwin documents the 'Big List Of Dodgy Apps' or 'BLODA'. The link is
http://cygwin.com/faq/faq.using.html#faq.using.bloda and it is worth a
look. You will see a large number of common pieces of software found on Windows
systems that can cause problems. My testing has been performed with NOD32
running and I have seen some failures. The list is for all of Cygwin so I am
not sure which of the listed programs effect the RTEMS Source Biulder. The
following FAQ item talks about +fork+ failures and presents some technical
reasons they cannot be avoided in all cases. Cygwin and it's fork MSYS are
fantastic pieces of software in a difficult environment. I have found building
a single tool tends to work, building all at once is harder.



Build Status By Host
~~~~~~~~~~~~~~~~~~~~

This table lists the current build and testing status for reported hosts:

[grid="rows",format="csv"]
[options="header",cols="<,<,<,<,<,<"]
|===========================
OS,Uname,4.9,4.10,4.11,Comments
include::host-results.csv[]
|===========================

[NOTE]
==================================================================
Report any unlisted hosts as a patch.
==================================================================

History
-------

The RTEMS Source Builder is a stand alone tool based on another tool called the
'SpecBuilder'. The SpecBuilder was written for the RTEMS project to give me a
way to build tools on hosts that did not support RPMs. At the time the RTEMS
tools maintainer only used spec files to create various packages. This meant I
had either spec files, RPM files or SRPM files. The RPM and SPRM files where
useless because you needed an 'rpm' type tool to extract and manage them. There
are versions of 'rpm' for a number of non-RPM hosts however these proved to be
in various broken states and randomly maintained. The solution I settled on was
to use spec files so I wrote a Python based tool that parsed the spec file
format and allowed me to create a shell script I could run to build the
package. This approach proved successful and I was able to track the RPM
version of the RTEMS tools on a non-RPM host over a number of years. however
the SpecBuilder tool did not help me build tools or other packages not related
to the RTEMS project where there was no spec file I could use so I needed
another tool. Rather than start again I decided to take the parsing code for
the spec file format and build a new tool called the RTEMS Source Builder.