[d7a023a] | 1 | Source Builder |
---|
| 2 | ============== |
---|
| 3 | Chris Johns <chrisj@rtems.org> |
---|
| 4 | 1.0, November 2012 |
---|
| 5 | :doctype: book |
---|
| 6 | :toc: |
---|
| 7 | :icons: |
---|
| 8 | :numbered: |
---|
| 9 | |
---|
| 10 | image:images/rtemswhitebg.jpg["RTEMS",width="20%"] |
---|
| 11 | |
---|
| 12 | Introduction |
---|
| 13 | ------------ |
---|
| 14 | |
---|
| 15 | The Source Builder is a tool to aid building packages from source. It is not a |
---|
| 16 | package manager. It is just helps consolidate the details that you need to know |
---|
| 17 | to build a package from source. The tool is mainly aimed at those users who |
---|
| 18 | need to maintain tool sets for embedded type development, that is |
---|
| 19 | cross-compiled compiled tool chains, debuggers, and debugging aids. It is not |
---|
| 20 | limited to this role but designed to fit with-in that specific niche. |
---|
| 21 | |
---|
| 22 | The Source Builder attempts to support any host environment that runs Python |
---|
| 23 | and you can build the package on. It is not some sort of magic that can take |
---|
| 24 | any piece of source code and make it build. Someone at some point in time has |
---|
| 25 | figured out how to build that package from source and taught this tool. |
---|
| 26 | |
---|
| 27 | The Source Builder has two types configuration data. The first is configuration |
---|
| 28 | files and these are scripts based on the RPM spec file format that detail the |
---|
| 29 | steps needed to build a package. The steps are 'preparation', 'building', and |
---|
| 30 | 'installing'. The second set of configuration files are 'build sets'. A build |
---|
| 31 | set describes a collection of packages you want built together. For example the |
---|
| 32 | GNU tool set is autoconf, automake, binutils, gcc, and gdb. This is the typical |
---|
| 33 | suite of tools you need for an embedded cross-development type project. |
---|
| 34 | |
---|
| 35 | The Source Builder does not interact with any host package management |
---|
| 36 | system. There is no automatic dependence checking between various packages you |
---|
| 37 | build or your host system may have installed. We assume you know what are doing |
---|
| 38 | or the build sets and configuration files you are using have been created by |
---|
| 39 | developers who do. A buld set should provide a known working configuration. |
---|
| 40 | |
---|
| 41 | Why build from source ? |
---|
| 42 | ~~~~~~~~~~~~~~~~~~~~~~~ |
---|
| 43 | |
---|
| 44 | If you are developing a system or product that has a long shelf life or is used |
---|
| 45 | in a critical piece of infastructure that has a long life cycle being able to |
---|
| 46 | build from source is important. It insulates the project from the fast ever |
---|
| 47 | changing world of the host development machines. If your tool set is binary and |
---|
| 48 | you have lost the ability to build it you have lost a degree of control and |
---|
| 49 | flexibility open source gives you. Fast moving host environments are |
---|
| 50 | fantastic. We have powerful multi-core computers with huge amounts of memory |
---|
| 51 | and state of the art operating systems to running on them. The product or |
---|
| 52 | project you are part of may need to be maintained well past the life time of |
---|
| 53 | these host. Being able to build from source an important and critical part of |
---|
| 54 | this process because you can move to a newer host and create an equivalent tool |
---|
| 55 | set. |
---|
| 56 | |
---|
| 57 | Building from source provides you with control over the configuration of the |
---|
| 58 | package you are building. If all or the most important dependent parts are |
---|
| 59 | built from source you limit the exposure to host variations. For example the |
---|
| 60 | GNU C compiler (gcc) currently uses a number of 3rd party libraries internally |
---|
| 61 | (gmp, mpfr, etc). If your validated compiler generating code for your target |
---|
| 62 | processor is dynamically linked against the host's version of these libraries |
---|
| 63 | any change in the host's configuration may effect you. The changes the host's |
---|
| 64 | package management system makes may be perfectly reasonible in relation to the |
---|
| 65 | distribution being managed how-ever this may not extend to you and your |
---|
| 66 | tools. Building your tools from source and controlling the specific version of |
---|
| 67 | these dependent parts means you are not exposing yourself to unexpected and |
---|
| 68 | often difficult to resolve problems. On the other side you need to make sure |
---|
| 69 | your tools build and work with newer versions of the host operating |
---|
| 70 | sytem. Given the stability of standards based libraries like 'libc' and ever |
---|
| 71 | improving support for standard header file locations this task is becoming |
---|
| 72 | easier. |
---|
| 73 | |
---|
| 74 | History |
---|
| 75 | ~~~~~~~ |
---|
| 76 | |
---|
| 77 | The Source Builder is a stand alone tool based on another tool called the |
---|
| 78 | 'SpecBuilder'. The SpecBuilder was written for the RTEMS project too give me a |
---|
| 79 | way to build tools on hosts that did not support RPMs. At the time the RTEMS |
---|
| 80 | tools maintainer only used spec files to create various packages. This meant I |
---|
| 81 | had either spec files, RPM files or SRPM files. The RPM and SPRM files where |
---|
| 82 | useless because you needed an 'rpm' type tool to extract and manage them. There |
---|
| 83 | are versions of 'rpm' for a number of non-RPM hosts how-ever these proved to be |
---|
| 84 | in various broken states and randomally maintained. The solution I settled on |
---|
| 85 | was to use spec files so I wrote a Python based tool that parsed the spec file |
---|
| 86 | format and allowed me to create a shell script I could run to build the |
---|
| 87 | package. This approach proved successful and I was able to track the RPM |
---|
| 88 | version of the RTEMS tools on a non-RPM host over a number of years. How-ever |
---|
| 89 | the SpecBuilder tool did not help me build tools or other packages not related |
---|
| 90 | to the RTEMS project where there was no spec file I could use so I needed |
---|
| 91 | another tool. Rather than start again I decided to take the parsing code for |
---|
| 92 | the spec file format and build a new tool called the Source Builder. |
---|
| 93 | |
---|
| 94 | Quick Start |
---|
| 95 | ----------- |
---|
| 96 | |
---|
| 97 | Check out the Source Builder tool from git: |
---|
| 98 | |
---|
| 99 | ------------------------------------------------------------- |
---|
[ccb4cb1] | 100 | $ git clone git://git.rtems.org/chrisj/rtems-source-builder.git |
---|
[d7a023a] | 101 | ------------------------------------------------------------- |
---|
| 102 | |
---|
| 103 | The first step is to check if your host is set up correctly: |
---|
| 104 | |
---|
| 105 | ------------------------------------------------------------- |
---|
[ccb4cb1] | 106 | $ rtems-source-builder/source-builder/sb-check |
---|
[d7a023a] | 107 | warning: exe: absolute exe found in path: (__objcopy) /usr/local/bin/objcopy <1> |
---|
| 108 | warning: exe: absolute exe found in path: (__objdump) /usr/local/bin/objdump |
---|
| 109 | error: exe: not found: (_xz) /usr/local/bin/xz <2> |
---|
| 110 | Source Builder environent is not correctly set up |
---|
| 111 | $ source-builder/sb-check |
---|
| 112 | Source Builder environent is ok <3> |
---|
| 113 | ------------------------------------------------------------- |
---|
| 114 | |
---|
| 115 | <1> A tool is in the environment path but does not match the shown path. |
---|
| 116 | <2> The executable 'xz' is not found. |
---|
| 117 | <3> The host's environment is set up correct. |
---|
| 118 | |
---|
| 119 | If there are problems you are given a list of executables that cannot be |
---|
| 120 | found. You may also be given a list of warnings about executables not in the |
---|
| 121 | expected location how-ever the executable was located somewhere in your |
---|
| 122 | environment's path. You will need to check the specific host section to resolve |
---|
| 123 | these issues. |
---|
| 124 | |
---|
| 125 | Create a suitable build directory away from the Source Builder source change |
---|
| 126 | into that directory and build a GNU tool set: |
---|
| 127 | |
---|
| 128 | ------------------------------------------------------------- |
---|
| 129 | $ mkdir gnu-tools <1> |
---|
| 130 | $ cd gnu-tools |
---|
[ccb4cb1] | 131 | $ ../rtems-source-builder/source-builder/sb-set-builder <2> --log=l.txt <3> --force <4> \ |
---|
[d7a023a] | 132 | --prefix=$HOME/gnu-tools-1 <5> --target=arm-eabi <6> gnu-toolset-4.6 <7> |
---|
| 133 | ------------------------------------------------------------- |
---|
| 134 | |
---|
| 135 | <1> Make a build directory you can delete when finished. |
---|
| 136 | <2> The Source Builder command to build a set of tools. |
---|
| 137 | <3> Capture the output to a log file. |
---|
| 138 | <4> The force option will create any needed directories and allow the build to |
---|
| 139 | proceed if your host is not set up. |
---|
| 140 | <5> Give the tools a suitable prefix. This is the location you install the |
---|
| 141 | tools into once they have built. |
---|
| 142 | <6> The gnu-toolset requires you set a target. In this case the tool set will |
---|
[689f6d0] | 143 | be a generic unpatched version of GCC 4.6 for a bare metal the ARM processor. |
---|
[d7a023a] | 144 | <7> The build set. |
---|
| 145 | |
---|
[689f6d0] | 146 | To view the build sets lets change to the RTEMS project's source builder |
---|
| 147 | configuration and then list the build sets: |
---|
| 148 | |
---|
| 149 | ------------------------------------------------------------- |
---|
[ccb4cb1] | 150 | $ cd ../rtems-source-builder/rtems |
---|
[689f6d0] | 151 | $ ../source-builder/sb-set-builder --list-bsets |
---|
| 152 | Source Builder - Set Builder, v0.1 |
---|
| 153 | Examining: /usr/home/chris/development/rtems/src/rtems-source-builder/config <1> |
---|
| 154 | Examining: /usr/home/chris/development/rtems/src/source-builder/config <2> |
---|
| 155 | gnu-tools-4.6 <3> |
---|
| 156 | rtems-tools-4.10 <4> |
---|
| 157 | ------------------------------------------------------------- |
---|
| 158 | |
---|
| 159 | <1> The local RTEMS configuration directory. Searched first. |
---|
| 160 | <2> The Source Builder configuration directory. |
---|
| 161 | <3> The Source Builder provided GNU tools GCC 4.6 build set. |
---|
| 162 | <4> The RTEMS Source Builder provided RTEMS 4.10 build set. |
---|
| 163 | |
---|
| 164 | And to view the configurations you can: |
---|
| 165 | |
---|
| 166 | ------------------------------------------------------------- |
---|
| 167 | $ ../source-builder/sb-set-builder --list-configs |
---|
| 168 | Source Builder - Set Builder, v0.1 |
---|
| 169 | Examining: /usr/home/chris/development/rtems/src/rtems-source-builder/config |
---|
| 170 | Examining: /usr/home/chris/development/rtems/src/source-builder/config |
---|
| 171 | autoconf-2-1 <1> |
---|
| 172 | autoconf-2.68-1 |
---|
| 173 | autoconf-2.69-1 |
---|
| 174 | autoconf-internal-2.68-1 |
---|
| 175 | automake-1-1 |
---|
| 176 | automake-1.12-1 |
---|
| 177 | automake-internal-1.12-1 |
---|
| 178 | base |
---|
| 179 | binutils-2-1 |
---|
| 180 | binutils-2.22-1 |
---|
| 181 | checks |
---|
| 182 | expat-2-1 |
---|
| 183 | expat-2.1.0-1 |
---|
| 184 | gcc-4.4-1 |
---|
| 185 | gcc-4.6-1 |
---|
| 186 | gcc-4.6-newlib-1.20-1 |
---|
| 187 | gdb-7-1 |
---|
| 188 | gdb-7.5-1 |
---|
| 189 | libusb-1-1 |
---|
| 190 | libusb-1.0.9-1 |
---|
| 191 | m4-1-1 |
---|
| 192 | m4-1.4.16-1 |
---|
| 193 | texane-stlink-1 |
---|
| 194 | rtems-binutils-2.20.1-1 |
---|
| 195 | rtems-gcc-4.4.7-newlib-1.18.0-1 |
---|
| 196 | rtems-gdb-7.3.1-1 |
---|
| 197 | ------------------------------------------------------------- |
---|
| 198 | |
---|
| 199 | <1> Configurations are built by using the builder. This creates a stand alone |
---|
| 200 | package. |
---|
| 201 | |
---|
| 202 | The Source Builder |
---|
| 203 | ------------------ |
---|
| 204 | |
---|
| 205 | The Source Builder provides a few generic build sets and the configuration |
---|
| 206 | support to build a number of packages. A project that uses the Source Builder |
---|
| 207 | can create a specialised set of configuration files that provides the specific |
---|
| 208 | configurations thet project uses. |
---|
| 209 | |
---|
| 210 | For example the RTEMS project provides its own set of configuration files. In |
---|
| 211 | the build set list in the 'Quick Start' section you can see a build set |
---|
| 212 | +rtems-tools-4.10+. This build set defines the extact configration to use for |
---|
| 213 | the RTEMS 4.10 release. |
---|
| 214 | |
---|