Changeset 68ddf13 in rtems-source-builder


Ignore:
Timestamp:
Nov 10, 2013, 1:30:49 AM (7 years ago)
Author:
Chris Johns <chrisj@…>
Branches:
4.10, 4.11, 4.9, 5, master
Children:
5b00910
Parents:
46dcbfc
Message:

New Configurations documentation.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • doc/source-builder.txt

    r46dcbfc r68ddf13  
    614614should be just above it.
    615615
     616[[_contributing]]
     617Contributing
     618------------
     619
     620The RSB is open source and open to contributions. These can be bug fixes, new
     621features or new configurations. Please break patches down into changes to the
     622core Python code, configuration changes or new configurations.
     623
     624Pleease email me patches via git so I can maintain your commit messages so you
     625are acknowledged as the contributor.
     626
    616627Project Sets
    617628------------
     
    11891200revision number will not be effected by the change. This locks down a specific
    11901201configuration over time.
     1202
     1203Personal Configurations
     1204~~~~~~~~~~~~~~~~~~~~~~~
     1205
     1206The RSB supports personal configurations. You can view the RTEMS support in the
     1207+rtems+ directory as a private configuration tree that resides within the RSB
     1208source. There is also the +bare+ set of configurations. You can create your own
     1209configurations away from the RSB source tree yet use all that the RSB provides.
     1210
     1211To create a private configuration change to a suitable directory:
     1212
     1213-------------------------------------------------------------
     1214$ cd ~/work
     1215$ mkdir test
     1216$ cd test
     1217$ mkdir config
     1218-------------------------------------------------------------
     1219
     1220and create a +config+ directory. Here you can add a new configuration or build
     1221set file. The section 'Adding New Configurations' details how to add a new
     1222confguration.
     1223
     1224New Configurations
     1225~~~~~~~~~~~~~~~~~~
     1226
     1227This section describes how to add a new configuration to the RSB. We will add a
     1228configuration to build the Device Tree Compiler. The Device Tree Compiler or
     1229DTC is part of the Flattened Device Tree project and compiles Device Tree
     1230Source (DTS) files into Device Tree Blobs (DTB). DTB files can be loaded by
     1231operating systems and used to locate the various resources such as base
     1232addresses of devices or interrupt numbers allocated to devices. The Device Tree
     1233Compiler source code can be downloaded from http://www.jdl.com/software. The
     1234DTC is supported in the RSB and you can find the configuration files under the
     1235+bare/config+ tree. I suggest you have a brief look over these files.
     1236
     1237Layering by Including
     1238^^^^^^^^^^^^^^^^^^^^^
     1239
     1240Configurations can be layered using the +%include+ directive. The user invokes
     1241the outter layers which include inner layers until all the required
     1242configuration is present and the package can be built. The outter layers can
     1243provide high level details such as the version and the release and the inner
     1244layers provide generic configuration details that do not change from one
     1245release to another. Macro variables are used to provide the specific
     1246configuration details.
     1247
     1248Configuration File Numbering
     1249^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     1250
     1251Configuration files have a number at the end. This is a release number for that
     1252configuraiton and it gives us the ability to track a specific configuration for
     1253a specific version. For example lets say the developers of the DTC package
     1254change the build system from a single makefile to autoconf and automake between
     1255version 1.3.0 and version 1.4.0. The configuration file used to build the
     1256package would change have to change. If we did not number the configuration
     1257files the ability to build 1.1.0, 1.2.0 or 1.3.0 would be lost if we update a
     1258common configuration file to build an autoconf and automake version. For
     1259version 1.2.0 the same build script can be used so we can share the same
     1260configuration file between version 1.1.0 and version 1.2.0. An update to any
     1261previous release lets us still build the package.
     1262
     1263Common Configuration Scripts
     1264^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     1265
     1266Common configuration scripts that are independent of version, platform and
     1267architecture are useful to everyone. These live in the Source Builder's
     1268configuration directory. Currrently there are scripts to build binutils, expat,
     1269DTC, GCC, GDB and libusb. These files contain the recipes to build these
     1270package without the specific details of the versions or patches being
     1271built. They expect to be wrapped by a confguration file that ties the package
     1272to a specific version and optionally specific patches.
     1273
     1274DTC Example
     1275^^^^^^^^^^^
     1276
     1277We will be building the DTC for your host rather than a package for RTEMS. We
     1278will create a file called +source-builder/config/dtc-1-1.cfg+. This is a common
     1279script that can be used to build a specific version using a general recipe. The
     1280file name is 'dtc-1-1.cfg' where the 'cfg' extension indicates this is a
     1281configuration file. The first *1* says this is for the major release 1 of the
     1282package and the last *1* is the build configuration version.
     1283
     1284The file starts with some comments that detail the configuration. If there is
     1285anything unsual about the configuration it is a good idea to add something in
     1286the comments here. The comments are followed by a check for the release. In
     1287this case if a release is not provided a default of 1 is used.
     1288
     1289-------------------------------------------------------------
     1290#
     1291# DTC 1.x.x Version 1.
     1292#
     1293# This configuration file configure's, make's and install's DTC.
     1294#
     1295
     1296%if %{release} == %{nil}
     1297%define release 1
     1298%endif
     1299-------------------------------------------------------------
     1300
     1301The next section defines some information about the package. It does not effect
     1302the build and is used to annotate the reports. It is recommended this
     1303information is kept updated and accurate.
     1304
     1305-------------------------------------------------------------
     1306Name:      dtc-%{dtc_version}-%{_host}-%{release}
     1307Summary:   Device Tree Compiler v%{dtc_version} for target %{_target} on host %{_host}
     1308Version:   %{dtc_version}
     1309Release:   %{release}
     1310URL:       http://www.jdl.com/software/
     1311BuildRoot: %{_tmppath}/%{name}-root-%(%{__id_u} -n)
     1312-------------------------------------------------------------
     1313
     1314The next section defines the source and any patches. In this case there is a
     1315single source package and it can be downloaded using the HTTP protocol. The RSB
     1316knows this is GZip'ped tar file. If more than one package package is needed add
     1317them increasing the index. The +gcc-4.8-1.cfg+ configuration contains examples
     1318of more than one source package as well as conditionally including source
     1319packages based on the outter configuration options.
     1320
     1321-------------------------------------------------------------
     1322#
     1323# Source
     1324#
     1325Source0: http://www.jdl.com/software/dtc-v%{dtc_version}.tgz
     1326-------------------------------------------------------------
     1327
     1328The remainder of the script is broken in to the various phases of a build. They
     1329are:
     1330
     1331. Preperation
     1332. Bulding
     1333. Installing, and
     1334. Cleaning
     1335
     1336Preperation is the unpacking of the source, applying any patches as well as any
     1337package specific set ups. This part of the script is a standard Unix shell
     1338script. Becareful with the the use of '%' and '$'. The RSB uses '%' while the
     1339shell scripts use '$'.
     1340
     1341A standard pattern you will observe is the saving of the build's top
     1342directory. This is used instead of changing into a subdirectory and then
     1343changing to the parent when finished. Some hosts will change in a subdirectory
     1344that is a link however changing to the parent does not change back to the
     1345parent of the link rather it changes to the parent of the target of the link
     1346and that is something the RSB nor you can track easily. The RSB configuration
     1347script's are a collection of various subtle issues so please ask if you are
     1348unsure why something is being done a particular way.
     1349
     1350The preparation phase will often include a number of conditional patch
     1351commands. Outter layers can provide patches as needed while being able to use a
     1352common recipe. Users can override the standard build and supply a custom patch
     1353for testing using the user macro command line interface.
     1354
     1355-------------------------------------------------------------
     1356#
     1357# Prepare the source code.
     1358#
     1359%prep
     1360  build_top=$(pwd)
     1361
     1362  %setup -q -n dtc-v%{dtc_version}
     1363  %{?patch0:%patch0 -p1}
     1364  %{?patch1:%patch1 -p1}
     1365  %{?patch2:%patch2 -p1}
     1366  %{?patch3:%patch3 -p1}
     1367  %{?patch4:%patch4 -p1}
     1368  %{?patch5:%patch5 -p1}
     1369  %{?patch6:%patch6 -p1}
     1370  %{?patch7:%patch7 -p1}
     1371
     1372  cd ${build_top}
     1373-------------------------------------------------------------
     1374
     1375The configuration file 'gcc-common-1.cfg' is a complex example of source
     1376preparation. It contains a number of source packages and patches and it
     1377combines these into a single source tree for building. It uses links to map
     1378source into the GCC source tree so GCC can be built using the _single source
     1379tree_ method. It also shows how to fetch source code from version
     1380control. Newlib is taken directly from its CVS repository.
     1381
     1382Next is the building phase and for the DTC example this is simply a matter of
     1383running +make+. Note the use of the RSB macros for commands. In the case of
     1384'%{\__make}' it maps to the correct make for your host. In the case of BSD
     1385systems we need to use the GNU make and not the GNU make.
     1386
     1387If your package requires a configuration stage you need to run this before the
     1388make stage. Again the GCC common confguration file provides a detailed example.
     1389
     1390-------------------------------------------------------------
     1391%build
     1392  build_top=$(pwd)
     1393
     1394  cd dtc-v%{dtc_version}
     1395
     1396  %{build_build_flags}
     1397
     1398  %{__make} PREFIX=%{_prefix}
     1399
     1400  cd ${build_top}
     1401-------------------------------------------------------------
     1402
     1403You can invoke make with the macro '%{?_smp_flags}' as a command line
     1404argument. This macro is controlled by the '--jobs' command line option and the
     1405host CPU detection support in the RSB. If you are on a multicore host you can
     1406increase the build speed using this macro. It also lets you disabled bulding on
     1407multicores to aid debugging when tetsing.
     1408
     1409Next is the install phase. This phase is a little more complex because you may
     1410be building a tar file and the end result of the build is never actually
     1411installed into the prefix on the build host and you may not even have
     1412permissions to perform a real install. Most packages install to the +prefix+
     1413and the prefix is typically supplied via the command to the RSB or the
     1414package's default is used. The default can vary depending on the host's
     1415operating system. To install to a path that is not the prefix the +DESTDIR+
     1416make variable is used. Most packages should honour the +DISTDIR+ make variables
     1417and you can typically specify it on the command line to make when invoking the
     1418install target. This results in the package being installed to a loction that
     1419is not the prefix but one you can control. The RSB provides a shell variable
     1420called +SB_BUILD_ROOT+ you can use. In a build set where you are building a
     1421number of packages you can collect all the built packages in a single tree that
     1422is captured in the tar file.
     1423
     1424Also note the use of the macro +%{\__rmdir}+. The use of these macros allow the
     1425RSB to vary specific commands based on the host. This can help on hosts like
     1426Windows where bugs can effect the standard commands such as 'rm'. There are
     1427many many macros to help you. You can find these listed in the +defaults.mc+
     1428file and in the trace output. If you are new to creating and editing
     1429configurations learning these can take a little time.
     1430
     1431-------------------------------------------------------------
     1432%install
     1433  build_top=$(pwd)
     1434
     1435  %{__rmdir} -rf $SB_BUILD_ROOT
     1436
     1437  cd dtc-v%{dtc_version}
     1438  %{__make} DESTDIR=$SB_BUILD_ROOT PREFIX=%{_prefix} install
     1439
     1440  cd ${build_top}
     1441-------------------------------------------------------------
     1442
     1443Finally there is an optional clean section. The RSB will run this section if
     1444+--no-clean+ has not been provided on the command line. The RSB does clean up
     1445for you.
     1446
     1447Once we have the configuration files we can execute the build using the
     1448`sb-builder` command. The command will perform the build and create a tar file
     1449in the +tar+ directory.
     1450
     1451-------------------------------------------------------------
     1452$  ../source-builder/sb-builder --prefix=/usr/local \
     1453     --log=log_dtc devel/dtc-1.2.0
     1454RTEMS Source Builder, Package Builder v0.2.0
     1455config: devel/dtc-1.2.0
     1456package: dtc-1.2.0-x86_64-freebsd9.1-1
     1457download: http://www.jdl.com/software/dtc-v1.2.0.tgz -> sources/dtc-v1.2.0.tgz
     1458building: dtc-1.2.0-x86_64-freebsd9.1-1
     1459$ ls tar
     1460dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2
     1461-------------------------------------------------------------
     1462
     1463If you want to have the package installed automatically you need to create a
     1464build set. A build set can build one or more packages from their configurations
     1465at once to create a single package. For example the GNU tools is typically seen
     1466as binutils, GCC and GDB and a build set will build each of these packages and
     1467create a single build set tar file or install the tools on the host into the
     1468prefix path.
     1469
     1470The DTC build set file is called +dtc.bset+ and contains:
     1471
     1472-------------------------------------------------------------
     1473#
     1474# Build the DTC.
     1475#
     1476
     1477%define release 1
     1478
     1479devel/dtc-1.2.0.cfg
     1480-------------------------------------------------------------
     1481
     1482To build this you can use something similar to:
     1483
     1484-------------------------------------------------------------
     1485$ ../source-builder/sb-set-builder --prefix=/usr/local --log=log_dtc \
     1486   --trace --bset-tar-file --no-install dtc
     1487RTEMS Source Builder - Set Builder, v0.2.0
     1488Build Set: dtc
     1489config: devel/dtc-1.2.0.cfg
     1490package: dtc-1.2.0-x86_64-freebsd9.1-1
     1491building: dtc-1.2.0-x86_64-freebsd9.1-1
     1492tarball: tar/x86_64-freebsd9.1-dtc-set.tar.bz2
     1493cleaning: dtc-1.2.0-x86_64-freebsd9.1-1
     1494Build Set: Time 0:00:02.865758
     1495$ ls tar
     1496dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2   x86_64-freebsd9.1-dtc-set.tar.bz2
     1497-------------------------------------------------------------
     1498
     1499The build is for a FreeBSD host and the prefix is for user installed
     1500packages. In this example I cannot let the source builder perform the install
     1501because I never run the RSB with root priviledges so a build set or bset tar
     1502file is created. This can then be installed using root privildges.
     1503
     1504The command also supplies the --trace option. The output in the log file will
     1505contian all the macros.
     1506
     1507Debugging
     1508^^^^^^^^^
     1509
     1510New configuration files require debugging. There are two types of
     1511debugging. The first is debugging RSB script bugs. The +--dry-run+ option is
     1512used here. Suppling this option will result in most of the RSB processing to be
     1513performed and suitable output placed in the log file. This with the +--trace+
     1514option should help you resolve any issues.
     1515
     1516The second type of bug to fix are related to the execution of one of
     1517phases. These are usually a mix of shell script bugs or package set up or
     1518configuration bugs. Here you can use any normal shell script type debug
     1519technique such as +set -x+ to output the commands or +echo+
     1520statements. Debugging package related issues may require you start a build with
     1521teh RSB and supply +--no-clean+ option and then locate the build directories
     1522and change directory into them and manually run commands until to figure what
     1523the package requires.
    11911524
    11921525Snapshot Testing
     
    18432176specific option on the command line with the +--without-<label>+ option. This
    18442177option is only available with the +sb-builder+ command.
    1845 
    18462178
    18472179Commands
Note: See TracChangeset for help on using the changeset viewer.