source: rtems-libbsd/CONTRIBUTING.md @ 4c086a2

5-freebsd-12
Last change on this file since 4c086a2 was 854427b, checked in by Christian Mauderer <christian.mauderer@…>, on Apr 6, 2018 at 8:35:42 AM

waf: Add configurations with different modules.

Update #3351

  • Property mode set to 100644
File size: 12.7 KB
Line 
1Guidelines for Developing and Contributing Code
2===============================================
3
4Introduction
5------------
6
7This guide aims to help developing and contributing code to the libbsd.  One
8goal of the libbsd is to stay in synchronization with FreeBSD.  This is only
9feasible if certain rules are in place.  Otherwise, managing more than a
10thousand imported source files will become too labour intensive eventually.
11
12What is in the Git Repository
13-----------------------------
14
15The libbsd a self-contained kit with FreeBSD and RTEMS components pre-merged.
16The Waf wscript in libbsd is automatically generated.
17
18Any changes to source in the `freebsd` directories will need to be merged
19upstream into our master FreeBSD checkout, the `freebsd-org` submodule.
20
21The repository contains two FreeBSD source trees.  In the `freebsd` directory
22are the so called *managed* FreeBSD sources used to build the BSD library.  The
23FreeBSD source in `freebsd-org` is the *master* version.  The
24`freebsd-to-rtems.py` script is used to transfer files between the two trees.
25In general terms, if you have modified managed FreeBSD sources, you will need
26to run the script in *revert* or *reverse* mode using the `-R` switch.  This
27will copy the source back to your local copy of the master FreeBSD source so
28you can run `git diff` against the upstream FreeBSD source.  If you want to
29transfer source files from the master FreeBSD source to the manged FreeBSD
30sources, then you must run the script in *forward* mode (the default).
31
32Organization
33------------
34
35The top level directory contains a few directories and files. The following
36are important to understand
37
38* `freebsd-to-rtems.py` - script to convert to and free FreeBSD and RTEMS trees,
39* `create-kernel-namespace.sh` - script to create the kernel namespace header <machine/rtems-bsd-kernel-namespace.h,
40* `wscript` - automatically generated,
41* `freebsd/` - from FreeBSD by script,
42* `rtemsbsd/` - RTEMS specific implementations of FreeBSD kernel support routines,
43* `testsuite/` - RTEMS specific tests, and
44* `libbsd.txt` - documentation in Asciidoc.
45
46Moving Code Between Managed and Master FreeBSD Source
47-----------------------------------------------------
48
49The script `freebsd-to-rtems.py` is used to copy code from FreeBSD to the
50rtems-libbsd tree and to reverse this process. This script attempts to
51automate this process as much as possible and performs some transformations
52on the FreeBSD code. Its command line arguments are shown below:
53
54```
55freebsd-to-rtems.py [args]
56  -?|-h|--help      print this and exit
57  -d|--dry-run      run program but no modifications
58  -D|--diff         provide diff of files between trees
59  -e|--early-exit   evaluate arguments, print results, and exit
60  -m|--makefile     Warning: depreciated and will be removed
61  -b|--buildscripts just generate the build scripts
62  -S|--stats        Print a statistics report
63  -R|--reverse      default FreeBSD -> RTEMS, reverse that
64  -r|--rtems        RTEMS Libbsd directory (default: '.')
65  -f|--freebsd      FreeBSD SVN directory (default: 'freebsd-org')
66  -v|--verbose      enable verbose output mode
67```
68
69In its default mode of operation, freebsd-to-rtems.py is used to copy code
70from FreeBSD to the rtems-libbsd tree and perform transformations.
71
72In *reverse mode*, this script undoes those transformations and copies
73the source code back to the *master* FreeBSD tree. This allows us to do
74'git diff', evaluate changes made by the RTEMS Project, and report changes
75back to FreeBSD upstream.
76
77In either mode, the script may be asked to perform a dry-run or be verbose.
78Also, in either mode, the script is also smart enough to avoid copying over
79files which have not changed. This means that the timestamps of files are
80not changed unless the contents change. The script will also report the
81number of files which changed. In verbose mode, the script will print
82the name of the files which are changed.
83
84To add or update files in the RTEMS FreeBSD tree first run the *reverse mode*
85and move the current set of patches FreeBSD. The script may warn you if a file
86is not present at the destination for the direction. This can happen as files
87not avaliable at the FreeBSD snapshot point have been specially added to the
88RTEMS FreeBSD tree. Warnings can also appear if you have changed the list of
89files in libbsd.py. The reverse mode will result in the FreeBSD having
90uncommitted changes. You can ignore these. Once the reverse process has
91finished edit libbsd.py and add any new files then run the forwad mode to bring
92those files into the RTEMS FreeBSD tree.
93
94The following is an example forward run with no changes.
95
96```
97$ ./freebsd-to-rtems.py -v
98Verbose:                     yes (1)
99Dry Run:                     no
100Diff Mode Enabled:           no
101Only Generate Build Scripts: no
102RTEMS Libbsd Directory:      .
103FreeBSD SVN Directory:       freebsd-org
104Direction:                   forward
105Forward from FreeBSD GIT into  .
1060 file(s) were changed:
107```
108
109The script may also be used to generate a diff in either forward or reverse
110direction.
111
112You can add more than one verbose option (-v) to the command line and get more
113detail and debug level information from the command.
114
115FreeBSD Version of Imported Files and Directories
116-------------------------------------------------
117
118* *, trunk, 2017-04-04, 642b174daddbd0efd9bb5f242c43f4ab4db6869f.
119
120How to import code from FreeBSD
121-------------------------------
122
123* In case you import files from a special FreeBSD version, then update the list above.
124* Run `git status` and make sure your working directory is clean.
125* Run `./freebsd-to-rtems.py -R`
126* Run `./freebsd-to-rtems.py`
127* Run `git status` and make sure your working directory is clean.  If you see modified files, then the `freebsd-to-rtems.py` script needs to be fixed first.
128* Add the files to import to `libbsd.py` and your intended build set (for example `buildset/default.ini`.
129* Run `./freebsd-to-rtems.py`
130* Immediately check in the imported files without the changes to `libbsd.py` and the buildsets.  Do not touch the imported files yourself at this point.
131* Port the imported files to RTEMS.  See 'Rules for Modifying FreeBSD Source'.
132* Add a test to the testsuite if possible.
133* Run `./create-kernel-namespace.sh` if you imported kernel space headers.  Add only your new defines via `git add -p rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h`.
134* Create one commit from this.
135
136The -S or --stats option generates reports the changes we have made to
137FreeBSD. If the code has been reserved into the original FreeBSD tree it will
138show nothing has changed. To see what we have change:
139
140```
141$ cd freebsd-org
142$ git checkout -- .
143$ cd ..
144$ ./freebsd-to-rtems.py -R -S -d
145 ```
146
147The report lists the files change based on the opacity level. The opacity is a
148measure on how much of a file differs from the original FreeBSD source. The
149lower the value the more transparent the source file it.
150
151Porting of User-Space Utilities
152------------------------------
153
154The theory behind the described method is to put all BSS and initialized data
155objects into a named section. This section then will be saved before the code is
156executed and restored after it has finished. This method limits to a single
157threaded execution of the application but minimizes the necessary changes to the
158original FreeBSD code.
159
160* Import and commit the unchanged source files like described above.
161* Add the files to the [libbsd.py](libbsd.py) and build them.
162* Check the sources for everything that can be made const. This type of patches
163  should go back to the upstream FreeBSD sources.
164* Move static variables out of functions if necessary (search for
165  "\tstatic"). These patches most likely will not be accepted into FreeBSD.
166* Add a rtems_bsd_command_PROGNAME() wrapper function to the source file
167  containing the main function (e.g. PROGNAME = pfctl). For an example look at
168  `rtems_bsd_command_pfctl()` in [pfctl.c](freebsd/sbin/pfctl/pfctl.c).
169* You probably have to use getopt_r() instead of getopt(). Have a look at
170  [pfctl.c](freebsd/sbin/pfctl/pfctl.c).
171* Build the libbsd without optimization.
172* Use the `userspace-header-gen.py` to generate some necessary header
173  files. It will generate one `rtems-bsd-PROGNAME-MODULE-data.h` per object file, one
174  `rtems-bsd-PROGNAME-namespace.h` and one `rtems-bsd-PROGNAME-data.h`. To call
175  the script, you have to compile the objects and afterwards run the helper
176  script with a call similar to this one:
177  `python ./userspace-header-gen.py build/arm-rtems4.12-xilinx_zynq_a9_qemu/freebsd/sbin/pfctl/*.o -p pfctl`
178  Replace the name (given via -p option) by the name of the userspace tool. It
179  has to match the name that is used in the RTEMS linker set further below.
180* If you regenerated files that have already been generated, you may have to
181  remove RTEMS-specific names from the namespace. The defaults (linker set names
182  and rtems_bsd_program_.*) should already be filtered.
183* Put the generated header files into the same folder like the source files.
184* At the top of each source file place the following right after the user-space header:
185  ```c
186  #ifdef __rtems__
187  #include <machine/rtems-bsd-program.h>
188  #include "rtems-bsd-PROGNAME-namespace.h"
189  #endif /* __rtems__ */
190  ```
191  The following command may be useful:
192  ```
193  sed -i 's%#include <machine/rtems-bsd-user-space.h>%#include <machine/rtems-bsd-user-space.h>\n#ifdef __rtems__\n#include <machine/rtems-bsd-program.h>\n#include "rtems-bsd-PROGNAME-namespace.h"\n#endif /* __rtems__ */%' *.c
194  ```
195* At the bottom of each source file place the follwing:
196  ```c
197  #ifdef __rtems__
198  #include "rtems-bsd-PROGNAME-FILE-data.h"
199  #endif /* __rtems__ */
200  ```
201  The following command may be useful:
202  ```
203  for i in *.c ; do n=$(basename $i .c) ; echo -e "#ifdef __rtems__\n#include \"rtems-bsd-PROGNAME-$n-data.h\"\n#endif /* __rtems__ */" >> $i ; done
204  ```
205* Create one compilable commit.
206
207Rules for Modifying FreeBSD Source
208----------------------------------
209
210Changes in FreeBSD files must be done using `__rtems__` C pre-processor guards.
211This makes synchronization with the FreeBSD upstream easier and is very
212important.  Patches which do not follow these rules will be rejected.  Only add
213lines.  If your patch contains lines starting with a `-`, then this is wrong.
214Subtract code by added `#ifndef __rtems__`.  For example:
215
216```c
217/* Global variables for the kernel. */
218
219#ifndef __rtems__
220/* 1.1 */
221extern char kernelname[MAXPATHLEN];
222#endif /* __rtems__ */
223
224extern int tick;                        /* usec per tick (1000000 / hz) */
225```
226
227```c
228#if defined(_KERNEL) || defined(_WANT_FILE)
229#ifdef __rtems__
230#include <rtems/libio_.h>
231#include <sys/fcntl.h>
232#endif /* __rtems__ */
233/*
234 * Kernel descriptor table.
235 * One entry for each open kernel vnode and socket.
236 *
237 * Below is the list of locks that protects members in struct file.
238 *
239 * (f) protected with mtx_lock(mtx_pool_find(fp))
240 * (d) cdevpriv_mtx
241 * none not locked
242 */
243```
244
245```c
246extern int profprocs;                   /* number of process's profiling */
247#ifndef __rtems__
248extern volatile int ticks;
249#else /* __rtems__ */
250#include <rtems/score/watchdogimpl.h>
251#define ticks _Watchdog_Ticks_since_boot
252#endif /* __rtems__ */
253
254#endif /* _KERNEL */
255```
256
257Add nothing (even blank lines) before or after the `__rtems__` guards.  Always
258include a `__rtems__` in the guards to make searches easy, so use
259
260* `#ifndef __rtems__`,
261* `#ifdef __rtems__`,
262* `#else /* __rtems__ */`, and
263* `#endif /* __rtems__ */`.
264
265The guards must start at the begin of the line.  Examples for wrong guards:
266
267```c
268static void
269guards_must_start_at_the_begin_of_the_line(int j)
270{
271
272        /* WRONG */
273        #ifdef __rtems__
274        return (j + 1);
275        #else /* __rtems__ */
276        return (j + 2);
277        #endif /* __rtems__ */
278}
279
280static void
281missing_rtems_comments_in_the_guards(int j)
282{
283
284#ifdef __rtems__
285        return (j + 3);
286/* WRONG */
287#else
288        return (j + 4);
289#endif
290}
291```
292
293The FreeBSD build and configuration system uses option header files, e.g.
294`#include "opt_xyz.h"` in an unmodified FreeBSD file.  This include is
295transformed by the import script into `#include <rtems/bsd/local/opt_xyz.h>`.  Do
296not disable option header includes via guards.  Instead, add an empty option
297header, e.g. `touch rtemsbsd/include/rtems/bsd/local/opt_xyz.h`.
298```c
299/* WRONG */
300#ifndef __rtems__
301#include <rtems/bsd/local/opt_xyz.h>
302#endif /* __rtems__ */
303```
304
305In general, provide empty header files and do not guard includes.
306
307For new code use
308[STYLE(9)](http://www.freebsd.org/cgi/man.cgi?query=style&apropos=0&sektion=9).
309
310Do not format original FreeBSD code.  Do not perform white space changes even
311if you get git commit warnings.
312
313Automatically Generated FreeBSD Files
314-------------------------------------
315
316Some source and header files are automatically generated during the FreeBSD
317build process.  The `Makefile.todo` file performs this manually.  The should be
318included in `freebsd-to-rtems.py` script some time in the future.  For details,
319see also
320[KOBJ(9)](http://www.freebsd.org/cgi/man.cgi?query=kobj&sektion=9&apropos=0).
Note: See TracBrowser for help on using the repository browser.