source: rtems-libbsd/libbsd.txt @ a3d2498

4.1155-freebsd-126-freebsd-12freebsd-9.3
Last change on this file since a3d2498 was a3d2498, checked in by Sebastian Huber <sebastian.huber@…>, on 04/16/12 at 09:57:45

Add SVN checkout command description

  • Property mode set to 100644
File size: 11.2 KB
Line 
1RTEMS BSD USB and TCP/IP Developers Guide
2=========================================
3Joel Sherrill <joel.sherrill@oarcorp.com>
4:Author Initials: JRS
5:toc:
6:icons:
7:numbered:
8:website: http://www.rtems.org/
9
10RTEMS uses FreeBSD as the source of its TCP/IP and USB stacks.
11This is a developers guide which captures information on the
12process of merging code from FreeBSD, building this library,
13RTEMS specific support files, and general guidelines on what
14modifications to the FreeBSD source are permitted.
15
16Goals of this effort are:
17
18* Update TCP/IP and provide USB in RTEMS
19* Ease updating to future FreeBSD versions
20* Ease tracking changes in FreeBSD code
21* Minimize manual changes in FreeBSD code
22* Define stable kernel/device driver API which is implemented
23by both RTEMS and FreeBSD. This is the foundation of the port.
24
25We will work to push our changes upstream to the FreeBSD Project
26and minimize changes required at each update point.
27
28**************************************************************
29This is a work in progress and is very likely to be incomplete.
30Please help by adding to it.
31**************************************************************
32
33== Source Code Version Information
34
35* FreeBSD 8.2 SVN r222496
36* RTEMS 4.11
37  - BSP must have support for all new BSD sys sections
38  - It is preferable if the BSP uses linkcmds.base.
39  - BSP must be from an architecture with Programmable Interrupt Controller
40    interrupt model.
41
42The FreeBSD 8.2 SVN checkout will generally be referred to as the
43FreeBSD source in this document. An archive of the FreeBSD 8.2 SVN
44archive used is located at
45  http://www.rtems.org/ftp/pub/rtems/people/joel/freebsd/
46The SVN checkout command is this
47  svn co http://svn.freebsd.org/base/releng/8.2/sys/ -r222496 freebsd-8.2
48
49== Issues and To Do
50* Sebastian Huber: mentioned some simple test code which would verify
51  that the BSD code/and or USB stack was initialized.  This has been
52  sent to Joel Sherrill and is pending merger.
53
54* Sebastian Huber and Joel Sherrill discussed the need for a a basic USB
55  functionality test that is known to work on qemu pc.
56
57* Adapt generic IRQ PIC interface code to Simple Vectored Interrupt Model
58  so that those architectures can use new TCP/IP and USB code.
59
60* in_cksum implementations for architectures not supported by FreeBSD.
61  This will require figuring out where to put implementations that do
62  not originate from FreeBSD and are populated via the script.
63
64* FreeBSD generic in_cksum implementation is missing in_cksum_split so
65  currently cannot be used.
66
67* How does one initialize the TCP/IP stack?
68
69* linker section issues: I have undefined symbols for
70  _bsd__start_set_sysinit_set and _bsd__stop_set_sysinit_set.
71  Is this the only type of new section magic?  What about the old sysctl_set?
72  I added this to my linkcmds. 
73
74[listing]
75----
76    /* sysinit section? */
77    . = ALIGN (16);
78    _bsd__start_set_sysinit_set = .;
79    *(set_sys_init_*);
80    _bsd__stop_set_sysinit_set = .;
81
82----
83
84* Convert all BSP linkcmds to use a linkcmds.base so the sections are
85easier to insert.
86
87* rtems-bsd-init-with-irq.c:
88  rtems_bsd_initialize_with_interrupt_server() has reference to
89    rtems_interrupt_server_initialize() and this method is unimplemented
90    - XXX BSP implements pieces
91    - BSPs using this software stack must support it apparently.
92    - What about Simple Vectored architectures?
93
94* maxproc variable referenced by rtems-bsd-resource.c.  What should it
95be set to?
96
97* ngroups_max variable referenced by rtems-bsd-prot.c.  - What should
98it be set to?
99
100* NIC Device Drivers
101- Only common NIC drivers have been included in the initial set. These do not
102include any system on chip or ISA drivers.
103- The ISA drivers require more BSD infrastructure to be addressed. This was
104outside the scope of the initial porting effort.
105
106== FreeBSD Source
107
108You should be able to rely on FreebSD manual pages and documentation
109for details on the code itself.
110
111=== Automatically Generated FreeBSD Files
112
113The FreeBSD source tarball includes a file named Makefile.rtems which
114has stanzas to automatically generate some files using awk. For details
115on this, see http://www.freebsd.org/cgi/man.cgi?query=kobj&apropos=0&sektion=0&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html
116
117XXX This needs more detail.
118
119=== Rules for Modifying FreeBSD Source
120
121* Only add lines.  Subtract code by added "ifndef __rtems__". This makes
122merging easier in the future.
123
124== libbsd Source
125
126=== What is in git
127
128The git source is a self-contained kit with FreeBSD and RTEMS components
129pre-merged. The Makefile in this kit is automatically generated.
130
131Any changes to sources in the freebsd or contrib directories will need to
132be merged upstream into our master FreeBSD svn checkout.
133
134The FreeBSD sources managed in RTEMS libbsd git repository (e.g. contrib
135and freebsd directories) contain the "managed" version of the
136FreeBSD source.  The FreeBSD SVN source is the "master" version. The
137freebsd-to-rtems.py script is used to transfer files between the two
138trees. In general terms, if you have modified FreeBSD in the RTEMS libbsd
139tree, you want to run the script in "revert" or "reverse" mode to move
140the source back so you can run an "svn diff" against the upstream FreeBSD
141source. If you want to transfer source from the FreeBSD SVN checkout to
142the RTEMS libbsd tree, then you want to run the script in "forward" or
143default mode.
144
145=== Building RTEMS libbsd source
146
147You need to configure RTEMS for the desired BSP and install it. The
148following is the script used to build the powerpc/psim BSP for our
149internal testing purposes:
150
151[listing]
152----
153#! /bin/sh
154
155cd ${HOME}/newbsd
156rm -rf b-psim
157mkdir b-psim
158cd b-psim
159../git/rtems/configure --target=powerpc-rtems4.11 \
160  --enable-rtemsbsp=psim --disable-networking \
161  --enable-tests=samples \
162  --prefix=${HOME}/newbsd/bsp-install >c.log 2>&1 && \
163  make >b.log 2>&1 && \
164  make install >i.log 2>&1
165echo $?
166----
167
168Then edit the file config.inc to set RTEMS_MAKEFILE_PATH appropriately
169to indicate the ${prefix}/${target}/${BSP}.  Continuing on the above,
170the config.inc used to match the above is:
171
172[listing]
173----
174RTEMS_MAKEFILE_PATH = ${HOME}/newbsd/bsp-install/powerpc-rtems4.11/psim/
175INSTALL_BASE = ${HOME}/newbsd/install
176----
177
178The above installs the RTEMS libbsd kit into a separate place from
179RTEMS and the BSP. The RTEMS libbsd tests are built against an installed
180image of the RTEMS libbsd. By keeping it in a separate installation point
181from RTEMS itself, this makes it easier to remove a libbsd installation
182and have a clean test point.
183
184[listing]
185----
186make
187make install
188make -C testsuite
189----
190
191At this point, we expect multiple linker errors. That is what we are
192currently working on.
193
194=== Organization
195
196The top level directory contains a few directories and files. The following
197are important to understand:
198
199* freebsd-to-rtems.py - script to convert to and free FreeBSD and RTEMS trees
200* Makefile - automatically generated
201* contrib/ - from FreeBSD by script.
202* freebsd/ - from FreeBSD by script.
203* rtemsbsd/ - RTEMS specific implementations of FreeBSD kernel support routines.
204* testsuite/ - RTEMS specific tests
205* libbsd.txt - Documentation in Asciidoc
206
207== Moving Code Between FreeBSD SVN and RTEMS libbsd
208
209The script freebsd-to-rtems.py is used to copy code from FreeBSD to the
210RTEMS libbsd tree and to reverse this process. This script attempts to
211automate this process as much as possible and performs some transformations
212on the FreeBSD code. Its command line arguments are shown below:
213
214[listing]
215----
216freebsd-to-rtems.py [args]
217  -?|-h|--help     print this and exit
218  -d|--dry-run     run program but no modifications
219  -D|--diff        provide diff of files between trees
220  -e|--early-exit  evaluate arguments, print results, and exit
221  -m|--makefile    just generate Makefile
222  -R|--reverse     default FreeBSD -> RTEMS, reverse that
223  -r|--rtems       RTEMS directory
224  -f|--freebsd     FreeBSD directory
225  -v|--verbose     enable verbose output mode
226----
227
228In its default mode of operation, freebsd-to-rtems.py is used to copy code
229from FreeBSD to the RTEMS libbsd tree and perform transformations.  In forward
230mode, the script may be requested to just generate the Makefile.
231
232In "reverse mode", this script undoes those transformations and copies
233the source code back to the FreeBSD SVN tree. This allows us to do
234'svn diff', evaluate changes made by the RTEMS Project, and report changes
235back to FreeBSD upstream.
236
237In either mode, the script may be asked to perform a dry-run or be verbose.
238Also, in either mode, the script is also smart enough to avoid copying over
239files which have not changed. This means that the timestamps of files are
240not changed unless the contents change. The script will also report the
241number of files which changed. In verbose mode, the script will print
242the name of the files which are changed.
243
244The following is an example forward run with no changes.
245
246[listing]
247----
248$ ~/newbsd/git/libbsd-8.2/freebsd-to-rtems.py \
249    -r /home/joel/newbsd/git/libbsd-8.2 \
250    -f /home/joel/newbsd/libbsd/freebsd-8.2 -v
251Verbose:                yes
252Dry Run:                no
253Only Generate Makefile: no
254RTEMS Directory:        /home/joel/newbsd/git/libbsd-8.2
255FreeBSD Directory:      /home/joel/newbsd/libbsd/freebsd-8.2
256Direction:              forward
257Generating into /home/joel/newbsd/git/libbsd-8.2
2580 files were changed.
259----
260
261The script may also be used to generate a diff in either forward or reverse
262direction.
263
264== Initialization of RTEMS Libbsd
265
266The initialization of the RTEMS Libbsd is based on the FreeBSD SYSINIT
267infrastructure. This is simply because we are initializing a subset of
268FreeBSD. For details refer to http://www.freebsd.org/cgi/man.cgi?query=SYSINIT&sektion=9&apropos=0&manpath=FreeBSD+9-current
269
270The key to initializing a system is to ensure that the desired device
271drivers are explicitly pulled into the linked application. This plus
272linking against the Libsd library will pull in the necessary FreeBSD
273infrastructure. The SYSINIT structures are automatically built at link
274time and the various initialization routines will thus be executed in'
275the correct order.
276
277XXX This needs more details.
278
279=== SYSCTL_NODE Example
280
281During development, we had an undefined reference to
282_bsd_sysctl__net_children that we had trouble tracking down. Thanks to
283Chris Johns, we located it. He explained how to read SYSCTL_NODE
284definitions. This line from freebsd/netinet/in_proto.c is attempting
285to add the "inet" node to the parent node "_net".
286
287[listing]
288----
289SYSCTL_NODE(_net,      PF_INET,         inet,   CTLFLAG_RW, 0,
290        "Internet Family");
291----
292
293Our problem was that we could not find where _bsd_sysctl__net_children
294was defined. Chris suggested that when in doubt compile with -save-temps
295and look at the preprocessed .i files. But he did not need that. He
296explained that this the symbol name _bsd_sysctl__net_children was
297automatically generated by a SYSCTL_NODE as follows:
298
299* _bsd_ - added by RTEMS modifications to SYSCTL_NODE macro
300* sysctl_ - boilerplace added by SYSCTL_NODE macro
301* "" - empty string for parent node
302* net - name of SYSCTL_NODE
303* children - added by SYSCTL macros
304 
305This was all generated by a support macro declaring the node as this:
306
307[listing]
308----
309struct sysctl_oid_list SYSCTL_NODE_CHILDREN(parent, name);
310----
311
312Given this information, we located this SYSCTL_NODE declaration in
313kern/kern_mib.c
314
315[listing]
316----
317SYSCTL_NODE(, CTL_KERN,   kern,   CTLFLAG_RW, 0,
318        "High kernel, proc, limits &c");
319----
320
Note: See TracBrowser for help on using the repository browser.