1 | RTEMS BSD USB and TCP/IP Developers Guide |
---|
2 | ========================================= |
---|
3 | Joel Sherrill <joel.sherrill@oarcorp.com> |
---|
4 | :Author Initials: JRS |
---|
5 | :toc: |
---|
6 | :icons: |
---|
7 | :numbered: |
---|
8 | :website: http://www.rtems.org/ |
---|
9 | |
---|
10 | RTEMS uses FreeBSD as the source of its TCP/IP and USB stacks. |
---|
11 | This is a developers guide which captures information on the |
---|
12 | process of merging code from FreeBSD, building this library, |
---|
13 | RTEMS specific support files, and general guidelines on what |
---|
14 | modifications to the FreeBSD source are permitted. |
---|
15 | |
---|
16 | Goals of this effort are: |
---|
17 | |
---|
18 | * Update TCP/IP and provide USB in RTEMS |
---|
19 | * Ease updating to future FreeBSD versions |
---|
20 | * Ease tracking changes in FreeBSD code |
---|
21 | * Minimize manual changes in FreeBSD code |
---|
22 | * Define stable kernel/device driver API which is implemented |
---|
23 | by both RTEMS and FreeBSD. This is the foundation of the port. |
---|
24 | |
---|
25 | We will work to push our changes upstream to the FreeBSD Project |
---|
26 | and minimize changes required at each update point. |
---|
27 | |
---|
28 | ************************************************************** |
---|
29 | This is a work in progress and is very likely to be incomplete. |
---|
30 | Please help by adding to it. |
---|
31 | ************************************************************** |
---|
32 | |
---|
33 | == Source Code Version Information |
---|
34 | |
---|
35 | * FreeBSD 8.2 SVN r222496 |
---|
36 | * RTEMS 4.11 |
---|
37 | - BSP must have support for all new BSD sys sections |
---|
38 | - It is preferable if the BSP uses linkcmds.base. |
---|
39 | - BSP must be from an architecture with Programmable Interrupt Controller |
---|
40 | interrupt model. |
---|
41 | |
---|
42 | The FreeBSD 8.2 SVN checkout will generally be referred to as the |
---|
43 | FreeBSD source in this document. An archive of the FreeBSD 8.2 SVN |
---|
44 | archive used is located at |
---|
45 | http://www.rtems.org/ftp/pub/rtems/people/joel/freebsd/ |
---|
46 | The SVN checkout command is this |
---|
47 | svn co http://svn.freebsd.org/base/releng/8.2/sys/ -r222496 freebsd-8.2 |
---|
48 | |
---|
49 | == Issues and To Do |
---|
50 | * Sebastian Huber: mentioned some simple test code which would verify |
---|
51 | that the BSD code/and or USB stack was initialized. This has been |
---|
52 | sent to Joel Sherrill and is pending merger. |
---|
53 | |
---|
54 | * Sebastian Huber and Joel Sherrill discussed the need for a a basic USB |
---|
55 | functionality test that is known to work on qemu pc. |
---|
56 | |
---|
57 | * Adapt generic IRQ PIC interface code to Simple Vectored Interrupt Model |
---|
58 | so that those architectures can use new TCP/IP and USB code. |
---|
59 | |
---|
60 | * in_cksum implementations for architectures not supported by FreeBSD. |
---|
61 | This will require figuring out where to put implementations that do |
---|
62 | not originate from FreeBSD and are populated via the script. |
---|
63 | |
---|
64 | * FreeBSD generic in_cksum implementation is missing in_cksum_split so |
---|
65 | currently cannot be used. |
---|
66 | |
---|
67 | * How does one initialize the TCP/IP stack? |
---|
68 | |
---|
69 | * linker section issues: I have undefined symbols for |
---|
70 | _bsd__start_set_sysinit_set and _bsd__stop_set_sysinit_set. |
---|
71 | Is this the only type of new section magic? What about the old sysctl_set? |
---|
72 | I added this to my linkcmds. |
---|
73 | |
---|
74 | [listing] |
---|
75 | ---- |
---|
76 | /* sysinit section? */ |
---|
77 | . = ALIGN (16); |
---|
78 | _bsd__start_set_sysinit_set = .; |
---|
79 | *(set_sys_init_*); |
---|
80 | _bsd__stop_set_sysinit_set = .; |
---|
81 | |
---|
82 | ---- |
---|
83 | |
---|
84 | * Convert all BSP linkcmds to use a linkcmds.base so the sections are |
---|
85 | easier to insert. |
---|
86 | |
---|
87 | * rtems-bsd-init-with-irq.c: |
---|
88 | rtems_bsd_initialize_with_interrupt_server() has reference to |
---|
89 | rtems_interrupt_server_initialize() and this method is unimplemented |
---|
90 | - XXX BSP implements pieces |
---|
91 | - BSPs using this software stack must support it apparently. |
---|
92 | - What about Simple Vectored architectures? |
---|
93 | |
---|
94 | * maxproc variable referenced by rtems-bsd-resource.c. What should it |
---|
95 | be set to? |
---|
96 | |
---|
97 | * ngroups_max variable referenced by rtems-bsd-prot.c. - What should |
---|
98 | it be set to? |
---|
99 | |
---|
100 | * NIC Device Drivers |
---|
101 | - Only common NIC drivers have been included in the initial set. These do not |
---|
102 | include any system on chip or ISA drivers. |
---|
103 | - The ISA drivers require more BSD infrastructure to be addressed. This was |
---|
104 | outside the scope of the initial porting effort. |
---|
105 | |
---|
106 | == FreeBSD Source |
---|
107 | |
---|
108 | You should be able to rely on FreebSD manual pages and documentation |
---|
109 | for details on the code itself. |
---|
110 | |
---|
111 | === Automatically Generated FreeBSD Files |
---|
112 | |
---|
113 | The FreeBSD source tarball includes a file named Makefile.rtems which |
---|
114 | has stanzas to automatically generate some files using awk. For details |
---|
115 | on this, see http://www.freebsd.org/cgi/man.cgi?query=kobj&apropos=0&sektion=0&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html |
---|
116 | |
---|
117 | XXX This needs more detail. |
---|
118 | |
---|
119 | === Rules for Modifying FreeBSD Source |
---|
120 | |
---|
121 | * Only add lines. Subtract code by added "ifndef __rtems__". This makes |
---|
122 | merging easier in the future. |
---|
123 | |
---|
124 | == libbsd Source |
---|
125 | |
---|
126 | === What is in git |
---|
127 | |
---|
128 | The git source is a self-contained kit with FreeBSD and RTEMS components |
---|
129 | pre-merged. The Makefile in this kit is automatically generated. |
---|
130 | |
---|
131 | Any changes to sources in the freebsd or contrib directories will need to |
---|
132 | be merged upstream into our master FreeBSD svn checkout. |
---|
133 | |
---|
134 | The FreeBSD sources managed in RTEMS libbsd git repository (e.g. contrib |
---|
135 | and freebsd directories) contain the "managed" version of the |
---|
136 | FreeBSD source. The FreeBSD SVN source is the "master" version. The |
---|
137 | freebsd-to-rtems.py script is used to transfer files between the two |
---|
138 | trees. In general terms, if you have modified FreeBSD in the RTEMS libbsd |
---|
139 | tree, you want to run the script in "revert" or "reverse" mode to move |
---|
140 | the source back so you can run an "svn diff" against the upstream FreeBSD |
---|
141 | source. If you want to transfer source from the FreeBSD SVN checkout to |
---|
142 | the RTEMS libbsd tree, then you want to run the script in "forward" or |
---|
143 | default mode. |
---|
144 | |
---|
145 | === Building RTEMS libbsd source |
---|
146 | |
---|
147 | You need to configure RTEMS for the desired BSP and install it. The |
---|
148 | following is the script used to build the powerpc/psim BSP for our |
---|
149 | internal testing purposes: |
---|
150 | |
---|
151 | [listing] |
---|
152 | ---- |
---|
153 | #! /bin/sh |
---|
154 | |
---|
155 | cd ${HOME}/newbsd |
---|
156 | rm -rf b-psim |
---|
157 | mkdir b-psim |
---|
158 | cd b-psim |
---|
159 | ../git/rtems/configure --target=powerpc-rtems4.11 \ |
---|
160 | --enable-rtemsbsp=psim --disable-networking \ |
---|
161 | --enable-tests=samples \ |
---|
162 | --prefix=${HOME}/newbsd/bsp-install >c.log 2>&1 && \ |
---|
163 | make >b.log 2>&1 && \ |
---|
164 | make install >i.log 2>&1 |
---|
165 | echo $? |
---|
166 | ---- |
---|
167 | |
---|
168 | Then edit the file config.inc to set RTEMS_MAKEFILE_PATH appropriately |
---|
169 | to indicate the ${prefix}/${target}/${BSP}. Continuing on the above, |
---|
170 | the config.inc used to match the above is: |
---|
171 | |
---|
172 | [listing] |
---|
173 | ---- |
---|
174 | RTEMS_MAKEFILE_PATH = ${HOME}/newbsd/bsp-install/powerpc-rtems4.11/psim/ |
---|
175 | INSTALL_BASE = ${HOME}/newbsd/install |
---|
176 | ---- |
---|
177 | |
---|
178 | The above installs the RTEMS libbsd kit into a separate place from |
---|
179 | RTEMS and the BSP. The RTEMS libbsd tests are built against an installed |
---|
180 | image of the RTEMS libbsd. By keeping it in a separate installation point |
---|
181 | from RTEMS itself, this makes it easier to remove a libbsd installation |
---|
182 | and have a clean test point. |
---|
183 | |
---|
184 | [listing] |
---|
185 | ---- |
---|
186 | make |
---|
187 | make install |
---|
188 | make -C testsuite |
---|
189 | ---- |
---|
190 | |
---|
191 | At this point, we expect multiple linker errors. That is what we are |
---|
192 | currently working on. |
---|
193 | |
---|
194 | === Organization |
---|
195 | |
---|
196 | The top level directory contains a few directories and files. The following |
---|
197 | are important to understand: |
---|
198 | |
---|
199 | * freebsd-to-rtems.py - script to convert to and free FreeBSD and RTEMS trees |
---|
200 | * Makefile - automatically generated |
---|
201 | * contrib/ - from FreeBSD by script. |
---|
202 | * freebsd/ - from FreeBSD by script. |
---|
203 | * rtemsbsd/ - RTEMS specific implementations of FreeBSD kernel support routines. |
---|
204 | * testsuite/ - RTEMS specific tests |
---|
205 | * libbsd.txt - Documentation in Asciidoc |
---|
206 | |
---|
207 | == Moving Code Between FreeBSD SVN and RTEMS libbsd |
---|
208 | |
---|
209 | The script freebsd-to-rtems.py is used to copy code from FreeBSD to the |
---|
210 | RTEMS libbsd tree and to reverse this process. This script attempts to |
---|
211 | automate this process as much as possible and performs some transformations |
---|
212 | on the FreeBSD code. Its command line arguments are shown below: |
---|
213 | |
---|
214 | [listing] |
---|
215 | ---- |
---|
216 | freebsd-to-rtems.py [args] |
---|
217 | -?|-h|--help print this and exit |
---|
218 | -d|--dry-run run program but no modifications |
---|
219 | -D|--diff provide diff of files between trees |
---|
220 | -e|--early-exit evaluate arguments, print results, and exit |
---|
221 | -m|--makefile just generate Makefile |
---|
222 | -R|--reverse default FreeBSD -> RTEMS, reverse that |
---|
223 | -r|--rtems RTEMS directory |
---|
224 | -f|--freebsd FreeBSD directory |
---|
225 | -v|--verbose enable verbose output mode |
---|
226 | ---- |
---|
227 | |
---|
228 | In its default mode of operation, freebsd-to-rtems.py is used to copy code |
---|
229 | from FreeBSD to the RTEMS libbsd tree and perform transformations. In forward |
---|
230 | mode, the script may be requested to just generate the Makefile. |
---|
231 | |
---|
232 | In "reverse mode", this script undoes those transformations and copies |
---|
233 | the source code back to the FreeBSD SVN tree. This allows us to do |
---|
234 | 'svn diff', evaluate changes made by the RTEMS Project, and report changes |
---|
235 | back to FreeBSD upstream. |
---|
236 | |
---|
237 | In either mode, the script may be asked to perform a dry-run or be verbose. |
---|
238 | Also, in either mode, the script is also smart enough to avoid copying over |
---|
239 | files which have not changed. This means that the timestamps of files are |
---|
240 | not changed unless the contents change. The script will also report the |
---|
241 | number of files which changed. In verbose mode, the script will print |
---|
242 | the name of the files which are changed. |
---|
243 | |
---|
244 | The following is an example forward run with no changes. |
---|
245 | |
---|
246 | [listing] |
---|
247 | ---- |
---|
248 | $ ~/newbsd/git/libbsd-8.2/freebsd-to-rtems.py \ |
---|
249 | -r /home/joel/newbsd/git/libbsd-8.2 \ |
---|
250 | -f /home/joel/newbsd/libbsd/freebsd-8.2 -v |
---|
251 | Verbose: yes |
---|
252 | Dry Run: no |
---|
253 | Only Generate Makefile: no |
---|
254 | RTEMS Directory: /home/joel/newbsd/git/libbsd-8.2 |
---|
255 | FreeBSD Directory: /home/joel/newbsd/libbsd/freebsd-8.2 |
---|
256 | Direction: forward |
---|
257 | Generating into /home/joel/newbsd/git/libbsd-8.2 |
---|
258 | 0 files were changed. |
---|
259 | ---- |
---|
260 | |
---|
261 | The script may also be used to generate a diff in either forward or reverse |
---|
262 | direction. |
---|
263 | |
---|
264 | == Initialization of RTEMS libbsd |
---|
265 | |
---|
266 | The initialization of the RTEMS libbsd is based on the FreeBSD SYSINIT(9) |
---|
267 | infrastructure. The key to initializing a system is to ensure that the desired |
---|
268 | device drivers are explicitly pulled into the linked application. This plus |
---|
269 | linking against the libbsd library will pull in the necessary FreeBSD |
---|
270 | infrastructure. |
---|
271 | |
---|
272 | The FreeBSD kernel is not a library like the RTEMS kernel. It is a bunch of |
---|
273 | object files linked together. If we have a library, then creating the |
---|
274 | executable is simple. We begin with a start symbol and recursively resolve all |
---|
275 | references. With a bunch of object files linked together we need a different |
---|
276 | mechanism. Most object files don't know each other. Lets say we have a driver |
---|
277 | module. The rest of the system has no references to this driver module. The |
---|
278 | driver module needs a way to tell the rest of the system: Hey, kernel I am |
---|
279 | here, please use my services! |
---|
280 | |
---|
281 | This registration of independent components is performed by SYSINIT(9) and |
---|
282 | specializations: |
---|
283 | |
---|
284 | http://www.freebsd.org/cgi/man.cgi?query=SYSINIT |
---|
285 | |
---|
286 | The SYSINIT(9) uses some global data structures that are placed in a certain |
---|
287 | section. In the linker command file we need this: |
---|
288 | |
---|
289 | [listing] |
---|
290 | ---- |
---|
291 | .robsdsets : { |
---|
292 | _bsd__start_set_modmetadata_set = .; |
---|
293 | *(_bsd_set_modmetadata_set); |
---|
294 | _bsd__stop_set_modmetadata_set = .; |
---|
295 | _bsd__start_set_sysctl_set = .; |
---|
296 | *(_bsd_set_sysctl_set); |
---|
297 | _bsd__stop_set_sysctl_set = .; |
---|
298 | } > REGION_RODATA AT > REGION_RODATA_LOAD |
---|
299 | |
---|
300 | .rwbsdsets : { |
---|
301 | _bsd__start_set_sysinit_set = .; |
---|
302 | *(_bsd_set_sysinit_set); |
---|
303 | _bsd__stop_set_sysinit_set = .; |
---|
304 | } > REGION_DATA AT > REGION_DATA_LOAD |
---|
305 | ---- |
---|
306 | |
---|
307 | Here you can see, that these global data structures are collected into |
---|
308 | continuous memory areas. This memory area can be identified by start and stop |
---|
309 | symbols. This constructs a table of uniform items. |
---|
310 | |
---|
311 | The low level FreeBSD code calls at some time during the initialization the |
---|
312 | mi_startup() function (machine independent startup). This function will sort |
---|
313 | the SYSINIT(9) set and call handler functions which perform further |
---|
314 | initialization. The last step is the scheduler invocation. |
---|
315 | |
---|
316 | The SYSINIT(9) routines are run in mi_startup() which is called by |
---|
317 | rtems_bsd_initialize(). |
---|
318 | |
---|
319 | This is also explained in "The Design and Implementation of the FreeBSD |
---|
320 | Operating System" section 14.3 "Kernel Initialization". |
---|
321 | |
---|
322 | In RTEMS we have a library and not a bunch of object files. Thus we need a way |
---|
323 | to pull-in the desired services out of the libbsd. Here the |
---|
324 | "rtems-bsd-sysinit.h" comes into play. The SYSINIT(9) macros have been |
---|
325 | modified and extended for RTEMS in "sys/kernel.h": |
---|
326 | |
---|
327 | [listing] |
---|
328 | ---- |
---|
329 | #ifndef __rtems__ |
---|
330 | #define C_SYSINIT(uniquifier, subsystem, order, func, ident) \ |
---|
331 | static struct sysinit uniquifier ## _sys_init = { \ |
---|
332 | subsystem, \ |
---|
333 | order, \ |
---|
334 | func, \ |
---|
335 | (ident) \ |
---|
336 | }; \ |
---|
337 | DATA_SET(sysinit_set,uniquifier ## _sys_init) |
---|
338 | #else /* __rtems__ */ |
---|
339 | #define SYSINIT_ENTRY_NAME(uniquifier) \ |
---|
340 | _bsd_ ## uniquifier ## _sys_init |
---|
341 | #define SYSINIT_REFERENCE_NAME(uniquifier) \ |
---|
342 | _bsd_ ## uniquifier ## _sys_init_ref |
---|
343 | #define C_SYSINIT(uniquifier, subsystem, order, func, ident) \ |
---|
344 | struct sysinit SYSINIT_ENTRY_NAME(uniquifier) = { \ |
---|
345 | subsystem, \ |
---|
346 | order, \ |
---|
347 | func, \ |
---|
348 | (ident) \ |
---|
349 | }; \ |
---|
350 | DATA_SET(sysinit_set,SYSINIT_ENTRY_NAME(uniquifier)) |
---|
351 | #define SYSINIT_REFERENCE(uniquifier) \ |
---|
352 | extern struct sysinit SYSINIT_ENTRY_NAME(uniquifier); \ |
---|
353 | static struct sysinit const * const \ |
---|
354 | SYSINIT_REFERENCE_NAME(uniquifier) __used \ |
---|
355 | = &SYSINIT_ENTRY_NAME(uniquifier) |
---|
356 | #define SYSINIT_MODULE_REFERENCE(mod) \ |
---|
357 | SYSINIT_REFERENCE(mod ## module) |
---|
358 | #define SYSINIT_DRIVER_REFERENCE(driver, bus) \ |
---|
359 | SYSINIT_MODULE_REFERENCE(driver ## _ ## bus) |
---|
360 | #endif /* __rtems__ */ |
---|
361 | ---- |
---|
362 | |
---|
363 | Here you see that the SYSINIT(9) entries are no longer static. The |
---|
364 | *_REFERENCE() macros will create references to the corresponding modules which |
---|
365 | are later resolved by the linker. The application has to provide an object |
---|
366 | file with references to all required FreeBSD modules. |
---|
367 | |
---|
368 | The FreeBSD device model is quite elaborated (with follow-ups): |
---|
369 | |
---|
370 | http://www.freebsd.org/cgi/man.cgi?query=driver |
---|
371 | |
---|
372 | The devices form a tree with the Nexus device at a high-level. This Nexus |
---|
373 | device is architecture specific in FreeBSD. In RTEMS we have our own Nexus |
---|
374 | device, see "rtems-bsd-nexus.c". It uses a table to add child devices: |
---|
375 | |
---|
376 | [listing] |
---|
377 | ---- |
---|
378 | const char *const _bsd_nexus_devices [] = { |
---|
379 | #ifdef NEED_USB_OHCI |
---|
380 | "ohci", |
---|
381 | #endif |
---|
382 | #ifdef NEED_USB_EHCI |
---|
383 | "ehci", |
---|
384 | #endif |
---|
385 | #ifdef NEED_SDHC |
---|
386 | "sdhci", |
---|
387 | #endif |
---|
388 | NULL |
---|
389 | }; |
---|
390 | ---- |
---|
391 | |
---|
392 | This table must be provided by the application. |
---|
393 | |
---|
394 | === SYSCTL_NODE Example |
---|
395 | |
---|
396 | During development, we had an undefined reference to |
---|
397 | _bsd_sysctl__net_children that we had trouble tracking down. Thanks to |
---|
398 | Chris Johns, we located it. He explained how to read SYSCTL_NODE |
---|
399 | definitions. This line from freebsd/netinet/in_proto.c is attempting |
---|
400 | to add the "inet" node to the parent node "_net". |
---|
401 | |
---|
402 | [listing] |
---|
403 | ---- |
---|
404 | SYSCTL_NODE(_net, PF_INET, inet, CTLFLAG_RW, 0, |
---|
405 | "Internet Family"); |
---|
406 | ---- |
---|
407 | |
---|
408 | Our problem was that we could not find where _bsd_sysctl__net_children |
---|
409 | was defined. Chris suggested that when in doubt compile with -save-temps |
---|
410 | and look at the preprocessed .i files. But he did not need that. He |
---|
411 | explained that this the symbol name _bsd_sysctl__net_children was |
---|
412 | automatically generated by a SYSCTL_NODE as follows: |
---|
413 | |
---|
414 | * _bsd_ - added by RTEMS modifications to SYSCTL_NODE macro |
---|
415 | * sysctl_ - boilerplace added by SYSCTL_NODE macro |
---|
416 | * "" - empty string for parent node |
---|
417 | * net - name of SYSCTL_NODE |
---|
418 | * children - added by SYSCTL macros |
---|
419 | |
---|
420 | This was all generated by a support macro declaring the node as this: |
---|
421 | |
---|
422 | [listing] |
---|
423 | ---- |
---|
424 | struct sysctl_oid_list SYSCTL_NODE_CHILDREN(parent, name); |
---|
425 | ---- |
---|
426 | |
---|
427 | Given this information, we located this SYSCTL_NODE declaration in |
---|
428 | kern/kern_mib.c |
---|
429 | |
---|
430 | [listing] |
---|
431 | ---- |
---|
432 | SYSCTL_NODE(, CTL_KERN, kern, CTLFLAG_RW, 0, |
---|
433 | "High kernel, proc, limits &c"); |
---|
434 | ---- |
---|
435 | |
---|