source: rtems-libbsd/libbsd.txt @ e78b3dc

5-freebsd-12freebsd-9.3
Last change on this file since e78b3dc was e78b3dc, checked in by Sebastian Huber <sebastian.huber@…>, on Oct 1, 2015 at 7:47:05 AM

doc: Qemu network

  • Property mode set to 100644
File size: 40.3 KB
Line 
1RTEMS BSD Library Guide
2=======================
3:toc:
4:icons:
5:numbered:
6:website: http://www.rtems.org/
7
8RTEMS uses FreeBSD 9.2 as the source of its TCP/IP and USB stacks.
9This is a guide which captures information on the
10process of merging code from FreeBSD, building this library,
11RTEMS specific support files, and general guidelines on what
12modifications to the FreeBSD source are permitted.
13
14Goals of this effort are
15
16* update TCP/IP and provide USB in RTEMS,
17* ease updating to future FreeBSD versions,
18* ease tracking changes in FreeBSD code,
19* minimize manual changes in FreeBSD code, and
20* define stable kernel/device driver API which is implemented
21by both RTEMS and FreeBSD. This is the foundation of the port.
22
23We will work to push our changes upstream to the FreeBSD Project
24and minimize changes required at each update point.
25
26*******************************************************************************
27This is a work in progress and is very likely to be incomplete.
28Please help by adding to it.
29*******************************************************************************
30
31== Getting Started
32
33=== Tool Chain ===
34
35You need a tool chain for RTEMS based on at least
36
37* Binutils 2.24, and
38* Newlib 2.1.0.
39
40The Binutils version is required to ease the handling of linker command files.
41The Newlib version is required since some standard files like `<sys/types.h>`
42must be compatible enough for the files provided by the FreeBSD sources, e.g.
43`<sys/socket.h>`.
44
45=== Installation Overview ===
46
47. You must configure your BSP with the +--disable-networking+ option to disable
48the old network stack.  Make sure no header files of the old network stack are
49installed.
50. Clone the Git repository +git clone git://git.rtems.org/rtems-libbsd.git+.
51. Change into the RTEMS BSD library root directory.
52. Edit the `config.inc` Makefile configuration file and adjust it to your environment.
53. Run +make clean+.
54. Run +make install+.
55
56=== Board Support Package Requirements ===
57
58The RTEMS version must be at least 4.11.  The Board Support Package (BSP)
59should support the
60http://www.rtems.org/onlinedocs/doxygen/cpukit/html/group\__rtems\__interrupt__extension.html[Interrupt Manager Extension]
61// The first underscores have to be masked to stop asciidoc interpreting them
62to make use of generic FreeBSD based drivers.
63
64The linker command file of the BSP must contain the following section
65definitions:
66
67-------------------------------------------------------------------------------
68.rtemsroset : {
69        KEEP (*(SORT(.rtemsroset.*)))
70}
71
72.rtemsrwset : {
73        KEEP (*(SORT(.rtemsrwset.*)))
74}
75-------------------------------------------------------------------------------
76
77The first output section can be placed in read-only memory.  The second output
78section must be placed in read-write memory.  The output section name is not
79relevant.  The output sections may also contain other input sections.
80
81=== Board Support Package Configuration and Build ===
82
83You need to configure RTEMS for the desired BSP and install it.  The BSP should
84be configured with a disabled network stack.  The BSD library containing the
85new network stack is a separate package.  Using a BSP installation containing
86the old network stack may lead to confusion and unpredictable results.
87
88The following script is used to build the `arm/realview_pbx_a9_qemu` BSP for
89our internal testing purposes:
90
91-------------------------------------------------------------------------------
92#!/bin/sh
93
94cd ${HOME}/sandbox
95rm -rf b-realview_pbx_a9_qemu
96mkdir b-realview_pbx_a9_qemu
97cd b-realview_pbx_a9_qemu
98${HOME}/git-rtems/configure \
99        --prefix=${HOME}/sandbox/install \
100        --target=arm-rtems4.11 \
101        --enable-rtemsbsp=realview_pbx_a9_qemu \
102        --disable-networking && \
103        make && \
104        make install
105-------------------------------------------------------------------------------
106
107The `arm/realview_pbx_a9_qemu` BSP running on the Qemu simulator has some
108benefits for development and test of the BSD library
109
110* it offers a NULL pointer read and write protection,
111* Qemu is a fast simulator,
112* Qemu provides support for GDB watchpoints,
113* Qemu provides support for virtual Ethernet networks, e.g. TUN and bridge
114devices (you can run multiple test instances on one virtual network).
115
116=== BSD Library Configuration and Build ===
117
118There are currently 2 build systems supported. The first is based on the RTEMS
119Makefile support in RTEMS and installed with the BSP and the second is a stand
120alone environment based on the Waf build system. The Makefile build system will
121be removed when RTEMS moves away from its existing build system and Waf will
122be the preferred build environment.
123
124To build with Waf please refer to the README.waf file.
125
126===== Makefile Building =====
127
128In the BSD library source directory edit the file `config.inc`.  Continuing on
129the above, the `config.inc` used to match the above is:
130
131-------------------------------------------------------------------------------
132# Mandatory: Select your BSP and installation prefix
133TARGET = arm-rtems4.11
134BSP = realview_pbx_a9_qemu
135PREFIX = $(HOME)/sandbox/install
136
137# Optional: Separate installation base directory
138INSTALL_BASE = $(PREFIX)/$(TARGET)/$(BSP)
139
140# Optional: Network test configuration
141TEST_RUNNER = $(BSP)
142NET_CFG_SELF_IP = 10.0.0.2
143NET_CFG_NETMASK = 255.255.0.0
144NET_CFG_PEER_IP = 10.0.0.1
145NET_CFG_GATEWAY_IP = 10.0.0.1
146NET_TAP_INTERFACE = tap0
147-------------------------------------------------------------------------------
148
149Now you can build the BSD library and run the tests:
150
151-------------------------------------------------------------------------------
152make clean
153make
154make run_tests
155-------------------------------------------------------------------------------
156
157You can only run the tests directly in case a test runner is available.  The
158following tests run without an external network.  It is strongly advised to run
159them.
160
161* commands01
162* init01
163* loopback01
164* rwlock01
165* selectpollkqueue01
166* sleep01
167* swi01
168* syscalls01
169* thread01
170* timeout01
171* unix01
172
173To install the BSD library use this:
174
175-------------------------------------------------------------------------------
176make install
177-------------------------------------------------------------------------------
178
179=== BSD Library Initialization ===
180
181Use the following code to initialize the BSD library:
182
183-------------------------------------------------------------------------------
184#include <assert.h>
185#include <sysexits.h>
186
187#include <machine/rtems-bsd-commands.h>
188#include <rtems/bsd/bsd.h>
189
190static void
191network_ifconfig_lo0(void)
192{
193        int exit_code;
194        char *lo0[] = {
195                "ifconfig",
196                "lo0",
197                "inet",
198                "127.0.0.1",
199                "netmask",
200                "255.255.255.0",
201                NULL
202        };
203        char *lo0_inet6[] = {
204                "ifconfig",
205                "lo0",
206                "inet6",
207                "::1",
208                "prefixlen",
209                "128",
210                NULL
211        };
212
213        exit_code = rtems_bsd_command_ifconfig(RTEMS_BSD_ARGC(lo0), lo0);
214        assert(exit_code == EX_OK);
215
216        exit_code = rtems_bsd_command_ifconfig(RTEMS_BSD_ARGC(lo0_inet6), lo0_inet6);
217        assert(exit_code == EX_OK);
218}
219
220void
221network_init(void)
222{
223        rtems_status_code sc;
224
225        sc = rtems_bsd_initialize();
226        assert(sc == RTEMS_SUCCESSFUL);
227
228        network_ifconfig_lo0();
229}
230-------------------------------------------------------------------------------
231
232This performs the basic network stack initialization with a loopback interface.
233Further initialization must be done using the standard BSD network
234configuration commands
235http://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[IFCONFIG(8)]
236using `rtems_bsd_command_ifconfig()` and
237http://www.freebsd.org/cgi/man.cgi?query=route&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[ROUTE(8)]
238using `rtems_bsd_command_route()`.  For an example please have a look at
239`testsuite/include/rtems/bsd/test/default-network-init.h`.
240
241=== Task Priorities and Stack Size ===
242
243The default task priority is 96 for the interrupt server task (name "IRQS"), 98
244for the timer server task (name "TIME") and 100 for all other tasks.  The
245application may provide their own implementation of the
246`rtems_bsd_get_task_priority()` function (for example in the module which calls
247`rtems_bsd_initialize()`) if different values are desired.
248
249The task stack size is determined by the `rtems_bsd_get_task_stack_size()`
250function which may be provided by the application in case the default is not
251appropriate.
252
253=== Size for Allocator Domains ===
254
255The size for an allocator domain can be specified via the
256`rtems_bsd_get_allocator_domain_size()` function.  The application may provide
257their own implementation of the `rtems_bsd_get_allocator_domain_size()`
258function (for example in the module which calls `rtems_bsd_initialize()`) if
259different values are desired.  The default size is 8MiB for all domains.
260
261== Network Stack Features
262
263http://roy.marples.name/projects/dhcpcd/index[DHCPCD(8)]:: DHCP client
264
265https://developer.apple.com/library/mac/documentation/Networking/Reference/DNSServiceDiscovery_CRef/Reference/reference.html[dns_sd.h]:: DNS Service Discovery
266
267http://www.opensource.apple.com/source/mDNSResponder/mDNSResponder-320.10/mDNSCore/mDNSEmbeddedAPI.h[mDNS]:: Multi-Cast DNS
268
269http://www.freebsd.org/cgi/man.cgi?query=unix&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[UNIX(4)]:: UNIX-domain protocol family
270
271http://www.freebsd.org/cgi/man.cgi?query=inet&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[INET(4)]:: Internet protocol family
272
273http://www.freebsd.org/cgi/man.cgi?query=inet6&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[INET6(4)]:: Internet protocol version 6 family
274
275http://www.freebsd.org/cgi/man.cgi?query=tcp&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[TCP(4)]:: Internet Transmission Control Protocol
276
277http://www.freebsd.org/cgi/man.cgi?query=udp&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[UDP(4)]:: Internet User Datagram Protocol
278
279http://www.freebsd.org/cgi/man.cgi?query=route&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[ROUTE(4)]:: Kernel packet forwarding database
280
281http://www.freebsd.org/cgi/man.cgi?query=bpf&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[BPF(4)]:: Berkeley Packet Filter
282
283http://www.freebsd.org/cgi/man.cgi?query=socket&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[SOCKET(2)]:: Create an endpoint for communication
284
285http://www.freebsd.org/cgi/man.cgi?query=kqueue&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[KQUEUE(2)]:: Kernel event notification mechanism
286
287http://www.freebsd.org/cgi/man.cgi?query=select&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[SELECT(2)]:: Synchronous I/O multiplexing
288
289http://www.freebsd.org/cgi/man.cgi?query=poll&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[POLL(2)]:: Synchronous I/O multiplexing
290
291http://www.freebsd.org/cgi/man.cgi?query=route&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[ROUTE(8)]:: Manually manipulate the routing tables
292
293http://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[IFCONFIG(8)]:: Configure network interface parameters
294
295http://www.freebsd.org/cgi/man.cgi?query=netstat&apropos=0&sektion=1&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[NETSTAT(1)]:: Show network status
296
297http://www.freebsd.org/cgi/man.cgi?query=ping&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[PING(8)]:: Send ICMP ECHO_REQUEST packets to network hosts
298
299http://www.freebsd.org/cgi/man.cgi?query=ping6&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[PING6(8)]:: Send ICMPv6 ECHO_REQUEST packets to network hosts
300
301http://www.freebsd.org/cgi/man.cgi?query=sysctl&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[SYSCTL(3)]:: Get or set system information
302
303http://www.freebsd.org/cgi/man.cgi?query=resolver&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[RESOLVER(3)]:: Resolver routines
304
305http://www.freebsd.org/cgi/man.cgi?query=gethostbyname&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[GETHOSTBYNAME(3)]:: Get network host entry
306
307== Network Interface Drivers
308
309=== Link Up/Down Events
310
311You can notifiy the application space of link up/down events in your network
312interface driver via the if_link_state_change(LINK_STATE_UP/LINK_STATE_DOWN)
313function.  The DHCPCD(8) client is a consumer of these events for example.
314Make sure that the interface flag IFF_UP and the interface driver flag
315IFF_DRV_RUNNING is set in case the link is up, otherwise ether_output() will
316return the error status ENETDOWN.
317
318== Shell Commands
319
320=== HOSTNAME(1)
321
322In addition to the standard options the RTEMS version of the HOSTNAME(1)
323command supports the -m flag to set/get the multicast hostname of the
324mDNS resolver instance.  See also rtems_mdns_sethostname() and
325rtems_mdns_gethostname().
326
327== Qemu
328
329Use the following script to set up a virtual network with three tap devices
330connected via one bridge device.
331
332-------------------------------------------------------------------------------
333#!/bin/sh -x
334
335user=`whoami`
336interfaces=(1 2 3)
337
338tap=qtap
339bri=qbri
340
341case $1 in
342        up)
343                sudo -i brctl addbr $bri
344                for i in ${interfaces[@]} ; do
345                        sudo -i tunctl -t $tap$i -u $user ;
346                        sudo -i ifconfig $tap$i up ;
347                        sudo -i brctl addif $bri $tap$i ;
348                done
349                sudo -i ifconfig $bri up
350                ;;
351        down)
352                for i in ${interfaces[@]} ; do
353                        sudo -i ifconfig $tap$i down ;
354                        sudo -i tunctl -d $tap$i ;
355                done
356                sudo -i ifconfig $bri down
357                sudo -i brctl delbr $bri
358                ;;
359esac
360-------------------------------------------------------------------------------
361
362Connect your Qemu instance to one of the tap devices, e.g.
363
364-------------------------------------------------------------------------------
365qemu-system-i386 -m 512 -boot a -cpu pentium3 \
366        -drive file=$HOME/qemu/pc386_fda,index=0,if=floppy,format=raw \
367        -drive file=fat:$HOME/qemu/hd,format=raw \
368        -net nic,model=e1000,macaddr=0e:b0:ba:5e:ba:11 \
369        -net tap,ifname=qtap1,script=no,downscript=no \
370        -nodefaults -nographic -serial stdio
371-------------------------------------------------------------------------------
372
373Make sure that each Qemu instance uses its own MAC address to avoid an address
374conflict (or otherwise use it as a test).
375
376To connect the Qemu instances with your local network use the following
377(replace 'eth0' with the network interface of your host).
378
379-------------------------------------------------------------------------------
380ifconfig eth0 0.0.0.0
381brctl addif qbri eth0
382dhclient qbri
383-------------------------------------------------------------------------------
384
385== Issues and TODO
386
387* PCI support on x86 uses a quick and dirty hack, see pci_reserve_map().
388
389* Priority queues are broken with clustered scheduling.
390
391* Per-CPU data should be enabled once the new stack is ready for SMP.
392
393* Per-CPU NETISR(9) should be enabled onece the new stack is ready for SMP.
394
395* Multiple routing tables are not supported.  Every FIB value is set to zero
396  (= BSD_DEFAULT_FIB).
397
398* Process identifiers are not supported.  Every PID value is set to zero
399  (= BSD_DEFAULT_PID).
400
401* User credentials are not supported.  The following functions allow the
402  operation for everyone
403  - prison_equal_ip4(),
404  - chgsbsize(),
405  - cr_cansee(),
406  - cr_canseesocket() and
407  - cr_canseeinpcb().
408
409* A basic USB functionality test that is known to work on Qemu is desirable.
410
411* Adapt generic IRQ PIC interface code to Simple Vectored Interrupt Model
412  so that those architectures can use new TCP/IP and USB code.
413
414* freebsd-userspace/rtems/include/sys/syslog.h is a copy from the old
415  RTEMS TCP/IP stack. For some reason, the __printflike markers do not
416  compile in this environment. We may want to use the FreeBSD syslog.h
417  and get this addressed.
418
419* in_cksum implementations for architectures not supported by FreeBSD.
420  This will require figuring out where to put implementations that do
421  not originate from FreeBSD and are populated via the script.
422
423* MAC support functions are not thread-safe ("freebsd/lib/libc/posix1e/mac.c").
424
425* IFCONFIG(8): IEEE80211 support is disabled.  This module depends on a XML
426  parser and mmap().
427
428* get_cyclecount(): The implementation is a security problem.
429
430* What to do with the priority parameter present in the FreeBSD synchronization
431  primitives and the thread creation functions?
432
433* TASKQUEUE(9): Support spin mutexes.
434
435* ZONE(9): Review allocator lock usage in rtems-bsd-chunk.c.
436
437* KQUEUE(2): Choose proper lock for global kqueue list.
438
439* TIMEOUT(9): Maybe use special task instead of timer server to call
440  callout_tick().
441
442* sysctl_handle_opaque(): Implement reliable snapshots.
443
444* PING6(8): What to do with SIGALARM?
445
446* <sys/param.h>: Update Newlib to use a MSIZE of 256.
447
448* BPF(4): Add support for zero-copy buffers.
449
450* UNIX(4): Fix race conditions in the area of socket object and file node
451  destruction.  Add support for file descriptor transmission via control
452  messages.
453
454* PRINTF(9): Add support for log(), the %D format specifier is missing in the
455  normal printf() family.
456
457* Why is the interrupt server used?  The BSD interrupt handlers can block on
458synchronization primitives like mutexes.  This is in contrast to RTEMS
459interrupt service routines.  The BSPs using the generic interrupt support must
460implement the `bsp_interrupt_vector_enable()` and
461`bsp_interrupt_vector_disable()` routines.  They normally enable/disable a
462particular interrupt source at the interrupt controller.  This can be used to
463implement the interrupt server.  The interrupt server is a task that wakes-up
464in case an associated interrupt happens.  The interrupt source is disabled in
465a generic interrupt handler that wakes-up the interrupt server task.   Once the
466postponed interrupt processing is performed in the interrupt server the
467interrupt source is enabled again.
468
469* Convert all BSP linkcmds to use a linkcmds.base so the sections are
470easier to insert.
471
472* NIC Device Drivers
473- Only common PCI NIC drivers have been included in the initial set. These
474do not include any system on chip or ISA drivers.
475- PCI configuration probe does not appear to happen to determine if a
476NIC is in I/O or memory space. We have worked around this by using a
477static hint to tell the fxp driver the correct mode. But this needs to
478be addressed.
479- The ISA drivers require more BSD infrastructure to be addressed. This was
480outside the scope of the initial porting effort.
481
482== FreeBSD Source
483
484You should be able to rely on FreebSD manual pages and documentation
485for details on the code itself.
486
487=== Automatically Generated FreeBSD Files
488
489Some source and header files are automatically generated during the FreeBSD
490build process.  The `Makefile.todo` file performs this manually.  The should be
491included in `freebsd-to-rtems.py` script some time in the future.  For details,
492see also
493http://www.freebsd.org/cgi/man.cgi?query=kobj&sektion=9&apropos=0&manpath=FreeBSD+9.2-RELEASE[KOBJ(9)].
494
495=== Rules for Modifying FreeBSD Source
496
497Only add lines.  If your patch contains lines starting with a '-', then this is
498wrong.  Subtract code by added `#ifndef __rtems__`.  This makes merging easier
499in the future.  For example:
500
501-------------------------------------------------------------------------------
502/* Global variables for the kernel. */
503
504#ifndef __rtems__
505/* 1.1 */
506extern char kernelname[MAXPATHLEN];
507#endif /* __rtems__ */
508
509extern int tick;                        /* usec per tick (1000000 / hz) */
510-------------------------------------------------------------------------------
511
512-------------------------------------------------------------------------------
513#if defined(_KERNEL) || defined(_WANT_FILE)
514#ifdef __rtems__
515#include <rtems/libio_.h>
516#include <sys/fcntl.h>
517#endif /* __rtems__ */
518/*
519 * Kernel descriptor table.
520 * One entry for each open kernel vnode and socket.
521 *
522 * Below is the list of locks that protects members in struct file.
523 *
524 * (f) protected with mtx_lock(mtx_pool_find(fp))
525 * (d) cdevpriv_mtx
526 * none not locked
527 */
528-------------------------------------------------------------------------------
529
530-------------------------------------------------------------------------------
531extern int profprocs;                   /* number of process's profiling */
532#ifndef __rtems__
533extern volatile int ticks;
534#else /* __rtems__ */
535#include <rtems/score/watchdogimpl.h>
536#define ticks _Watchdog_Ticks_since_boot
537#endif /* __rtems__ */
538
539#endif /* _KERNEL */
540-------------------------------------------------------------------------------
541
542Add nothing (even blank lines) before or after the `__rtems__` guards.  Always
543include a `__rtems__` in the guards to make searches easy, so use
544
545* `#ifndef __rtems__`,
546* `#ifdef __rtems__`,
547* `#else /* __rtems__ */`, and
548* `#endif /* __rtems__ */`.
549
550For new code use
551http://www.freebsd.org/cgi/man.cgi?query=style&apropos=0&sektion=9&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[STYLE(9)].
552Do not format original FreeBSD code.
553
554== BSD Library Source
555
556=== What is in the Git Repository
557
558There is a self-contained kit with FreeBSD and RTEMS components pre-merged. The
559Makefile in this kit is automatically generated.
560
561Any changes to source in the `freebsd` directories will need to be merged
562upstream into our master FreeBSD checkout, the `freebsd-org` submodule.
563
564The repository contains two FreeBSD source trees.  In the `freebsd` directory
565are the so called 'managed' FreeBSD sources used to build the BSD library.  The
566FreeBSD source in `freebsd-org` is the 'master' version.  The
567`freebsd-to-rtems.py` script is used to transfer files between the two trees.
568In general terms, if you have modified managed FreeBSD sources, you will need
569to run the script in 'revert' or 'reverse' mode using the `-R` switch.  This
570will copy the source back to your local copy of the master FreeBSD source so
571you can run `git diff` against the upstream FreeBSD source.  If you want to
572transfer source files from the master FreeBSD source to the manged FreeBSD
573sources, then you must run the script in 'forward' mode (the default).
574
575=== Organization
576
577The top level directory contains a few directories and files. The following
578are important to understand
579
580* `freebsd-to-rtems.py` - script to convert to and free FreeBSD and RTEMS trees,
581* `Makefile` - automatically generated,
582* `freebsd/` - from FreeBSD by script,
583* `rtemsbsd/` - RTEMS specific implementations of FreeBSD kernel support routines,
584* `testsuite/` - RTEMS specific tests, and
585* `libbsd.txt` - documentation in Asciidoc.
586
587== Moving Code Between Managed and Master FreeBSD Source
588
589The script `freebsd-to-rtems.py` is used to copy code from FreeBSD to the
590rtems-libbsd tree and to reverse this process. This script attempts to
591automate this process as much as possible and performs some transformations
592on the FreeBSD code. Its command line arguments are shown below:
593
594----
595freebsd-to-rtems.py [args]
596  -?|-h|--help     print this and exit
597  -d|--dry-run     run program but no modifications
598  -D|--diff        provide diff of files between trees
599  -e|--early-exit  evaluate arguments, print results, and exit
600  -m|--makefile    just generate Makefile
601  -R|--reverse     default FreeBSD -> RTEMS, reverse that
602  -r|--rtems       RTEMS directory
603  -f|--freebsd     FreeBSD directory
604  -v|--verbose     enable verbose output mode
605----
606
607In its default mode of operation, freebsd-to-rtems.py is used to copy code
608from FreeBSD to the rtems-libbsd tree and perform transformations.  In forward
609mode, the script may be requested to just generate the Makefile and Waf script.
610
611In "reverse mode", this script undoes those transformations and copies
612the source code back to the FreeBSD SVN tree. This allows us to do
613'svn diff', evaluate changes made by the RTEMS Project, and report changes
614back to FreeBSD upstream.
615
616In either mode, the script may be asked to perform a dry-run or be verbose.
617Also, in either mode, the script is also smart enough to avoid copying over
618files which have not changed. This means that the timestamps of files are
619not changed unless the contents change. The script will also report the
620number of files which changed. In verbose mode, the script will print
621the name of the files which are changed.
622
623The following is an example forward run with no changes.
624
625----
626$ ~/newbsd/git/libbsd-8.2/freebsd-to-rtems.py \
627    -r /home/joel/newbsd/git/libbsd-8.2 \
628    -f /home/joel/newbsd/libbsd/freebsd-8.2 -v
629Verbose:                yes
630Dry Run:                no
631Only Generate Makefile: no
632RTEMS Directory:        /home/joel/newbsd/git/libbsd-8.2
633FreeBSD Directory:      /home/joel/newbsd/libbsd/freebsd-8.2
634Direction:              forward
635Generating into /home/joel/newbsd/git/libbsd-8.2
6360 files were changed.
637----
638
639The script may also be used to generate a diff in either forward or reverse
640direction.
641
642== Initialization of the BSD Library
643
644The initialization of the BSD library is based on the FreeBSD SYSINIT(9)
645infrastructure.  The key to initializing a system is to ensure that the desired
646device drivers are explicitly pulled into the linked application.  This plus
647linking against the BSD library (`libbsd.a`) will pull in the necessary FreeBSD
648infrastructure.
649
650The FreeBSD kernel is not a library like the RTEMS kernel.  It is a bunch of
651object files linked together.  If we have a library, then creating the
652executable is simple.  We begin with a start symbol and recursively resolve all
653references.  With a bunch of object files linked together we need a different
654mechanism.  Most object files don't know each other.  Lets say we have a driver
655module.  The rest of the system has no references to this driver module.  The
656driver module needs a way to tell the rest of the system: Hey, kernel I am
657here, please use my services!
658
659This registration of independent components is performed by SYSINIT(9) and
660specializations:
661
662http://www.freebsd.org/cgi/man.cgi?query=SYSINIT
663
664The SYSINIT(9) uses some global data structures that are placed in a certain
665section.  In the linker command file we need this:
666
667-------------------------------------------------------------------------------
668.rtemsroset : {
669        KEEP (*(SORT(.rtemsroset.*)))
670}
671
672.rtemsrwset : {
673        KEEP (*(SORT(.rtemsrwset.*)))
674}
675-------------------------------------------------------------------------------
676
677This results for example in this executable layout:
678
679-------------------------------------------------------------------------------
680[...]
681 *(SORT(.rtemsroset.*))
682 .rtemsroset.bsd.modmetadata_set.begin
683                0x000000000025fe00        0x0 libbsd.a(rtems-bsd-init.o)
684                0x000000000025fe00                _bsd__start_set_modmetadata_set
685 .rtemsroset.bsd.modmetadata_set.content
686                0x000000000025fe00        0x8 libbsd.a(rtems-bsd-nexus.o)
687 .rtemsroset.bsd.modmetadata_set.content
688                0x000000000025fe08        0x4 libbsd.a(kern_module.o)
689[...]
690 .rtemsroset.bsd.modmetadata_set.content
691                0x000000000025fe68        0x4 libbsd.a(mii.o)
692 .rtemsroset.bsd.modmetadata_set.content
693                0x000000000025fe6c        0x4 libbsd.a(mii_bitbang.o)
694 .rtemsroset.bsd.modmetadata_set.end
695                0x000000000025fe70        0x0 libbsd.a(rtems-bsd-init.o)
696                0x000000000025fe70                _bsd__stop_set_modmetadata_set
697[...]
698.rtemsrwset     0x000000000030bad0      0x290
699 *(SORT(.rtemsrwset.*))
700 .rtemsrwset.bsd.sysinit_set.begin
701                0x000000000030bad0        0x0 libbsd.a(rtems-bsd-init.o)
702                0x000000000030bad0                _bsd__start_set_sysinit_set
703 .rtemsrwset.bsd.sysinit_set.content
704                0x000000000030bad0        0x4 libbsd.a(rtems-bsd-nexus.o)
705 .rtemsrwset.bsd.sysinit_set.content
706                0x000000000030bad4        0x8 libbsd.a(rtems-bsd-thread.o)
707 .rtemsrwset.bsd.sysinit_set.content
708                0x000000000030badc        0x4 libbsd.a(init_main.o)
709[...]
710 .rtemsrwset.bsd.sysinit_set.content
711                0x000000000030bd54        0x4 libbsd.a(frag6.o)
712 .rtemsrwset.bsd.sysinit_set.content
713                0x000000000030bd58        0x8 libbsd.a(uipc_accf.o)
714 .rtemsrwset.bsd.sysinit_set.end
715                0x000000000030bd60        0x0 libbsd.a(rtems-bsd-init.o)
716                0x000000000030bd60                _bsd__stop_set_sysinit_set
717[...]
718-------------------------------------------------------------------------------
719
720Here you can see, that some global data structures are collected into
721continuous memory areas.  This memory area can be identified by start and stop
722symbols.  This constructs a table of uniform items.
723
724The low level FreeBSD code calls at some time during the initialization the
725mi_startup() function (machine independent startup).  This function will sort
726the SYSINIT(9) set and call handler functions which perform further
727initialization.  The last step is the scheduler invocation.
728
729The SYSINIT(9) routines are run in mi_startup() which is called by
730rtems_bsd_initialize().
731
732This is also explained in "The Design and Implementation of the FreeBSD
733Operating System" section 14.3 "Kernel Initialization".
734
735In RTEMS we have a library and not a bunch of object files.  Thus we need a way
736to pull-in the desired services out of the libbsd.  Here the
737`rtems-bsd-sysinit.h` comes into play.  The SYSINIT(9) macros have been
738modified and extended for RTEMS in `<sys/kernel.h>`:
739
740-------------------------------------------------------------------------------
741#ifndef __rtems__
742#define C_SYSINIT(uniquifier, subsystem, order, func, ident)    \
743        static struct sysinit uniquifier ## _sys_init = {       \
744                subsystem,                                      \
745                order,                                          \
746                func,                                           \
747                (ident)                                         \
748        };                                                      \
749        DATA_SET(sysinit_set,uniquifier ## _sys_init)
750#else /* __rtems__ */
751#define SYSINIT_ENTRY_NAME(uniquifier)                          \
752        _bsd_ ## uniquifier ## _sys_init
753#define SYSINIT_REFERENCE_NAME(uniquifier)                      \
754        _bsd_ ## uniquifier ## _sys_init_ref
755#define C_SYSINIT(uniquifier, subsystem, order, func, ident)    \
756        struct sysinit SYSINIT_ENTRY_NAME(uniquifier) = {       \
757                subsystem,                                      \
758                order,                                          \
759                func,                                           \
760                (ident)                                         \
761        };                                                      \
762        RWDATA_SET(sysinit_set,SYSINIT_ENTRY_NAME(uniquifier))
763#define SYSINIT_REFERENCE(uniquifier)                           \
764        extern struct sysinit SYSINIT_ENTRY_NAME(uniquifier);   \
765        static struct sysinit const * const                     \
766        SYSINIT_REFERENCE_NAME(uniquifier) __used               \
767        = &SYSINIT_ENTRY_NAME(uniquifier)
768#define SYSINIT_MODULE_REFERENCE(mod)                           \
769        SYSINIT_REFERENCE(mod ## module)
770#define SYSINIT_DRIVER_REFERENCE(driver, bus)                   \
771        SYSINIT_MODULE_REFERENCE(driver ## _ ## bus)
772#define SYSINIT_DOMAIN_REFERENCE(dom)                           \
773        SYSINIT_REFERENCE(domain_add_ ## dom)
774#endif /* __rtems__ */
775-------------------------------------------------------------------------------
776
777Here you see that the SYSINIT(9) entries are no longer static.  The
778\*_REFERENCE() macros will create references to the corresponding modules which
779are later resolved by the linker.  The application has to provide an object
780file with references to all required FreeBSD modules.
781
782The FreeBSD device model is quite elaborated (with follow-ups):
783
784http://www.freebsd.org/cgi/man.cgi?query=driver
785
786The devices form a tree with the Nexus device at a high-level.  This Nexus
787device is architecture specific in FreeBSD.  In RTEMS we have our own Nexus
788device, see `rtemsbsd/bsp/bsp-bsd-nexus-devices.c`.
789
790=== SYSCTL_NODE Example
791
792During development, we had an undefined reference to
793_bsd_sysctl__net_children that we had trouble tracking down. Thanks to
794Chris Johns, we located it. He explained how to read SYSCTL_NODE
795definitions. This line from freebsd/netinet/in_proto.c is attempting
796to add the "inet" node to the parent node "_net".
797
798----
799SYSCTL_NODE(_net,      PF_INET,         inet,   CTLFLAG_RW, 0,
800        "Internet Family");
801----
802
803Our problem was that we could not find where _bsd_sysctl__net_children
804was defined. Chris suggested that when in doubt compile with -save-temps
805and look at the preprocessed .i files. But he did not need that. He
806explained that this the symbol name _bsd_sysctl__net_children was
807automatically generated by a SYSCTL_NODE as follows:
808
809* _bsd_ - added by RTEMS modifications to SYSCTL_NODE macro
810* sysctl_ - boilerplace added by SYSCTL_NODE macro
811* "" - empty string for parent node
812* net - name of SYSCTL_NODE
813* children - added by SYSCTL macros
814
815This was all generated by a support macro declaring the node as this:
816
817----
818struct sysctl_oid_list SYSCTL_NODE_CHILDREN(parent, name);
819----
820
821Given this information, we located this SYSCTL_NODE declaration in
822kern/kern_mib.c
823
824----
825SYSCTL_NODE(, CTL_KERN,   kern,   CTLFLAG_RW, 0,
826        "High kernel, proc, limits &c");
827----
828
829== Core FreeBSD APIs and RTEMS Replacements ==
830
831=== SX(9) (Shared/exclusive locks) ===
832
833http://www.freebsd.org/cgi/man.cgi?query=sx
834
835Binary semaphores (this neglects the ability to allow shared access).
836
837=== MUTEX(9) (Mutual exclusion) ===
838
839http://www.freebsd.org/cgi/man.cgi?query=mutex
840
841Binary semaphores (not recursive mutexes are not supported this way).
842
843=== RWLOCK(9) (Reader/writer lock) ===
844
845http://www.freebsd.org/cgi/man.cgi?query=rwlock
846
847POSIX r/w lock.
848
849=== RMLOCK(9) (Reader/writer lock optimized for mostly read access patterns) ===
850
851Note:  This object was implemented as a wrapper for RWLOCK in the rm_lock header file.
852
853http://www.freebsd.org/cgi/man.cgi?query=rmlock
854
855POSIX r/w lock.
856
857=== CONDVAR(9) (Condition variables) ===
858
859http://www.freebsd.org/cgi/man.cgi?query=condvar
860
861POSIX condition variables with modifications (hack).
862
863=== CALLOUT(9) (Timer functions) ===
864
865http://www.freebsd.org/cgi/man.cgi?query=callout
866
867Timer server.
868
869=== TASKQUEUE(9) (Asynchronous task execution) ===
870
871http://www.freebsd.org/cgi/man.cgi?query=taskqueue
872
873TBD.
874
875=== KTHREAD(9), KPROC(9) (Tasks) ===
876
877http://www.freebsd.org/cgi/man.cgi?query=kthread
878
879http://www.freebsd.org/cgi/man.cgi?query=kproc
880
881Tasks.
882
883=== ZONE(9) (Zone allocator) ===
884
885http://www.freebsd.org/cgi/man.cgi?query=zone
886
887TBD.
888
889=== devfs (Device file system) ===
890
891Dummy, IMFS or new implementation (currently dummy).
892
893=== psignal (Signals) ===
894
895TBD.  Seems to be not needed.
896
897=== poll, select ===
898
899TBD.  Seems to be not needed.
900
901=== RMAN(9) (Resource management) ===
902
903http://www.freebsd.org/cgi/man.cgi?query=rman
904
905TBD.  Seems to be not needed.
906
907=== DEVCLASS(9), DEVICE(9), DRIVER(9), MAKE_DEV(9) (Device management) ===
908
909http://www.freebsd.org/cgi/man.cgi?query=devclass
910
911http://www.freebsd.org/cgi/man.cgi?query=device
912
913http://www.freebsd.org/cgi/man.cgi?query=driver
914
915http://www.freebsd.org/cgi/man.cgi?query=make_dev
916
917Use FreeBSD implementation as far as possible.  FreeBSD has a nice API for
918dynamic device handling.  It may be interesting for RTEMS to use this API
919internally in the future.
920
921=== BUS_SPACE(9), BUS_DMA(9) (Bus and DMA access) ===
922
923http://www.freebsd.org/cgi/man.cgi?query=bus_space
924
925http://www.freebsd.org/cgi/man.cgi?query=bus_dma
926
927Likely BSP dependent.  A default implementation for memory mapped linear access
928is easy to provide.  The current heap implementation supports all properties
929demanded by bus_dma (including the boundary constraint).
930
931== RTEMS Replacements by File Description ==
932
933Note:  Files with a status of USB are used by the USB test and have at least
934been partially tested.  If they contain both USB and Nic, then they are used
935by both and MAY contain methods that have not been tested yet.  Files that
936are only used by the Nic test are the most suspect.
937
938----
939rtems-libbsd File:      rtems-bsd-assert.c
940FreeBSD File:           rtems-bsd-config.h redefines BSD_ASSERT.
941Description:            This file contains the support method rtems_bsd_assert_func().
942Status:                 USB, Nic
943
944rtems-libbsd File:      rtems-bsd-autoconf.c
945FreeBSD File:           FreeBSD has BSP specific autoconf.c
946Description:            This file contains configuration methods that are used to setup the system.
947Status:                 USB
948
949rtems-libbsd File:      rtems-bsd-bus-dma.c
950FreeBSD File:           FreeBSD has BSP specific busdma_machdep.c
951Description:
952Status:                 USB, Nic
953
954rtems-libbsd File:      rtems-bsd-bus-dma-mbuf.c
955FreeBSD File:           FreeBSD has BSP specific busdma_machdep.c
956Description:
957Status:                 Nic
958
959rtems-libbsd File:      rtems-bsd-callout.c
960FreeBSD File:           kern/kern_timeout.c
961Description:
962Status:                 USB, Nic
963
964rtems-libbsd File:      rtems-bsd-cam.c
965FreeBSD File:           cam/cam_sim.c
966Description:
967Status:                 USB
968
969rtems-libbsd File:      rtems-bsd-condvar.c
970FreeBSD File:           kern/kern_condvar.c
971Description:
972Status:                 USB
973
974rtems-libbsd File:      rtems-bsd-copyinout.c
975FreeBSD File:           bsp specific copyinout.c )
976Description:            Note: The FreeBSD file is split with some methods being in rtems-bsd-support
977Status:                 Nic
978
979rtems-libbsd File:      rtems-bsd-delay.c
980FreeBSD File:           bsp specific file with multiple names
981Description:
982Status:                 USB, Nic
983
984rtems-libbsd File:      rtems-bsd-descrip.c
985FreeBSD File:           kern/kern_descrip.c
986Description:
987Status:                 Nic
988
989rtems-libbsd File:      rtems-bsd-generic.c
990FreeBSD File:           kern/sys_generic.c
991Description:
992Status:                 Nic
993
994rtems-libbsd File:      rtems-bsd-init.c
995FreeBSD File:           N/A
996Description:
997Status:                 USB, Nic
998
999rtems-libbsd File:      rtems-bsd-init-with-irq.c
1000FreeBSD File:           N/A
1001Description:
1002Status:                 USB, Nic
1003
1004rtems-libbsd File:      rtems-bsd-jail.c
1005FreeBSD File:           kern/kern_jail.c
1006Description:
1007Status:                 USB, Nic
1008
1009rtems-libbsd File:      rtems-bsd-lock.c
1010FreeBSD File:           kern/subr_lock.c
1011Description:
1012Status:                 USB, Nic
1013
1014rtems-libbsd File:      rtems-bsd-log.c
1015FreeBSD File:           kern/subr_prf.c
1016Description:
1017Status:                 Nic
1018
1019rtems-libbsd File:      rtems-bsd-malloc.c
1020FreeBSD File:           kern/kern_malloc.c
1021Description:
1022Status:                 USB, Nic
1023
1024rtems-libbsd File:      rtems-bsd-mutex.c
1025FreeBSD File:           kern/kern_mutex.c
1026Description:
1027Status:                 USB, Nic
1028
1029rtems-libbsd File:      rtems-bsd-newproc.c
1030FreeBSD File:           N/A
1031Description:
1032Status:                 Nic
1033
1034rtems-libbsd File:      rtems-bsd-nexus.c
1035FreeBSD File:           bsp specific nexus.c
1036Description:
1037Status:                 USB
1038
1039rtems-libbsd File:      rtems-bsd-panic.c
1040FreeBSD File:           boot/common/panic.c
1041Description:
1042Status:                 USB, Nic
1043
1044rtems-libbsd File:      rtems-bsd-rwlock.c
1045FreeBSD File:           kern_rwlock.c
1046Description:
1047Status:                 USB, Nic
1048
1049rtems-libbsd File:      rtems-bsd-shell.c
1050FreeBSD File:           N/A
1051Description:
1052Status:                 USB
1053
1054rtems-libbsd File:      rtems-bsd-signal.c
1055FreeBSD File:           kern/kern_sig.c
1056Description:
1057Status:                 Nic
1058
1059rtems-libbsd File:      rtems-bsd-smp.c
1060FreeBSD File:           N/A
1061Description:
1062Status:                 Nic
1063
1064rtems-libbsd File:      rtems-bsd-support.c
1065FreeBSD File:           bsp specific copyinout.c
1066Description:            Note: the FreeBSD file is split with some methods being in rtems-bsd-copyinout.
1067Status:                 USB, Nic
1068
1069rtems-libbsd File:      rtems-bsd-sx.c
1070FreeBSD File:           kern/kern_sx.c
1071Description:            Status: USB, Nic
1072
1073rtems-libbsd File:      rtems-bsd-synch.c
1074FreeBSD File:           kern/kern_synch.c
1075Description:
1076Status:                 USB, Nic
1077
1078rtems-libbsd File:      rtems-bsd-syscalls.c
1079FreeBSD File:           User API for kern/uipc_syscalls.c
1080Description:
1081Status:                 Nic
1082
1083rtems-libbsd File:      rtems-bsd-sysctlbyname.c
1084FreeBSD File:           User API for sysctlbyname(3)
1085Description:
1086Status:
1087
1088rtems-libbsd File:      rtems-bsd-sysctl.c
1089FreeBSD File:           User API for sysctl(8)
1090Description:
1091Status:
1092
1093rtems-libbsd File:      rtems-bsd-sysctlnametomib.c
1094FreeBSD File:           User API for sysctlnametomib
1095Description:
1096Status:
1097
1098rtems-libbsd File:      rtems-bsd-taskqueue.c
1099FreeBSD File:           kern/subr_taskqueue.c
1100Description:
1101Status:                 Nic
1102
1103rtems-libbsd File:      rtems-bsd-thread.c
1104FreeBSD File:           kern/kern_kthread.c
1105Description:
1106Status:                 USB, Nic
1107
1108rtems-libbsd File:      rtems-bsd-timeout.c
1109FreeBSD File:           kern/kern_timeout.c
1110Description:
1111Status:                 Nic
1112
1113rtems-libbsd File:      rtems-bsd-timesupport.c
1114FreeBSD File:           kern/kern_clock.c
1115Description:
1116Status:                 Nic
1117
1118rtems-libbsd File:      rtems-bsd-vm_glue.c
1119FreeBSD File:           vm/vm_glue.c
1120Description:
1121Status:                 USB, Nic
1122----
1123
1124== Notes by File ==
1125
1126altq_subr.c - Arbitrary choices were made in this file that RTEMS would
1127not support tsc frequency change.  Additionally, the clock frequency
1128for machclk_freq is always measured for RTEMS.
1129
1130conf.h - In order to add make_dev and destroy_dev, variables in the cdev
1131structure that were not being used were conditionally compiled out. The
1132capability of supporting children did not appear to be needed and was
1133not implemented in the rtems version of these routines.
1134
1135== NICs Status ==
1136
1137----
1138Driver                  Symbol                          Status
1139======                  ======                          ======
1140RealTek                 _bsd_re_pcimodule_sys_init      Links
1141EtherExpress            _bsd_fxp_pcimodule_sys_init     Links
1142DEC tulip               _bsd_dc_pcimodule_sys_init      Links
1143Broadcom BCM57xxx       _bsd_bce_pcimodule_sys_init     Links
1144Broadcom BCM4401        _bsd_bfe_pcimodule_sys_init     Links
1145Broadcom BCM570x        _bsd_bge_pcimodule_sys_init     Needs Symbols (A)
1146E1000 IGB               _bsd_igb_pcimodule_sys_init     Links
1147E1000 EM                _bsd_em_pcimodule_sys_init      Links
1148Cadence                 ?                               Links, works.
1149----
1150
1151Symbols (A)
1152         pci_get_vpd_ident
1153
1154=== Cadence ===
1155
1156The cadence driver works on the Xilinx Zynq platform. The hardware checksum
1157support works on real hardware but does not seem to be supported on qemu
1158therefore the default state is to disable TXCSUM and RXCSUM and this can be
1159enabled from the shell with:
1160
1161  # ifconfig cgem0 rxcsum txcsum
1162
1163or with an ioctl call to the network interface driver with SIOCSIFCAP and the
1164mask IFCAP_TXCSUM and IFCAP_RXCSUM set.
1165
1166== Problems to report to FreeBSD ==
1167
1168The MMAP_NOT_AVAILABLE define is inverted on its usage.  When it is
1169defined the mmap method is called. Additionally, it is not used
1170thoroughly. It is not used in the unmap portion of the source.
1171The file rec_open.c uses the define MMAP_NOT_AVAILABLE to wrap
1172the call to mmap and file rec_close.c uses the munmap method.
Note: See TracBrowser for help on using the repository browser.