source: rtems-libbsd/libbsd.txt @ 613c341

55-freebsd-126-freebsd-12freebsd-9.3
Last change on this file since 613c341 was 613c341, checked in by Sebastian Huber <sebastian.huber@…>, on 10/01/15 at 06:03:19

doc: Add Qemu network setup

  • Property mode set to 100644
File size: 40.0 KB
Line 
1RTEMS BSD Library Guide
2=======================
3:toc:
4:icons:
5:numbered:
6:website: http://www.rtems.org/
7
8RTEMS uses FreeBSD 9.2 as the source of its TCP/IP and USB stacks.
9This is a guide which captures information on the
10process of merging code from FreeBSD, building this library,
11RTEMS specific support files, and general guidelines on what
12modifications to the FreeBSD source are permitted.
13
14Goals of this effort are
15
16* update TCP/IP and provide USB in RTEMS,
17* ease updating to future FreeBSD versions,
18* ease tracking changes in FreeBSD code,
19* minimize manual changes in FreeBSD code, and
20* define stable kernel/device driver API which is implemented
21by both RTEMS and FreeBSD. This is the foundation of the port.
22
23We will work to push our changes upstream to the FreeBSD Project
24and minimize changes required at each update point.
25
26*******************************************************************************
27This is a work in progress and is very likely to be incomplete.
28Please help by adding to it.
29*******************************************************************************
30
31== Getting Started
32
33=== Tool Chain ===
34
35You need a tool chain for RTEMS based on at least
36
37* Binutils 2.24, and
38* Newlib 2.1.0.
39
40The Binutils version is required to ease the handling of linker command files.
41The Newlib version is required since some standard files like `<sys/types.h>`
42must be compatible enough for the files provided by the FreeBSD sources, e.g.
43`<sys/socket.h>`.
44
45=== Installation Overview ===
46
47. You must configure your BSP with the +--disable-networking+ option to disable
48the old network stack.  Make sure no header files of the old network stack are
49installed.
50. Clone the Git repository +git clone git://git.rtems.org/rtems-libbsd.git+.
51. Change into the RTEMS BSD library root directory.
52. Edit the `config.inc` Makefile configuration file and adjust it to your environment.
53. Run +make clean+.
54. Run +make install+.
55
56=== Board Support Package Requirements ===
57
58The RTEMS version must be at least 4.11.  The Board Support Package (BSP)
59should support the
60http://www.rtems.org/onlinedocs/doxygen/cpukit/html/group\__rtems\__interrupt__extension.html[Interrupt Manager Extension]
61// The first underscores have to be masked to stop asciidoc interpreting them
62to make use of generic FreeBSD based drivers.
63
64The linker command file of the BSP must contain the following section
65definitions:
66
67-------------------------------------------------------------------------------
68.rtemsroset : {
69        KEEP (*(SORT(.rtemsroset.*)))
70}
71
72.rtemsrwset : {
73        KEEP (*(SORT(.rtemsrwset.*)))
74}
75-------------------------------------------------------------------------------
76
77The first output section can be placed in read-only memory.  The second output
78section must be placed in read-write memory.  The output section name is not
79relevant.  The output sections may also contain other input sections.
80
81=== Board Support Package Configuration and Build ===
82
83You need to configure RTEMS for the desired BSP and install it.  The BSP should
84be configured with a disabled network stack.  The BSD library containing the
85new network stack is a separate package.  Using a BSP installation containing
86the old network stack may lead to confusion and unpredictable results.
87
88The following script is used to build the `arm/realview_pbx_a9_qemu` BSP for
89our internal testing purposes:
90
91-------------------------------------------------------------------------------
92#!/bin/sh
93
94cd ${HOME}/sandbox
95rm -rf b-realview_pbx_a9_qemu
96mkdir b-realview_pbx_a9_qemu
97cd b-realview_pbx_a9_qemu
98${HOME}/git-rtems/configure \
99        --prefix=${HOME}/sandbox/install \
100        --target=arm-rtems4.11 \
101        --enable-rtemsbsp=realview_pbx_a9_qemu \
102        --disable-networking && \
103        make && \
104        make install
105-------------------------------------------------------------------------------
106
107The `arm/realview_pbx_a9_qemu` BSP running on the Qemu simulator has some
108benefits for development and test of the BSD library
109
110* it offers a NULL pointer read and write protection,
111* Qemu is a fast simulator,
112* Qemu provides support for GDB watchpoints,
113* Qemu provides support for virtual Ethernet networks, e.g. TUN and bridge
114devices (you can run multiple test instances on one virtual network).
115
116=== BSD Library Configuration and Build ===
117
118There are currently 2 build systems supported. The first is based on the RTEMS
119Makefile support in RTEMS and installed with the BSP and the second is a stand
120alone environment based on the Waf build system. The Makefile build system will
121be removed when RTEMS moves away from its existing build system and Waf will
122be the preferred build environment.
123
124To build with Waf please refer to the README.waf file.
125
126===== Makefile Building =====
127
128In the BSD library source directory edit the file `config.inc`.  Continuing on
129the above, the `config.inc` used to match the above is:
130
131-------------------------------------------------------------------------------
132# Mandatory: Select your BSP and installation prefix
133TARGET = arm-rtems4.11
134BSP = realview_pbx_a9_qemu
135PREFIX = $(HOME)/sandbox/install
136
137# Optional: Separate installation base directory
138INSTALL_BASE = $(PREFIX)/$(TARGET)/$(BSP)
139
140# Optional: Network test configuration
141TEST_RUNNER = $(BSP)
142NET_CFG_SELF_IP = 10.0.0.2
143NET_CFG_NETMASK = 255.255.0.0
144NET_CFG_PEER_IP = 10.0.0.1
145NET_CFG_GATEWAY_IP = 10.0.0.1
146NET_TAP_INTERFACE = tap0
147-------------------------------------------------------------------------------
148
149Now you can build the BSD library and run the tests:
150
151-------------------------------------------------------------------------------
152make clean
153make
154make run_tests
155-------------------------------------------------------------------------------
156
157You can only run the tests directly in case a test runner is available.  The
158following tests run without an external network.  It is strongly advised to run
159them.
160
161* commands01
162* init01
163* loopback01
164* rwlock01
165* selectpollkqueue01
166* sleep01
167* swi01
168* syscalls01
169* thread01
170* timeout01
171* unix01
172
173To install the BSD library use this:
174
175-------------------------------------------------------------------------------
176make install
177-------------------------------------------------------------------------------
178
179=== BSD Library Initialization ===
180
181Use the following code to initialize the BSD library:
182
183-------------------------------------------------------------------------------
184#include <assert.h>
185#include <sysexits.h>
186
187#include <machine/rtems-bsd-commands.h>
188#include <rtems/bsd/bsd.h>
189
190static void
191network_ifconfig_lo0(void)
192{
193        int exit_code;
194        char *lo0[] = {
195                "ifconfig",
196                "lo0",
197                "inet",
198                "127.0.0.1",
199                "netmask",
200                "255.255.255.0",
201                NULL
202        };
203        char *lo0_inet6[] = {
204                "ifconfig",
205                "lo0",
206                "inet6",
207                "::1",
208                "prefixlen",
209                "128",
210                NULL
211        };
212
213        exit_code = rtems_bsd_command_ifconfig(RTEMS_BSD_ARGC(lo0), lo0);
214        assert(exit_code == EX_OK);
215
216        exit_code = rtems_bsd_command_ifconfig(RTEMS_BSD_ARGC(lo0_inet6), lo0_inet6);
217        assert(exit_code == EX_OK);
218}
219
220void
221network_init(void)
222{
223        rtems_status_code sc;
224
225        sc = rtems_bsd_initialize();
226        assert(sc == RTEMS_SUCCESSFUL);
227
228        network_ifconfig_lo0();
229}
230-------------------------------------------------------------------------------
231
232This performs the basic network stack initialization with a loopback interface.
233Further initialization must be done using the standard BSD network
234configuration commands
235http://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[IFCONFIG(8)]
236using `rtems_bsd_command_ifconfig()` and
237http://www.freebsd.org/cgi/man.cgi?query=route&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[ROUTE(8)]
238using `rtems_bsd_command_route()`.  For an example please have a look at
239`testsuite/include/rtems/bsd/test/default-network-init.h`.
240
241=== Task Priorities and Stack Size ===
242
243The default task priority is 96 for the interrupt server task (name "IRQS"), 98
244for the timer server task (name "TIME") and 100 for all other tasks.  The
245application may provide their own implementation of the
246`rtems_bsd_get_task_priority()` function (for example in the module which calls
247`rtems_bsd_initialize()`) if different values are desired.
248
249The task stack size is determined by the `rtems_bsd_get_task_stack_size()`
250function which may be provided by the application in case the default is not
251appropriate.
252
253=== Size for Allocator Domains ===
254
255The size for an allocator domain can be specified via the
256`rtems_bsd_get_allocator_domain_size()` function.  The application may provide
257their own implementation of the `rtems_bsd_get_allocator_domain_size()`
258function (for example in the module which calls `rtems_bsd_initialize()`) if
259different values are desired.  The default size is 8MiB for all domains.
260
261== Network Stack Features
262
263http://roy.marples.name/projects/dhcpcd/index[DHCPCD(8)]:: DHCP client
264
265https://developer.apple.com/library/mac/documentation/Networking/Reference/DNSServiceDiscovery_CRef/Reference/reference.html[dns_sd.h]:: DNS Service Discovery
266
267http://www.opensource.apple.com/source/mDNSResponder/mDNSResponder-320.10/mDNSCore/mDNSEmbeddedAPI.h[mDNS]:: Multi-Cast DNS
268
269http://www.freebsd.org/cgi/man.cgi?query=unix&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[UNIX(4)]:: UNIX-domain protocol family
270
271http://www.freebsd.org/cgi/man.cgi?query=inet&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[INET(4)]:: Internet protocol family
272
273http://www.freebsd.org/cgi/man.cgi?query=inet6&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[INET6(4)]:: Internet protocol version 6 family
274
275http://www.freebsd.org/cgi/man.cgi?query=tcp&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[TCP(4)]:: Internet Transmission Control Protocol
276
277http://www.freebsd.org/cgi/man.cgi?query=udp&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[UDP(4)]:: Internet User Datagram Protocol
278
279http://www.freebsd.org/cgi/man.cgi?query=route&sektion=4&apropos=0&manpath=FreeBSD+9.2-RELEASE[ROUTE(4)]:: Kernel packet forwarding database
280
281http://www.freebsd.org/cgi/man.cgi?query=bpf&apropos=0&sektion=4&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[BPF(4)]:: Berkeley Packet Filter
282
283http://www.freebsd.org/cgi/man.cgi?query=socket&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[SOCKET(2)]:: Create an endpoint for communication
284
285http://www.freebsd.org/cgi/man.cgi?query=kqueue&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[KQUEUE(2)]:: Kernel event notification mechanism
286
287http://www.freebsd.org/cgi/man.cgi?query=select&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[SELECT(2)]:: Synchronous I/O multiplexing
288
289http://www.freebsd.org/cgi/man.cgi?query=poll&apropos=0&sektion=2&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[POLL(2)]:: Synchronous I/O multiplexing
290
291http://www.freebsd.org/cgi/man.cgi?query=route&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[ROUTE(8)]:: Manually manipulate the routing tables
292
293http://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[IFCONFIG(8)]:: Configure network interface parameters
294
295http://www.freebsd.org/cgi/man.cgi?query=netstat&apropos=0&sektion=1&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[NETSTAT(1)]:: Show network status
296
297http://www.freebsd.org/cgi/man.cgi?query=ping&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[PING(8)]:: Send ICMP ECHO_REQUEST packets to network hosts
298
299http://www.freebsd.org/cgi/man.cgi?query=ping6&apropos=0&sektion=8&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[PING6(8)]:: Send ICMPv6 ECHO_REQUEST packets to network hosts
300
301http://www.freebsd.org/cgi/man.cgi?query=sysctl&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[SYSCTL(3)]:: Get or set system information
302
303http://www.freebsd.org/cgi/man.cgi?query=resolver&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[RESOLVER(3)]:: Resolver routines
304
305http://www.freebsd.org/cgi/man.cgi?query=gethostbyname&sektion=3&apropos=0&manpath=FreeBSD+9.2-RELEASE[GETHOSTBYNAME(3)]:: Get network host entry
306
307== Network Interface Drivers
308
309=== Link Up/Down Events
310
311You can notifiy the application space of link up/down events in your network
312interface driver via the if_link_state_change(LINK_STATE_UP/LINK_STATE_DOWN)
313function.  The DHCPCD(8) client is a consumer of these events for example.
314Make sure that the interface flag IFF_UP and the interface driver flag
315IFF_DRV_RUNNING is set in case the link is up, otherwise ether_output() will
316return the error status ENETDOWN.
317
318== Shell Commands
319
320=== HOSTNAME(1)
321
322In addition to the standard options the RTEMS version of the HOSTNAME(1)
323command supports the -m flag to set/get the multicast hostname of the
324mDNS resolver instance.  See also rtems_mdns_sethostname() and
325rtems_mdns_gethostname().
326
327== Qemu
328
329Use the following script to set up a virtual network with three tap devices
330connected via one bridge device.
331
332-------------------------------------------------------------------------------
333#!/bin/sh -x
334
335user=`whoami`
336interfaces=(1 2 3)
337
338tap=qtap
339bri=qbri
340
341case $1 in
342        up)
343                sudo -i brctl addbr $bri
344                for i in ${interfaces[@]} ; do
345                        sudo -i tunctl -t $tap$i -u $user ;
346                        sudo -i ifconfig $tap$i up ;
347                        sudo -i brctl addif $bri $tap$i ;
348                done
349                sudo -i ifconfig $bri up
350                ;;
351        down)
352                for i in ${interfaces[@]} ; do
353                        sudo -i ifconfig $tap$i down ;
354                        sudo -i tunctl -d $tap$i ;
355                done
356                sudo -i ifconfig $bri down
357                sudo -i brctl delbr $bri
358                ;;
359esac
360-------------------------------------------------------------------------------
361
362Connect your Qemu instance to one of the tap devices, e.g.
363
364-------------------------------------------------------------------------------
365qemu-system-i386 -m 512 -boot a -cpu pentium3 \
366        -drive file=$HOME/qemu/pc386_fda,index=0,if=floppy,format=raw \
367        -drive file=fat:$HOME/qemu/hd,format=raw \
368        -net nic,model=e1000,macaddr=0e:b0:ba:5e:ba:11 \
369        -net tap,ifname=qtap1,script=no,downscript=no \
370        -nodefaults -nographic -serial stdio
371-------------------------------------------------------------------------------
372
373Make sure that each Qemu instance uses its own MAC address to avoid an address
374conflict (or otherwise use it as a test).
375
376== Issues and TODO
377
378* PCI support on x86 uses a quick and dirty hack, see pci_reserve_map().
379
380* Priority queues are broken with clustered scheduling.
381
382* Per-CPU data should be enabled once the new stack is ready for SMP.
383
384* Per-CPU NETISR(9) should be enabled onece the new stack is ready for SMP.
385
386* Multiple routing tables are not supported.  Every FIB value is set to zero
387  (= BSD_DEFAULT_FIB).
388
389* Process identifiers are not supported.  Every PID value is set to zero
390  (= BSD_DEFAULT_PID).
391
392* User credentials are not supported.  The following functions allow the
393  operation for everyone
394  - prison_equal_ip4(),
395  - chgsbsize(),
396  - cr_cansee(),
397  - cr_canseesocket() and
398  - cr_canseeinpcb().
399
400* A basic USB functionality test that is known to work on Qemu is desirable.
401
402* Adapt generic IRQ PIC interface code to Simple Vectored Interrupt Model
403  so that those architectures can use new TCP/IP and USB code.
404
405* freebsd-userspace/rtems/include/sys/syslog.h is a copy from the old
406  RTEMS TCP/IP stack. For some reason, the __printflike markers do not
407  compile in this environment. We may want to use the FreeBSD syslog.h
408  and get this addressed.
409
410* in_cksum implementations for architectures not supported by FreeBSD.
411  This will require figuring out where to put implementations that do
412  not originate from FreeBSD and are populated via the script.
413
414* MAC support functions are not thread-safe ("freebsd/lib/libc/posix1e/mac.c").
415
416* IFCONFIG(8): IEEE80211 support is disabled.  This module depends on a XML
417  parser and mmap().
418
419* get_cyclecount(): The implementation is a security problem.
420
421* What to do with the priority parameter present in the FreeBSD synchronization
422  primitives and the thread creation functions?
423
424* TASKQUEUE(9): Support spin mutexes.
425
426* ZONE(9): Review allocator lock usage in rtems-bsd-chunk.c.
427
428* KQUEUE(2): Choose proper lock for global kqueue list.
429
430* TIMEOUT(9): Maybe use special task instead of timer server to call
431  callout_tick().
432
433* sysctl_handle_opaque(): Implement reliable snapshots.
434
435* PING6(8): What to do with SIGALARM?
436
437* <sys/param.h>: Update Newlib to use a MSIZE of 256.
438
439* BPF(4): Add support for zero-copy buffers.
440
441* UNIX(4): Fix race conditions in the area of socket object and file node
442  destruction.  Add support for file descriptor transmission via control
443  messages.
444
445* PRINTF(9): Add support for log(), the %D format specifier is missing in the
446  normal printf() family.
447
448* Why is the interrupt server used?  The BSD interrupt handlers can block on
449synchronization primitives like mutexes.  This is in contrast to RTEMS
450interrupt service routines.  The BSPs using the generic interrupt support must
451implement the `bsp_interrupt_vector_enable()` and
452`bsp_interrupt_vector_disable()` routines.  They normally enable/disable a
453particular interrupt source at the interrupt controller.  This can be used to
454implement the interrupt server.  The interrupt server is a task that wakes-up
455in case an associated interrupt happens.  The interrupt source is disabled in
456a generic interrupt handler that wakes-up the interrupt server task.   Once the
457postponed interrupt processing is performed in the interrupt server the
458interrupt source is enabled again.
459
460* Convert all BSP linkcmds to use a linkcmds.base so the sections are
461easier to insert.
462
463* NIC Device Drivers
464- Only common PCI NIC drivers have been included in the initial set. These
465do not include any system on chip or ISA drivers.
466- PCI configuration probe does not appear to happen to determine if a
467NIC is in I/O or memory space. We have worked around this by using a
468static hint to tell the fxp driver the correct mode. But this needs to
469be addressed.
470- The ISA drivers require more BSD infrastructure to be addressed. This was
471outside the scope of the initial porting effort.
472
473== FreeBSD Source
474
475You should be able to rely on FreebSD manual pages and documentation
476for details on the code itself.
477
478=== Automatically Generated FreeBSD Files
479
480Some source and header files are automatically generated during the FreeBSD
481build process.  The `Makefile.todo` file performs this manually.  The should be
482included in `freebsd-to-rtems.py` script some time in the future.  For details,
483see also
484http://www.freebsd.org/cgi/man.cgi?query=kobj&sektion=9&apropos=0&manpath=FreeBSD+9.2-RELEASE[KOBJ(9)].
485
486=== Rules for Modifying FreeBSD Source
487
488Only add lines.  If your patch contains lines starting with a '-', then this is
489wrong.  Subtract code by added `#ifndef __rtems__`.  This makes merging easier
490in the future.  For example:
491
492-------------------------------------------------------------------------------
493/* Global variables for the kernel. */
494
495#ifndef __rtems__
496/* 1.1 */
497extern char kernelname[MAXPATHLEN];
498#endif /* __rtems__ */
499
500extern int tick;                        /* usec per tick (1000000 / hz) */
501-------------------------------------------------------------------------------
502
503-------------------------------------------------------------------------------
504#if defined(_KERNEL) || defined(_WANT_FILE)
505#ifdef __rtems__
506#include <rtems/libio_.h>
507#include <sys/fcntl.h>
508#endif /* __rtems__ */
509/*
510 * Kernel descriptor table.
511 * One entry for each open kernel vnode and socket.
512 *
513 * Below is the list of locks that protects members in struct file.
514 *
515 * (f) protected with mtx_lock(mtx_pool_find(fp))
516 * (d) cdevpriv_mtx
517 * none not locked
518 */
519-------------------------------------------------------------------------------
520
521-------------------------------------------------------------------------------
522extern int profprocs;                   /* number of process's profiling */
523#ifndef __rtems__
524extern volatile int ticks;
525#else /* __rtems__ */
526#include <rtems/score/watchdogimpl.h>
527#define ticks _Watchdog_Ticks_since_boot
528#endif /* __rtems__ */
529
530#endif /* _KERNEL */
531-------------------------------------------------------------------------------
532
533Add nothing (even blank lines) before or after the `__rtems__` guards.  Always
534include a `__rtems__` in the guards to make searches easy, so use
535
536* `#ifndef __rtems__`,
537* `#ifdef __rtems__`,
538* `#else /* __rtems__ */`, and
539* `#endif /* __rtems__ */`.
540
541For new code use
542http://www.freebsd.org/cgi/man.cgi?query=style&apropos=0&sektion=9&manpath=FreeBSD+9.2-RELEASE&arch=default&format=html[STYLE(9)].
543Do not format original FreeBSD code.
544
545== BSD Library Source
546
547=== What is in the Git Repository
548
549There is a self-contained kit with FreeBSD and RTEMS components pre-merged. The
550Makefile in this kit is automatically generated.
551
552Any changes to source in the `freebsd` directories will need to be merged
553upstream into our master FreeBSD checkout, the `freebsd-org` submodule.
554
555The repository contains two FreeBSD source trees.  In the `freebsd` directory
556are the so called 'managed' FreeBSD sources used to build the BSD library.  The
557FreeBSD source in `freebsd-org` is the 'master' version.  The
558`freebsd-to-rtems.py` script is used to transfer files between the two trees.
559In general terms, if you have modified managed FreeBSD sources, you will need
560to run the script in 'revert' or 'reverse' mode using the `-R` switch.  This
561will copy the source back to your local copy of the master FreeBSD source so
562you can run `git diff` against the upstream FreeBSD source.  If you want to
563transfer source files from the master FreeBSD source to the manged FreeBSD
564sources, then you must run the script in 'forward' mode (the default).
565
566=== Organization
567
568The top level directory contains a few directories and files. The following
569are important to understand
570
571* `freebsd-to-rtems.py` - script to convert to and free FreeBSD and RTEMS trees,
572* `Makefile` - automatically generated,
573* `freebsd/` - from FreeBSD by script,
574* `rtemsbsd/` - RTEMS specific implementations of FreeBSD kernel support routines,
575* `testsuite/` - RTEMS specific tests, and
576* `libbsd.txt` - documentation in Asciidoc.
577
578== Moving Code Between Managed and Master FreeBSD Source
579
580The script `freebsd-to-rtems.py` is used to copy code from FreeBSD to the
581rtems-libbsd tree and to reverse this process. This script attempts to
582automate this process as much as possible and performs some transformations
583on the FreeBSD code. Its command line arguments are shown below:
584
585----
586freebsd-to-rtems.py [args]
587  -?|-h|--help     print this and exit
588  -d|--dry-run     run program but no modifications
589  -D|--diff        provide diff of files between trees
590  -e|--early-exit  evaluate arguments, print results, and exit
591  -m|--makefile    just generate Makefile
592  -R|--reverse     default FreeBSD -> RTEMS, reverse that
593  -r|--rtems       RTEMS directory
594  -f|--freebsd     FreeBSD directory
595  -v|--verbose     enable verbose output mode
596----
597
598In its default mode of operation, freebsd-to-rtems.py is used to copy code
599from FreeBSD to the rtems-libbsd tree and perform transformations.  In forward
600mode, the script may be requested to just generate the Makefile and Waf script.
601
602In "reverse mode", this script undoes those transformations and copies
603the source code back to the FreeBSD SVN tree. This allows us to do
604'svn diff', evaluate changes made by the RTEMS Project, and report changes
605back to FreeBSD upstream.
606
607In either mode, the script may be asked to perform a dry-run or be verbose.
608Also, in either mode, the script is also smart enough to avoid copying over
609files which have not changed. This means that the timestamps of files are
610not changed unless the contents change. The script will also report the
611number of files which changed. In verbose mode, the script will print
612the name of the files which are changed.
613
614The following is an example forward run with no changes.
615
616----
617$ ~/newbsd/git/libbsd-8.2/freebsd-to-rtems.py \
618    -r /home/joel/newbsd/git/libbsd-8.2 \
619    -f /home/joel/newbsd/libbsd/freebsd-8.2 -v
620Verbose:                yes
621Dry Run:                no
622Only Generate Makefile: no
623RTEMS Directory:        /home/joel/newbsd/git/libbsd-8.2
624FreeBSD Directory:      /home/joel/newbsd/libbsd/freebsd-8.2
625Direction:              forward
626Generating into /home/joel/newbsd/git/libbsd-8.2
6270 files were changed.
628----
629
630The script may also be used to generate a diff in either forward or reverse
631direction.
632
633== Initialization of the BSD Library
634
635The initialization of the BSD library is based on the FreeBSD SYSINIT(9)
636infrastructure.  The key to initializing a system is to ensure that the desired
637device drivers are explicitly pulled into the linked application.  This plus
638linking against the BSD library (`libbsd.a`) will pull in the necessary FreeBSD
639infrastructure.
640
641The FreeBSD kernel is not a library like the RTEMS kernel.  It is a bunch of
642object files linked together.  If we have a library, then creating the
643executable is simple.  We begin with a start symbol and recursively resolve all
644references.  With a bunch of object files linked together we need a different
645mechanism.  Most object files don't know each other.  Lets say we have a driver
646module.  The rest of the system has no references to this driver module.  The
647driver module needs a way to tell the rest of the system: Hey, kernel I am
648here, please use my services!
649
650This registration of independent components is performed by SYSINIT(9) and
651specializations:
652
653http://www.freebsd.org/cgi/man.cgi?query=SYSINIT
654
655The SYSINIT(9) uses some global data structures that are placed in a certain
656section.  In the linker command file we need this:
657
658-------------------------------------------------------------------------------
659.rtemsroset : {
660        KEEP (*(SORT(.rtemsroset.*)))
661}
662
663.rtemsrwset : {
664        KEEP (*(SORT(.rtemsrwset.*)))
665}
666-------------------------------------------------------------------------------
667
668This results for example in this executable layout:
669
670-------------------------------------------------------------------------------
671[...]
672 *(SORT(.rtemsroset.*))
673 .rtemsroset.bsd.modmetadata_set.begin
674                0x000000000025fe00        0x0 libbsd.a(rtems-bsd-init.o)
675                0x000000000025fe00                _bsd__start_set_modmetadata_set
676 .rtemsroset.bsd.modmetadata_set.content
677                0x000000000025fe00        0x8 libbsd.a(rtems-bsd-nexus.o)
678 .rtemsroset.bsd.modmetadata_set.content
679                0x000000000025fe08        0x4 libbsd.a(kern_module.o)
680[...]
681 .rtemsroset.bsd.modmetadata_set.content
682                0x000000000025fe68        0x4 libbsd.a(mii.o)
683 .rtemsroset.bsd.modmetadata_set.content
684                0x000000000025fe6c        0x4 libbsd.a(mii_bitbang.o)
685 .rtemsroset.bsd.modmetadata_set.end
686                0x000000000025fe70        0x0 libbsd.a(rtems-bsd-init.o)
687                0x000000000025fe70                _bsd__stop_set_modmetadata_set
688[...]
689.rtemsrwset     0x000000000030bad0      0x290
690 *(SORT(.rtemsrwset.*))
691 .rtemsrwset.bsd.sysinit_set.begin
692                0x000000000030bad0        0x0 libbsd.a(rtems-bsd-init.o)
693                0x000000000030bad0                _bsd__start_set_sysinit_set
694 .rtemsrwset.bsd.sysinit_set.content
695                0x000000000030bad0        0x4 libbsd.a(rtems-bsd-nexus.o)
696 .rtemsrwset.bsd.sysinit_set.content
697                0x000000000030bad4        0x8 libbsd.a(rtems-bsd-thread.o)
698 .rtemsrwset.bsd.sysinit_set.content
699                0x000000000030badc        0x4 libbsd.a(init_main.o)
700[...]
701 .rtemsrwset.bsd.sysinit_set.content
702                0x000000000030bd54        0x4 libbsd.a(frag6.o)
703 .rtemsrwset.bsd.sysinit_set.content
704                0x000000000030bd58        0x8 libbsd.a(uipc_accf.o)
705 .rtemsrwset.bsd.sysinit_set.end
706                0x000000000030bd60        0x0 libbsd.a(rtems-bsd-init.o)
707                0x000000000030bd60                _bsd__stop_set_sysinit_set
708[...]
709-------------------------------------------------------------------------------
710
711Here you can see, that some global data structures are collected into
712continuous memory areas.  This memory area can be identified by start and stop
713symbols.  This constructs a table of uniform items.
714
715The low level FreeBSD code calls at some time during the initialization the
716mi_startup() function (machine independent startup).  This function will sort
717the SYSINIT(9) set and call handler functions which perform further
718initialization.  The last step is the scheduler invocation.
719
720The SYSINIT(9) routines are run in mi_startup() which is called by
721rtems_bsd_initialize().
722
723This is also explained in "The Design and Implementation of the FreeBSD
724Operating System" section 14.3 "Kernel Initialization".
725
726In RTEMS we have a library and not a bunch of object files.  Thus we need a way
727to pull-in the desired services out of the libbsd.  Here the
728`rtems-bsd-sysinit.h` comes into play.  The SYSINIT(9) macros have been
729modified and extended for RTEMS in `<sys/kernel.h>`:
730
731-------------------------------------------------------------------------------
732#ifndef __rtems__
733#define C_SYSINIT(uniquifier, subsystem, order, func, ident)    \
734        static struct sysinit uniquifier ## _sys_init = {       \
735                subsystem,                                      \
736                order,                                          \
737                func,                                           \
738                (ident)                                         \
739        };                                                      \
740        DATA_SET(sysinit_set,uniquifier ## _sys_init)
741#else /* __rtems__ */
742#define SYSINIT_ENTRY_NAME(uniquifier)                          \
743        _bsd_ ## uniquifier ## _sys_init
744#define SYSINIT_REFERENCE_NAME(uniquifier)                      \
745        _bsd_ ## uniquifier ## _sys_init_ref
746#define C_SYSINIT(uniquifier, subsystem, order, func, ident)    \
747        struct sysinit SYSINIT_ENTRY_NAME(uniquifier) = {       \
748                subsystem,                                      \
749                order,                                          \
750                func,                                           \
751                (ident)                                         \
752        };                                                      \
753        RWDATA_SET(sysinit_set,SYSINIT_ENTRY_NAME(uniquifier))
754#define SYSINIT_REFERENCE(uniquifier)                           \
755        extern struct sysinit SYSINIT_ENTRY_NAME(uniquifier);   \
756        static struct sysinit const * const                     \
757        SYSINIT_REFERENCE_NAME(uniquifier) __used               \
758        = &SYSINIT_ENTRY_NAME(uniquifier)
759#define SYSINIT_MODULE_REFERENCE(mod)                           \
760        SYSINIT_REFERENCE(mod ## module)
761#define SYSINIT_DRIVER_REFERENCE(driver, bus)                   \
762        SYSINIT_MODULE_REFERENCE(driver ## _ ## bus)
763#define SYSINIT_DOMAIN_REFERENCE(dom)                           \
764        SYSINIT_REFERENCE(domain_add_ ## dom)
765#endif /* __rtems__ */
766-------------------------------------------------------------------------------
767
768Here you see that the SYSINIT(9) entries are no longer static.  The
769\*_REFERENCE() macros will create references to the corresponding modules which
770are later resolved by the linker.  The application has to provide an object
771file with references to all required FreeBSD modules.
772
773The FreeBSD device model is quite elaborated (with follow-ups):
774
775http://www.freebsd.org/cgi/man.cgi?query=driver
776
777The devices form a tree with the Nexus device at a high-level.  This Nexus
778device is architecture specific in FreeBSD.  In RTEMS we have our own Nexus
779device, see `rtemsbsd/bsp/bsp-bsd-nexus-devices.c`.
780
781=== SYSCTL_NODE Example
782
783During development, we had an undefined reference to
784_bsd_sysctl__net_children that we had trouble tracking down. Thanks to
785Chris Johns, we located it. He explained how to read SYSCTL_NODE
786definitions. This line from freebsd/netinet/in_proto.c is attempting
787to add the "inet" node to the parent node "_net".
788
789----
790SYSCTL_NODE(_net,      PF_INET,         inet,   CTLFLAG_RW, 0,
791        "Internet Family");
792----
793
794Our problem was that we could not find where _bsd_sysctl__net_children
795was defined. Chris suggested that when in doubt compile with -save-temps
796and look at the preprocessed .i files. But he did not need that. He
797explained that this the symbol name _bsd_sysctl__net_children was
798automatically generated by a SYSCTL_NODE as follows:
799
800* _bsd_ - added by RTEMS modifications to SYSCTL_NODE macro
801* sysctl_ - boilerplace added by SYSCTL_NODE macro
802* "" - empty string for parent node
803* net - name of SYSCTL_NODE
804* children - added by SYSCTL macros
805
806This was all generated by a support macro declaring the node as this:
807
808----
809struct sysctl_oid_list SYSCTL_NODE_CHILDREN(parent, name);
810----
811
812Given this information, we located this SYSCTL_NODE declaration in
813kern/kern_mib.c
814
815----
816SYSCTL_NODE(, CTL_KERN,   kern,   CTLFLAG_RW, 0,
817        "High kernel, proc, limits &c");
818----
819
820== Core FreeBSD APIs and RTEMS Replacements ==
821
822=== SX(9) (Shared/exclusive locks) ===
823
824http://www.freebsd.org/cgi/man.cgi?query=sx
825
826Binary semaphores (this neglects the ability to allow shared access).
827
828=== MUTEX(9) (Mutual exclusion) ===
829
830http://www.freebsd.org/cgi/man.cgi?query=mutex
831
832Binary semaphores (not recursive mutexes are not supported this way).
833
834=== RWLOCK(9) (Reader/writer lock) ===
835
836http://www.freebsd.org/cgi/man.cgi?query=rwlock
837
838POSIX r/w lock.
839
840=== RMLOCK(9) (Reader/writer lock optimized for mostly read access patterns) ===
841
842Note:  This object was implemented as a wrapper for RWLOCK in the rm_lock header file.
843
844http://www.freebsd.org/cgi/man.cgi?query=rmlock
845
846POSIX r/w lock.
847
848=== CONDVAR(9) (Condition variables) ===
849
850http://www.freebsd.org/cgi/man.cgi?query=condvar
851
852POSIX condition variables with modifications (hack).
853
854=== CALLOUT(9) (Timer functions) ===
855
856http://www.freebsd.org/cgi/man.cgi?query=callout
857
858Timer server.
859
860=== TASKQUEUE(9) (Asynchronous task execution) ===
861
862http://www.freebsd.org/cgi/man.cgi?query=taskqueue
863
864TBD.
865
866=== KTHREAD(9), KPROC(9) (Tasks) ===
867
868http://www.freebsd.org/cgi/man.cgi?query=kthread
869
870http://www.freebsd.org/cgi/man.cgi?query=kproc
871
872Tasks.
873
874=== ZONE(9) (Zone allocator) ===
875
876http://www.freebsd.org/cgi/man.cgi?query=zone
877
878TBD.
879
880=== devfs (Device file system) ===
881
882Dummy, IMFS or new implementation (currently dummy).
883
884=== psignal (Signals) ===
885
886TBD.  Seems to be not needed.
887
888=== poll, select ===
889
890TBD.  Seems to be not needed.
891
892=== RMAN(9) (Resource management) ===
893
894http://www.freebsd.org/cgi/man.cgi?query=rman
895
896TBD.  Seems to be not needed.
897
898=== DEVCLASS(9), DEVICE(9), DRIVER(9), MAKE_DEV(9) (Device management) ===
899
900http://www.freebsd.org/cgi/man.cgi?query=devclass
901
902http://www.freebsd.org/cgi/man.cgi?query=device
903
904http://www.freebsd.org/cgi/man.cgi?query=driver
905
906http://www.freebsd.org/cgi/man.cgi?query=make_dev
907
908Use FreeBSD implementation as far as possible.  FreeBSD has a nice API for
909dynamic device handling.  It may be interesting for RTEMS to use this API
910internally in the future.
911
912=== BUS_SPACE(9), BUS_DMA(9) (Bus and DMA access) ===
913
914http://www.freebsd.org/cgi/man.cgi?query=bus_space
915
916http://www.freebsd.org/cgi/man.cgi?query=bus_dma
917
918Likely BSP dependent.  A default implementation for memory mapped linear access
919is easy to provide.  The current heap implementation supports all properties
920demanded by bus_dma (including the boundary constraint).
921
922== RTEMS Replacements by File Description ==
923
924Note:  Files with a status of USB are used by the USB test and have at least
925been partially tested.  If they contain both USB and Nic, then they are used
926by both and MAY contain methods that have not been tested yet.  Files that
927are only used by the Nic test are the most suspect.
928
929----
930rtems-libbsd File:      rtems-bsd-assert.c
931FreeBSD File:           rtems-bsd-config.h redefines BSD_ASSERT.
932Description:            This file contains the support method rtems_bsd_assert_func().
933Status:                 USB, Nic
934
935rtems-libbsd File:      rtems-bsd-autoconf.c
936FreeBSD File:           FreeBSD has BSP specific autoconf.c
937Description:            This file contains configuration methods that are used to setup the system.
938Status:                 USB
939
940rtems-libbsd File:      rtems-bsd-bus-dma.c
941FreeBSD File:           FreeBSD has BSP specific busdma_machdep.c
942Description:
943Status:                 USB, Nic
944
945rtems-libbsd File:      rtems-bsd-bus-dma-mbuf.c
946FreeBSD File:           FreeBSD has BSP specific busdma_machdep.c
947Description:
948Status:                 Nic
949
950rtems-libbsd File:      rtems-bsd-callout.c
951FreeBSD File:           kern/kern_timeout.c
952Description:
953Status:                 USB, Nic
954
955rtems-libbsd File:      rtems-bsd-cam.c
956FreeBSD File:           cam/cam_sim.c
957Description:
958Status:                 USB
959
960rtems-libbsd File:      rtems-bsd-condvar.c
961FreeBSD File:           kern/kern_condvar.c
962Description:
963Status:                 USB
964
965rtems-libbsd File:      rtems-bsd-copyinout.c
966FreeBSD File:           bsp specific copyinout.c )
967Description:            Note: The FreeBSD file is split with some methods being in rtems-bsd-support
968Status:                 Nic
969
970rtems-libbsd File:      rtems-bsd-delay.c
971FreeBSD File:           bsp specific file with multiple names
972Description:
973Status:                 USB, Nic
974
975rtems-libbsd File:      rtems-bsd-descrip.c
976FreeBSD File:           kern/kern_descrip.c
977Description:
978Status:                 Nic
979
980rtems-libbsd File:      rtems-bsd-generic.c
981FreeBSD File:           kern/sys_generic.c
982Description:
983Status:                 Nic
984
985rtems-libbsd File:      rtems-bsd-init.c
986FreeBSD File:           N/A
987Description:
988Status:                 USB, Nic
989
990rtems-libbsd File:      rtems-bsd-init-with-irq.c
991FreeBSD File:           N/A
992Description:
993Status:                 USB, Nic
994
995rtems-libbsd File:      rtems-bsd-jail.c
996FreeBSD File:           kern/kern_jail.c
997Description:
998Status:                 USB, Nic
999
1000rtems-libbsd File:      rtems-bsd-lock.c
1001FreeBSD File:           kern/subr_lock.c
1002Description:
1003Status:                 USB, Nic
1004
1005rtems-libbsd File:      rtems-bsd-log.c
1006FreeBSD File:           kern/subr_prf.c
1007Description:
1008Status:                 Nic
1009
1010rtems-libbsd File:      rtems-bsd-malloc.c
1011FreeBSD File:           kern/kern_malloc.c
1012Description:
1013Status:                 USB, Nic
1014
1015rtems-libbsd File:      rtems-bsd-mutex.c
1016FreeBSD File:           kern/kern_mutex.c
1017Description:
1018Status:                 USB, Nic
1019
1020rtems-libbsd File:      rtems-bsd-newproc.c
1021FreeBSD File:           N/A
1022Description:
1023Status:                 Nic
1024
1025rtems-libbsd File:      rtems-bsd-nexus.c
1026FreeBSD File:           bsp specific nexus.c
1027Description:
1028Status:                 USB
1029
1030rtems-libbsd File:      rtems-bsd-panic.c
1031FreeBSD File:           boot/common/panic.c
1032Description:
1033Status:                 USB, Nic
1034
1035rtems-libbsd File:      rtems-bsd-rwlock.c
1036FreeBSD File:           kern_rwlock.c
1037Description:
1038Status:                 USB, Nic
1039
1040rtems-libbsd File:      rtems-bsd-shell.c
1041FreeBSD File:           N/A
1042Description:
1043Status:                 USB
1044
1045rtems-libbsd File:      rtems-bsd-signal.c
1046FreeBSD File:           kern/kern_sig.c
1047Description:
1048Status:                 Nic
1049
1050rtems-libbsd File:      rtems-bsd-smp.c
1051FreeBSD File:           N/A
1052Description:
1053Status:                 Nic
1054
1055rtems-libbsd File:      rtems-bsd-support.c
1056FreeBSD File:           bsp specific copyinout.c
1057Description:            Note: the FreeBSD file is split with some methods being in rtems-bsd-copyinout.
1058Status:                 USB, Nic
1059
1060rtems-libbsd File:      rtems-bsd-sx.c
1061FreeBSD File:           kern/kern_sx.c
1062Description:            Status: USB, Nic
1063
1064rtems-libbsd File:      rtems-bsd-synch.c
1065FreeBSD File:           kern/kern_synch.c
1066Description:
1067Status:                 USB, Nic
1068
1069rtems-libbsd File:      rtems-bsd-syscalls.c
1070FreeBSD File:           User API for kern/uipc_syscalls.c
1071Description:
1072Status:                 Nic
1073
1074rtems-libbsd File:      rtems-bsd-sysctlbyname.c
1075FreeBSD File:           User API for sysctlbyname(3)
1076Description:
1077Status:
1078
1079rtems-libbsd File:      rtems-bsd-sysctl.c
1080FreeBSD File:           User API for sysctl(8)
1081Description:
1082Status:
1083
1084rtems-libbsd File:      rtems-bsd-sysctlnametomib.c
1085FreeBSD File:           User API for sysctlnametomib
1086Description:
1087Status:
1088
1089rtems-libbsd File:      rtems-bsd-taskqueue.c
1090FreeBSD File:           kern/subr_taskqueue.c
1091Description:
1092Status:                 Nic
1093
1094rtems-libbsd File:      rtems-bsd-thread.c
1095FreeBSD File:           kern/kern_kthread.c
1096Description:
1097Status:                 USB, Nic
1098
1099rtems-libbsd File:      rtems-bsd-timeout.c
1100FreeBSD File:           kern/kern_timeout.c
1101Description:
1102Status:                 Nic
1103
1104rtems-libbsd File:      rtems-bsd-timesupport.c
1105FreeBSD File:           kern/kern_clock.c
1106Description:
1107Status:                 Nic
1108
1109rtems-libbsd File:      rtems-bsd-vm_glue.c
1110FreeBSD File:           vm/vm_glue.c
1111Description:
1112Status:                 USB, Nic
1113----
1114
1115== Notes by File ==
1116
1117altq_subr.c - Arbitrary choices were made in this file that RTEMS would
1118not support tsc frequency change.  Additionally, the clock frequency
1119for machclk_freq is always measured for RTEMS.
1120
1121conf.h - In order to add make_dev and destroy_dev, variables in the cdev
1122structure that were not being used were conditionally compiled out. The
1123capability of supporting children did not appear to be needed and was
1124not implemented in the rtems version of these routines.
1125
1126== NICs Status ==
1127
1128----
1129Driver                  Symbol                          Status
1130======                  ======                          ======
1131RealTek                 _bsd_re_pcimodule_sys_init      Links
1132EtherExpress            _bsd_fxp_pcimodule_sys_init     Links
1133DEC tulip               _bsd_dc_pcimodule_sys_init      Links
1134Broadcom BCM57xxx       _bsd_bce_pcimodule_sys_init     Links
1135Broadcom BCM4401        _bsd_bfe_pcimodule_sys_init     Links
1136Broadcom BCM570x        _bsd_bge_pcimodule_sys_init     Needs Symbols (A)
1137E1000 IGB               _bsd_igb_pcimodule_sys_init     Links
1138E1000 EM                _bsd_em_pcimodule_sys_init      Links
1139Cadence                 ?                               Links, works.
1140----
1141
1142Symbols (A)
1143         pci_get_vpd_ident
1144
1145=== Cadence ===
1146
1147The cadence driver works on the Xilinx Zynq platform. The hardware checksum
1148support works on real hardware but does not seem to be supported on qemu
1149therefore the default state is to disable TXCSUM and RXCSUM and this can be
1150enabled from the shell with:
1151
1152  # ifconfig cgem0 rxcsum txcsum
1153
1154or with an ioctl call to the network interface driver with SIOCSIFCAP and the
1155mask IFCAP_TXCSUM and IFCAP_RXCSUM set.
1156
1157== Problems to report to FreeBSD ==
1158
1159The MMAP_NOT_AVAILABLE define is inverted on its usage.  When it is
1160defined the mmap method is called. Additionally, it is not used
1161thoroughly. It is not used in the unmap portion of the source.
1162The file rec_open.c uses the define MMAP_NOT_AVAILABLE to wrap
1163the call to mmap and file rec_close.c uses the munmap method.
Note: See TracBrowser for help on using the repository browser.