Modular Network Stacks

[Gedare Bloom]

Modular Network Stacks

Contents

1. Modular Network Stacks
   1. Mentors
   2. Students
   3. Status
2. Introduction
   1. Prerequisites
   2. Resources
3. Tasks
   1. Legacy Stack
   2. lwIP
   3. libbsd
4. Testing

Mentors

Chris Johns, Sebastian Huber, Pavel Pisa, Joel Sherrill

Students

TBD

Status

Looking for funding/motivated students.

Introduction

The goal of this project is to facilitate use of networking stacks in a more modular fashion.

RTEMS has 3 commonly used networking stacks known as:

• legacy (cpukit/libnetworking): used primarily by established users with networking needs. The most visible of these is the ​EPICS Project.
• libbsd (git.rtems.org/rtems-libbsd.git): state-of-the-art for high-performance networking. It is the recommended stack for new development for systems with sufficient resources to run the stack.
• lwIP: useful for low-resource (usually, RAM) systems to use.

Other networking stacks are also possible to use, but these are the mainly interesting ones.

Prerequisites

• Ability to read/write C programs (pointer-based data structures).
• Ability to debug C programs.
• Ability to read/write Python programs with OOP.
• Ability to compile and run networking applications in *nix and RTEMS.
• Ability to use Wireshark to view network packets.
• Knowledge of one or more assembly languages.
• Knowledge of OSI model.
• Knowledge of hardware and device drivers.

Resources

• Current RTEMS developers.
• ​RTEMS Networking Manual for the legacy stack.
• Packages/LWIP describes some of the prior work with lwIP.
• ​Ragunath's Blog describes his experience using lwIP with RTEMS on the Beaglebone Black.
• ​Beej's Guide to Network Programming is an excellent resource to understand the interface provided by networking stacks.

Tasks

The following are the high-level tasks to accomplish:

Legacy Stack

The legacy stack is loosely coupled to RTEMS. It is built from within rtems.git/cpukit/libnetworking when the --enable-networking option is given during the configure stage of building RTEMS. This subdirectory needs to be factored out of rtems.git and put in its own standalone repository that can be compiled separately from compiling RTEMS. Then, the build process for the legacy network stack should be added to the RSB.

lwIP

lwIP is meant to be compiled and customized with baremetal and RTOS code bases. The main problem with lwIP is that too much customization can make it hard to maintain, and then the port of lwIP bitrots and is no longer usable with RTEMS. Thus, the integration of lwIP with RTEMS needs to be made more seamless and maintainable. One of the first steps in that direction is to revive and fix the RSB recipe for building lwIP, see #3770 for more information. Other problems raised in the Packages/LWIP page are also relevant, and the integration may also require adding any missing bits to newlib as mentioned in #2833. It is important to make a tight integration between lwIP and RTEMS to avoid excessive indirection overhead, while balancing with the need to maintain and update the lwIP package.

libbsd

Not much needs to be done here, other than to avoid breaking anything. The libbsd already builds via RSB as a standalone module that can be linked with RTEMS.

Testing

A network-ready target and application should be used for testing. Consult with mentors during the planning phase. The most likely candidate is to use Qemu with a simple networking example.

[Gedare Bloom]

From Joel on the users mailing list:

One thing to be aware of is that all the POSIX networking header files for RTEMS are in newlib and always present. I had to address this and lwip when we did Deos+RTEMS. Deos uses lwip as their native stack running in a partition and other partitions use a client to get to it. The lwip constants had values that were not the same as the RTEMS BSD headers for POSIX defines. There were also some places where the structure definitions did not align. I had to write a bit of mapping in the client. When lwip works at all, it would be awesome to have a way for it to ignore their own minimal POSIX API files and build against ours.

This would be similar to how the newlib headers define a very complete POSIX API set but each target OS may only support a subset of it.

As it is, I wonder if there is a conflict between the RTEMS newlib network .h files and those provided by lwip which could cause issues.

[Joel Sherrill]

More lwip thoughts. I think lwip needs to have an rtems-lwip repository which is managed like rtems-libbsd. lwip has some issues which would be solved by doing this:

• Drivers are not integrated and are scattered across the net. rtems-lwip could serve as a gathering point for the ones users need.

• Build system could be ours to at least wrap, if not replace, theirs.

• Allows place for RTEMS specific examples and tests.

[Vijay Kumar Banerjee <vijay@…>]

In c6ba8dd/rtems:

spec: Remove RTEMS_NETWORKING options

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In a28eafb/rtems:

cpukit: Move ftpfs from libnetworking to libfs

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 2a13ad3/rtems:

cpukit: Remove libnetworking

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 369e6c5b/rtems:

cpukit: remove pppd

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 6692e03/rtems:

cpukit/libfs: Remove nfsclient

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In c90fa83/rtems:

bsps: Remove networking drivers

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 3dca9ed/rtems:

testsuites/libtests: Remove networking01

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 3299dda2/rtems:

cpukit: Remove telnetd

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 633526d2/rtems:

spec: Remove residue bsps spec files

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In df6d765c/rtems:

testsuites: Remove all legacy networking tests

Update #3850

[Vijay Kumar Banerjee <vijay@…>]

In 8383572/rtems:

cpukit/include: Remove telnetd.h

Update #3850

[Gedare Bloom]

A lot of progress has been made on this, but a lot more remains especially for the lwIP drivers.

[Kinsey Moore]

Since the last update on this issue, rtems-lwip has been promoted to the top level on git.rtems.org along with rtems-net-services which will house stack-agnostic network libraries. It already houses the NTP library/daemon and the TTCP library.

[Joel Sherrill]

I think moving the webserver (mghttpd), telnetd, and ftpd from rtems.git to network services is likely the next step.

There are a couple of things which are easy to overlook. The option of multiple network stacks plus services is likely not yet covered in the User Guide.

Also, this reminds me that libbsd, lwip, and network services do not have Coverity Scan jobs yet. This falls on me since I have Coverity on rtems.git and newlib set up locally via cron.