libbsd: Reduce footprint of minimal buildset

[Christian Mauderer]

Overview

The RTEMS libbsd is now the default network stack for a lot of BSPs. But it's footprint is quite a bit larger then the one of the "legacy" network stack. In 2018 we introduced buildsets that allow to disable features. Currently only a few features can be disabled (OpenSSL, IPv6). Target of this project would be to find more non-essential features that are always linked in and allow to disable them.

As a general guideline: The minimal buildset of libbsd should provide about the functionality of the legacy network stack.

The work must not increase the testing effort that is necessary for a release. That should not be a big problem as long as no new buildsets or new build options are necessary. If something like this is necessary, it will need a discussion on the mailing list first.

Requirements

• A target where libbsd can be used. That can be either real hardware or a simulator with network support (for example xilinx_zynq_a9_qemu).

• If you have a real hardware target, you'll most likely need some JTAG debugger for it. On a simulator you can use gdb. You have to know how to use the debugger.

• Basic C coding skills and very good skills to read C code. You'll have to dig through FreeBSD code which can be quite complex!

• Quite a bit of knowledge how to analyze generated elf files. You should at least be able to use objdump and read linker map files.

• Ability to build your own RTEMS application outside of the normal test tree to test stuff during development.

Difficulty

This is a medium (175-hour) project of medium difficulty.

More Information

Read the comments for a lot more discussion on this topic.

[Chris Johns]

This is a nice idea.

I would prefer we leave the FreeBSD source as close as possible to the upstream sources. I believe this has major benefits over attempts to minimize the scope for RTEMS. The reality is libbsd will grow in size as the FreeBSD file, file descriptor and VFS code is merged into our code base. My simplistic view is the linker garbage collection is removing unused pieces of that code even if it is being built. Analysis and review of what is happening is a great idea but we need to find the balance between transparent sources, the size of the built code and the functionality being provide.

I would like to suggesting extending the scope to look at how we use libbsd and how run-time memory effects performance and usability. We do not have a good write up on the effects of different memory sizes provided to the networking stack, what sysctl variables do to memory usage verses performance and how to tune the code for use on RTEMS. There will be a mix of use cases, those with small memory and limited performance to those who need to run at wire speed. Our users are RTEMS users and helping with the complexity of the FreeBSD side of things would be a major help as libbsd is used more and more.

[Christian Mauderer]

Thanks for the suggestion. Just to make it clear: I wouldn't like big changes in the FreeBSD sources either. The target should _not_ be to replace existing functionality with new code. The target would be to find locations where the garbage collection doesn't work. For example: As soon as WLAN is enabled (net80211 = on) ifconfig needs some additional stuff. To prevent the linker pulling that always in there is a #ifdef in ifconfig.c:

#ifdef RTEMS_BSD_MODULE_NET80211

ieee80211_ctor();

#endif

I assume that the same is true for some other functionality. For example: Do we always need the "vlan" functionality in ifconfig? Or would it be possible to disable it?

Run-time memory effects are a lot harder to analyze. Of course that would be nice too but I'm not sure whether someone who is new to RTEMS has the insight that is necessary to analyze that or has the insight to see the use cases that we have.

[Joel Sherrill]

I had no idea that you were thinking of a garbage collection bug. I was thinking like Chris that this is tuning and tailoring in terms of sysctl settings, kernel configuration, etc. that a user could do to minimize footprint. I agree the linker magic will take care of built but unused functionality but some things are built-in and may be tailorable.

I was thinking this would result in examples and documentation explaining the tuning and tailoring. I wouldn't want to duplicate FreeBSD documentation so if the RTEMS guide can explain the tuning and tailoring process in RTEMS terms and refer to existing FreeBSD documentation for more details on specific topics, that would be good. But a single sentence that says "this is FreeBSD, see their documentation" will not be acceptable.

A challenge in this is defining what we consider a useful minimum profile. This was a challenge for RTEMS itself and it has evolved. My first shot would be only those features that were in the legacy stack with provision to allow for newer versions of RFCs. This turns off at least IPV6, IPSEC, VLANs, and packet filtering. This would allow a side by side comparison of legacy to new stack size change.

Features between legacy and full new capabilities would be turned on/tuned.

Dropping below legacy would require thinking to see what could be left out.

We should be clear what adds build time and what shows up in all applications when you turn it on. I don't think we care as much about build time as we do about what shows up in the application after all the linker magic.

[Christian Mauderer]

Seems that I might should have discussed this on the list before creating a ticket.

I wouldn't say that this kind of stuff are garbage collection bugs. We just pull in a lot of parts that we don't always need. It would be great if we could reduce that stuff. There definitively is quite some of it but no one really has time to remove it or rather make it possible to remove it by disabling modules. On the other hand it's something where not a lot of knowledge is necessary to start. So I thought it could be a good GSoC project.

On the other hand documentation projects are always a bit tricky in my opinion. Basically a student has ask on the list how something works, wait for an answer use that answer to write documentation or maybe copy it more or less directly. So the student is mostly providing a skeleton and forces us to write stuff that can be put into the skeleton.

We already discussed the minimal buildset when I introduced the build sets. If I remember correctly it's basically what you said: About the same functionality that has been provided by the legacy stack. The header of the minimal buildset also tells about that (just with some other words): ​https://git.rtems.org/rtems-libbsd/tree/buildset/minimal.ini

There could be multiple starting points for such a project. One could be to analyze a small generated binary that more or less only initialize the network stack and take a look whether (for example) it includes stuff that belongs to SD card, USB, Display, ...

A second approach could be to analyze which options of the FreeBSD build system we have set. Then we could enable them conditionally. For example that is already done with INET6. But for example I'm not sure whether we set VIMAGE and whether that might make some difference.

This would allow to make our libbsd a bit more modular. And I'm sure that there are some surprises about what is pulled in by default.

[Chris Johns]

We need to be mindful that build options add extra overheads to the project because we need to regression build and test the supported variations. This also follows through to having a suitable point to make a release.

[Christian Mauderer]

My target with this project idea would have been to bring the existing minimal buildset forward by removing more functionality and making it more similar to the legacy stack. This wouldn't add another configuration and therefore shouldn't add overhead. We already discussed when introducing buildsets for libbsd that there should be only few official ones that are officially supported. Everything in between might or might not work and can be maintained if someone of the core maintainers is paid to do the work. If you want, I can highlight that in the description.

Please note: If you don't think the project would be a good one for a GSoC that's OK too. I don't mind discarding it. I have seen the problem multiple times on the mailing list that our legacy stack is smaller and therefore preferred in some applications. Working on the minimal buildset for libbsd could help to find out whether it is possible to reduce the difference a bit. Currently I think no one has budget to do that and (I think) currently no one want's to put too much hobby time into it either. So it seemed like a nice project. But it's not really necessary.

[Chris Johns]

I think the project is a good one. It has challengers which is good. I also think it is good we discuss and are clear on the bounds that exist when working libbsd.

[Christian Mauderer]

I updated the tickets with some additional requirements. I hope that that makes the target more clear.

[Joel Sherrill]

Is there any intent in this task to capture the process of enabling/disabling specific features and setting the tuning parameters? And (hopefully) point to FreeBSD documentation for what the various specifics are. I'm looking at the minimal instructions from an RTEMS perspective.

I don't want the project itself to maintain lots of configurations but I can see some user deployments wanting to tailor it.

[Christian Mauderer]

My intention for this project are compile time parameters and similar that map to "modules" in libbsd (like IPv6 or IPSEC) and have an influence on code size. Run time parameters that are controlled for example via sysctl are something different and I don't think that mixing them is a good idea (*).

Identifying modules that are not always necessary and disabling them in the minimal configuration allows a user to create a buildset that is in between minimal and default. But like Chris already pointed out: We can't test every configuration. So that will always be a "hope it works" configuration. The official buildsets should only provide some useful and maintainable subsets. Currently we have:

• minimal -> As small as possible but still a network and (currently) USB stack. I would be happy if it is reduced to the functionality of the legacy stack. Later we might discuss whether some additional features should be switched off.
• default -> what most users want if they don't have a size problem
• everything -> everything that the stack can offer
• umass -> only USB mass storage support

(*) Also note: Documenting parameters while having something other as main task is most likely hard for a student because we currently don't have a basic frame for libbsd. Maybe just playing with different features and writing documentation could be a separate project? Can we have a GSoC project with no code but documentation as target?