Next generation of RTEMS documentation

[Sebastian Huber]

The RTEMS documentation is inconsistent, confusing and outdated. This is the master ticket to define the next generation of RTEMS documentation. Individual work packages should be done in separate tickets.

Documentation audience

Who are the consumers of RTEMS documentation?

Users

People developing applications with RTEMS.

Developers

People doing RTEMS development, e.g. add/maintain BSPs, CPU ports, tools, etc.

Standards

Some documentation must be provided simply because some standard may require it.

Reviewers

People that review RTEMS to figure out if documentation, source code and executed test programs harmonize and meet certain conditions.

Existing documentation

README files

#2854

Documents

What to do with the following documents?

• Getting Started with RTEMS
• RTEMS Applications C User's Guide
• RTEMS POSIX API User's Guide
• RTEMS TCP/IP Networking Supplement
• RTEMS Shell User's Guide
• RTEMS Applications Ada User's Guide
• RTEMS BSP and Device Driver Development Guide
• RTEMS CPU Architecture Supplement
• RTEMS Development Environment Guide
• RTEMS Porting Guide
• RTEMS POSIX 1003.1 Compliance Guide
• RTEMS Filesystem Design Guide

Wiki

Purpose?

Doxygen

Purpose?

Duplicates with user manual?

Availability

Where is the documentation available?

Provide automatic documentation generation?

Standard compliance

How do deal with standard compliance, e.g. ​ECSS‐E‐ST‐40C?

Some standards require an excessive amount of documents. One option to simplify this would be a XML file which defines and documents the RTEMS interfaces (data types, functions, defines, macros, etc.) and then use a generator to produce header (maybe even with Doxygen comments) files, documentation files and also test cases.

[Chris Johns]

In regards to the ESA link, the license needs some clarification. We have ECSS members and non-members and it is not clear to me what the actual intent of the license is. Also accessing the docs needs a login and password and I am not sure what is involved in getting them, however RTEMS is open source so should we reference "behind a wall" docs? I would prefer we do not if that can be done.

The wiki needs to be general and contain information not version or release specific information. We cannot snapshot the wiki and provide it as part of a release. Take for example the removal of the SIS BSP in 4.12, that BSP still exists in 4.11, 4.10 etc, so if the BSP documentation, generated, post processed or otherwise is in the wiki do you delete all references to the SIS in the wiki so the content is valid for 4.12 or do we leave the information in the wiki for those 4.11, 4.10, etc users who still have that BSP. A wiki great for developer documentation and workflows, or fluid things like a step through of booting a PC with PXE, iPXE, etc to RTEMS.

Doxygen. I do not agree with formal API documentation being generated from the source for users. A user manual for a formal API needs to be user focused with all error, conditions of use, and real examples. An API interface is a formal contract between the developers of RTEMS and users that these interfaces are stable and do not change. There will be duplication for API functions however I do not see this as a problem because the churn on these interfaces should be almost nothing. For internal or developer documentation doxygen is great, there are others such as Linux Kernel Cross-Reference (LXR). I use ​http://fxr.watson.org/ from time to time.

Finally I would like say how excited I am by the chat, tickets and focus on the documentation. If getting the docs with what ever content into Sphinx has helped start this process then I am happy.

[Sebastian Huber]

Replying to chrisj:

In regards to the ESA link, the license needs some clarification. We have ECSS members and non-members and it is not clear to me what the actual intent of the license is. Also accessing the docs needs a login and password and I am not sure what is involved in getting them, however RTEMS is open source so should we reference "behind a wall" docs? I would prefer we do not if that can be done.

I think everyone can get access to this site. You are probably not allowed to redistribute the documents without permission.

Other standards are more difficult to access, e.g. IEC 61508 or DO-178B.

The wiki needs to be general and contain information not version or release specific information. We cannot snapshot the wiki and provide it as part of a release. Take for example the removal of the SIS BSP in 4.12, that BSP still exists in 4.11, 4.10 etc, so if the BSP documentation, generated, post processed or otherwise is in the wiki do you delete all references to the SIS in the wiki so the content is valid for 4.12 or do we leave the information in the wiki for those 4.11, 4.10, etc users who still have that BSP. A wiki great for developer documentation and workflows, or fluid things like a step through of booting a PC with PXE, iPXE, etc to RTEMS.

Ok, so the BSP pages from the wiki should move into BSP README files.

Doxygen. I do not agree with formal API documentation being generated from the source for users. A user manual for a formal API needs to be user focused with all error, conditions of use, and real examples. An API interface is a formal contract between the developers of RTEMS and users that these interfaces are stable and do not change. There will be duplication for API functions however I do not see this as a problem because the churn on these interfaces should be almost nothing. For internal or developer documentation doxygen is great, there are others such as Linux Kernel Cross-Reference (LXR). I use ​http://fxr.watson.org/ from time to time.

It would be really nice to generate the header files with Doxygen comments and the sphinx parts from one common file, e.g. in XML. This could be also used to contain the specification and test cases.

Finally I would like say how excited I am by the chat, tickets and focus on the documentation. If getting the docs with what ever content into Sphinx has helped start this process then I am happy.

[Sebastian Huber]

Including BSP README files from the source directory in a sphinx document seems to be a bit hard:

​http://stackoverflow.com/questions/10199233/can-sphinx-link-to-documents-that-are-not-located-in-directories-below-the-root

[Sebastian Huber]

What are the hard technical reasons to have a separate documentation repository?

[Sebastian Huber]

I propose to do the following:

1. Move the documentation back into the RTEMS sources.

2. Move the BSP README files to the documentation directory into the CPU Supplement.

[Chris Johns]

Replying to sebastian.huber:

I propose to do the following:

1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

2. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

[Sebastian Huber]

Replying to chrisj:

Replying to sebastian.huber:

I propose to do the following:

1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

Is this a hard buildbot limitation? Why can't you teach buildbot that there are separate build systems/components in different directories? We don't have that many commits per second. Is this a real problem?

I would even consider to move all parts that are required to work with RTEMS and that are maintained by the RTEMS project into one repository, e.g. the RSB and the RTEMS Tools.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

For the space qualification of RTEMS SMP we have to produce a lot of different documents. I didn't spend much time to this topic so far, but I guess that without automatic generation from a common source (XML documents) this will be infeasible to deliver.

2. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

Yes, we definitely have to reduce the amount of different documents. Maybe we should move the BSP stuff into the getting started guide. Somewhere there should be a list of supported chips/boards (section) grouped by architecture (chapter). The quirks, build options, and so on (subsections) should be available for each BSP.

[Chris Johns]

Replying to sebastian.huber:

Replying to chrisj:

Replying to sebastian.huber:

I propose to do the following:

1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

Is this a hard buildbot limitation? Why can't you teach buildbot that there are separate build systems/components in different directories? We don't have that many commits per second. Is this a real problem?

Currently yes it is. This adds extra complexity to something we have not got working and I am concerned about adding extra complexity.

I would even consider to move all parts that are required to work with RTEMS and that are maintained by the RTEMS project into one repository, e.g. the RSB and the RTEMS Tools.

I suppose you mean creating a world repo. I can see this being possible once the autotools build system goes and we have switched to waf. Doing so before that happens would create a level of complexity I would not like to consider. As you say we could move the RSB, RTEMS Tools, examples into a single repo. I am not sure about networking, ie the legacy and libbsd stacks, plus third party packages. They would need some sort of ports database.

I would like to add moving to an RTEMS world is a different topic and bringing the docs repo into a single "world" repo makes more sense. I see doing this now as premature.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

For the space qualification of RTEMS SMP we have to produce a lot of different documents.

It is difficult to discuss something like qualification documentation when we do not know what is needed. I doubt qualification documentation requires a suitable Getting Started Guide or how to use Eclipse, these are user focused. In relation to qualification artifact generation we should endeavor to build into our processes and tools support to generate the data the artifacts need, for example the XML reports created by the RSB.

I didn't spend much time to this topic so far, but I guess that without automatic generation from a common source (XML documents) this will be infeasible to deliver.

Sorry, I do not know what you mean by a common source and XML documents. I do not know what the requirements are you are considering here.

If qualifying requires extra information that is internal then it can be automatically generated from source. Again I would like to make a clear distinction between user API documentation and anything internal. They are not the same thing and do not or even should not be grouped together.

2. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

Yes, we definitely have to reduce the amount of different documents. Maybe we should move the BSP stuff into the getting started guide. Somewhere there should be a list of supported chips/boards (section) grouped by architecture (chapter). The quirks, build options, and so on (subsections) should be available for each BSP.

Should BSPs be a section in the User Manual?

I am currently adding a Host Tools section to cover rtems-ld, rtems-exeinfo, rtems-syms, and rtems-tld. Then I will be adding a section on Applications to cover that topic.

[Chris Johns]

Please review and update the milestone. Thanks.

[Joel Sherrill]

This ticket hasn't had a comment in approaching four years. A lot of documentation work has been done. This doesn't appear to have a specific goal which would allow it to ever close.

What to do with it?

[Sebastian Huber]

The goals of the ticket are too unspecific.