#2855 assigned enhancement

Next generation of RTEMS documentation

Reported by: Sebastian Huber Owned by: Sebastian Huber
Priority: normal Milestone: Indefinite
Component: doc Version: 5
Severity: normal Keywords:
Cc: Blocked By:
Blocking:

Description

The RTEMS documentation is inconsistent, confusing and outdated. This is the master ticket to define the next generation of RTEMS documentation. Individual work packages should be done in separate tickets.

Documentation audience

Who are the consumers of RTEMS documentation?

Users
People developing applications with RTEMS.
Developers
People doing RTEMS development, e.g. add/maintain BSPs, CPU ports, tools, etc.
Standards
Some documentation must be provided simply because some standard may require it.
Reviewers
People that review RTEMS to figure out if documentation, source code and executed test programs harmonize and meet certain conditions.

Existing documentation

README files

#2854

Documents

What to do with the following documents?

  • Getting Started with RTEMS
  • RTEMS Applications C User's Guide
  • RTEMS POSIX API User's Guide
  • RTEMS TCP/IP Networking Supplement
  • RTEMS Shell User's Guide
  • RTEMS Applications Ada User's Guide
  • RTEMS BSP and Device Driver Development Guide
  • RTEMS CPU Architecture Supplement
  • RTEMS Development Environment Guide
  • RTEMS Porting Guide
  • RTEMS POSIX 1003.1 Compliance Guide
  • RTEMS Filesystem Design Guide

Wiki

Purpose?

Doxygen

Purpose?

Duplicates with user manual?

Availability

Where is the documentation available?

Provide automatic documentation generation?

Standard compliance

How do deal with standard compliance, e.g. ECSS‐E‐ST‐40C?

Some standards require an excessive amount of documents. One option to simplify this would be a XML file which defines and documents the RTEMS interfaces (data types, functions, defines, macros, etc.) and then use a generator to produce header (maybe even with Doxygen comments) files, documentation files and also test cases.

Change History (11)

comment:1 Changed on Dec 22, 2016 at 12:34:09 AM by Chris Johns

In regards to the ESA link, the license needs some clarification. We have ECSS members and non-members and it is not clear to me what the actual intent of the license is. Also accessing the docs needs a login and password and I am not sure what is involved in getting them, however RTEMS is open source so should we reference "behind a wall" docs? I would prefer we do not if that can be done.

The wiki needs to be general and contain information not version or release specific information. We cannot snapshot the wiki and provide it as part of a release. Take for example the removal of the SIS BSP in 4.12, that BSP still exists in 4.11, 4.10 etc, so if the BSP documentation, generated, post processed or otherwise is in the wiki do you delete all references to the SIS in the wiki so the content is valid for 4.12 or do we leave the information in the wiki for those 4.11, 4.10, etc users who still have that BSP. A wiki great for developer documentation and workflows, or fluid things like a step through of booting a PC with PXE, iPXE, etc to RTEMS.

Doxygen. I do not agree with formal API documentation being generated from the source for users. A user manual for a formal API needs to be user focused with all error, conditions of use, and real examples. An API interface is a formal contract between the developers of RTEMS and users that these interfaces are stable and do not change. There will be duplication for API functions however I do not see this as a problem because the churn on these interfaces should be almost nothing. For internal or developer documentation doxygen is great, there are others such as Linux Kernel Cross-Reference (LXR). I use http://fxr.watson.org/ from time to time.

Finally I would like say how excited I am by the chat, tickets and focus on the documentation. If getting the docs with what ever content into Sphinx has helped start this process then I am happy.

comment:2 in reply to:  1 Changed on Dec 23, 2016 at 8:34:04 AM by Sebastian Huber

Replying to chrisj:

In regards to the ESA link, the license needs some clarification. We have ECSS members and non-members and it is not clear to me what the actual intent of the license is. Also accessing the docs needs a login and password and I am not sure what is involved in getting them, however RTEMS is open source so should we reference "behind a wall" docs? I would prefer we do not if that can be done.

I think everyone can get access to this site. You are probably not allowed to redistribute the documents without permission.

Other standards are more difficult to access, e.g. IEC 61508 or DO-178B.

The wiki needs to be general and contain information not version or release specific information. We cannot snapshot the wiki and provide it as part of a release. Take for example the removal of the SIS BSP in 4.12, that BSP still exists in 4.11, 4.10 etc, so if the BSP documentation, generated, post processed or otherwise is in the wiki do you delete all references to the SIS in the wiki so the content is valid for 4.12 or do we leave the information in the wiki for those 4.11, 4.10, etc users who still have that BSP. A wiki great for developer documentation and workflows, or fluid things like a step through of booting a PC with PXE, iPXE, etc to RTEMS.

Ok, so the BSP pages from the wiki should move into BSP README files.

Doxygen. I do not agree with formal API documentation being generated from the source for users. A user manual for a formal API needs to be user focused with all error, conditions of use, and real examples. An API interface is a formal contract between the developers of RTEMS and users that these interfaces are stable and do not change. There will be duplication for API functions however I do not see this as a problem because the churn on these interfaces should be almost nothing. For internal or developer documentation doxygen is great, there are others such as Linux Kernel Cross-Reference (LXR). I use http://fxr.watson.org/ from time to time.

It would be really nice to generate the header files with Doxygen comments and the sphinx parts from one common file, e.g. in XML. This could be also used to contain the specification and test cases.

Finally I would like say how excited I am by the chat, tickets and focus on the documentation. If getting the docs with what ever content into Sphinx has helped start this process then I am happy.

comment:3 Changed on Dec 23, 2016 at 9:04:09 AM by Sebastian Huber

Including BSP README files from the source directory in a sphinx document seems to be a bit hard:

http://stackoverflow.com/questions/10199233/can-sphinx-link-to-documents-that-are-not-located-in-directories-below-the-root

comment:4 Changed on Dec 23, 2016 at 9:04:23 AM by Sebastian Huber

What are the hard technical reasons to have a separate documentation repository?

comment:5 Changed on Dec 23, 2016 at 9:52:29 AM by Sebastian Huber

I propose to do the following:

  1. Move the documentation back into the RTEMS sources.
  1. Move the BSP README files to the documentation directory into the CPU Supplement.

comment:6 in reply to:  5 ; Changed on Jan 4, 2017 at 11:16:51 PM by Chris Johns

Replying to sebastian.huber:

I propose to do the following:

  1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

  1. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

Last edited on Jan 4, 2017 at 11:21:41 PM by Chris Johns (previous) (diff)

comment:7 in reply to:  6 ; Changed on Jan 5, 2017 at 9:31:14 AM by Sebastian Huber

Replying to chrisj:

Replying to sebastian.huber:

I propose to do the following:

  1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

Is this a hard buildbot limitation? Why can't you teach buildbot that there are separate build systems/components in different directories? We don't have that many commits per second. Is this a real problem?

I would even consider to move all parts that are required to work with RTEMS and that are maintained by the RTEMS project into one repository, e.g. the RSB and the RTEMS Tools.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

For the space qualification of RTEMS SMP we have to produce a lot of different documents. I didn't spend much time to this topic so far, but I guess that without automatic generation from a common source (XML documents) this will be infeasible to deliver.

  1. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

Yes, we definitely have to reduce the amount of different documents. Maybe we should move the BSP stuff into the getting started guide. Somewhere there should be a list of supported chips/boards (section) grouped by architecture (chapter). The quirks, build options, and so on (subsections) should be available for each BSP.

comment:8 in reply to:  7 Changed on Jan 5, 2017 at 9:52:27 PM by Chris Johns

Replying to sebastian.huber:

Replying to chrisj:

Replying to sebastian.huber:

I propose to do the following:

  1. Move the documentation back into the RTEMS sources.

Please do not. There is more to the project's documentation than the kernel and doing this would result in commit churn.

How a user uses the RSB or Eclipse is valid user documentation and it is not related to the kernel. Having this type of documentation in the unrelated kernel source repo would cause 2-way commit churn where user doc changes unrelated to the kernel would require buildbot rebuilds of the kernel and kernel changes would require documentation rebuilds.

Is this a hard buildbot limitation? Why can't you teach buildbot that there are separate build systems/components in different directories? We don't have that many commits per second. Is this a real problem?

Currently yes it is. This adds extra complexity to something we have not got working and I am concerned about adding extra complexity.

I would even consider to move all parts that are required to work with RTEMS and that are maintained by the RTEMS project into one repository, e.g. the RSB and the RTEMS Tools.

I suppose you mean creating a world repo. I can see this being possible once the autotools build system goes and we have switched to waf. Doing so before that happens would create a level of complexity I would not like to consider. As you say we could move the RSB, RTEMS Tools, examples into a single repo. I am not sure about networking, ie the legacy and libbsd stacks, plus third party packages. They would need some sort of ports database.

I would like to add moving to an RTEMS world is a different topic and bringing the docs repo into a single "world" repo makes more sense. I see doing this now as premature.

For me having the documentation source in the kernel source tree so a specific part of the documentation could be automatically updated is a questionable small gain at a larger cost. I question any formal user API documentation we have being automatically generated this way for a few reasons. It becomes difficult to audit changes because you need to de-reference the location in the documentation source to the location in the kernel source tree and I feel formal user API interfaces should be fixed and cannot change without careful consideration rather than being seen as kernel source file changes. Second, we have more than one place documentation can be located which means more complexity in how we maintain the documentation, and finally we would need to find a suitable format for use because there is no tool I know of that can do a suitable doxygen type extraction to ReST.

For the space qualification of RTEMS SMP we have to produce a lot of different documents.

It is difficult to discuss something like qualification documentation when we do not know what is needed. I doubt qualification documentation requires a suitable Getting Started Guide or how to use Eclipse, these are user focused. In relation to qualification artifact generation we should endeavor to build into our processes and tools support to generate the data the artifacts need, for example the XML reports created by the RSB.

I didn't spend much time to this topic so far, but I guess that without automatic generation from a common source (XML documents) this will be infeasible to deliver.

Sorry, I do not know what you mean by a common source and XML documents. I do not know what the requirements are you are considering here.

If qualifying requires extra information that is internal then it can be automatically generated from source. Again I would like to make a clear distinction between user API documentation and anything internal. They are not the same thing and do not or even should not be grouped together.

  1. Move the BSP README files to the documentation directory into the CPU Supplement.

I wonder if we should have a single document for BSPs and this be a chapter. I do not see a need for lots of separated manuals.

Yes, we definitely have to reduce the amount of different documents. Maybe we should move the BSP stuff into the getting started guide. Somewhere there should be a list of supported chips/boards (section) grouped by architecture (chapter). The quirks, build options, and so on (subsections) should be available for each BSP.

Should BSPs be a section in the User Manual?

I am currently adding a Host Tools section to cover rtems-ld, rtems-exeinfo, rtems-syms, and rtems-tld. Then I will be adding a section on Applications to cover that topic.

comment:9 Changed on Aug 14, 2017 at 12:55:55 AM by Chris Johns

Milestone: 5.04.12.0

Please review and update the milestone. Thanks.

comment:10 Changed on Aug 24, 2017 at 8:22:55 AM by Sebastian Huber

Milestone: 4.12.0Indefinite
Owner: set to Sebastian Huber
Status: newassigned

comment:11 Changed on Oct 10, 2017 at 6:06:29 AM by Sebastian Huber

Component: Documentationdoc
Note: See TracTickets for help on using tickets.