Raspberry Pi Framebuffer and HDMI Video Support

Introduction

Raspberry Pi is a series of popular, low-cost credit card-sized single-board computers, based on an ARM cpu, with the intention of promoting the teaching of basic computer science in schools. It would be great for both the project and RTOS education if RTEMS can have a full support on raspberry pi. The goal of the project is to improve the RTEMS board support packages for Raspberry Pi.

For my part, I will implement Framebuffer Support for RPI BSP, using its GPU. That would allow us to use HDMI port for graphic output and run graphic desktop environment. Finally, I'll port the graphic libraries from project items-graphic-tool-kits to rtems-source-builder so that we can build add-on graphic libraries much easier.

Arrangement

• Mentor: Pavel Pisa
• Student: YANG Qiao <yangqiao AT me DOT com>
• Periode: 25/05/2015 - 21/08/2015

Progress

Mailbox

Mailboxes are the primary means of communication between the ARM and the Video Core firmware running on the GPU.

The data is 32-bit long value. The upper 28 bits are reserved for data and the lower 4 bits represent the number of channel to use. Two functions are available in header <bsp/mailbox.h>

extern unsigned int  raspberrypi_mailbox_read(unsigned int channel);
extern void raspberrypi_mailbox_write(unsigned int channel, unsigned int data);

With the exception of the property tags mailbox channel, when passing memory addresses as the data part of a mailbox message, the addresses should be bus addresses as seen from the VC. These vary depending on whether the L2 cache is enabled. So in header file bsp.h the macro RPI_L2_CACHE_ENABLE should be set to 0 if the L2 cache is disabled in config.txt, so that the mailbox will use the right memory mapping.

Video Core

In order to access some properties of video core GPU, we may use the mailbox properties tag channel.

Not all properties are implemented yet, but I've added all of the tag definitions and useful macros in file vc_defines.h so that we can implement the features easily when we need. The interface is defined

in header vc.h, an entry structure and a function for each property. The request parameters should be set in the structure, and result of the query will update the structure after calling the related function.

In order to sed a request to mailbox property tags channel, All message buffers should start with a common header, called 'hdr'. Then it should be followed by a list of tags. An additional empty tag should be added at the end, called 'end_tag'. The struct should be 16-byte aligned. For ex:

struct {
    bcm2835_mbox_buf_hdr hdr;
    bcm2835_mbox_tag_display_size set_display_size;
    bcm2835_mbox_tag_virtual_size set_virtual_size;
    bcm2835_mbox_tag_depth set_depth;
    bcm2835_mbox_tag_pixel_order set_pixel_order;
    bcm2835_mbox_tag_alpha_mode set_alpha_mode;
    bcm2835_mbox_tag_virtual_offset set_virtual_offset;
    bcm2835_mbox_tag_overscan set_overscan;
    bcm2835_mbox_tag_allocate_buffer allocate_buffer;
    bcm2835_mbox_tag_get_pitch get_pitch;
    uint32_t end_tag;
  } buffer __attribute__((aligned (16)));

Tags should be processed in order except where an interface requires multiple tags for a single operation (like the frame buffer)

All tags should start with a common header called tag_hdr.Then it should be followed by a data buffer for request and response. For ex:

typedef struct {
  bcm2835_mbox_tag_hdr tag_hdr;
  union {
    struct {
      uint32_t width;
      uint32_t height;
    } req;
    struct {
      uint32_t width;
      uint32_t height;
    } resp;
  } body;
} bcm2835_mbox_tag_display_size;

Since the response overwrites the request. we use union in the structure for the request and response.

You may find more details on ​official github wiki.

cmdline

At the first stage of boot, gpu will read the text file config.txt for a BIOS-like configuration, it will also read the single-line text file cmdline.txt and append the content to the kernel command line which will be passed by ATAG structure to kernel later.

As documented in arm cpu documentation, when the kernel started, register r2 will contain the address of ATAG structures or the devicetree. For the moment, I’ve used the VC interface to get cmdline instead of parsing ATAG and devicetree. As the boot of VC is the early stage, if we use a bootloader linke UBOOT and append more kernel arguments in cmdline, this may cause problems. So for a long term solution, we may implement a more generic way for all arm bsps.

The cmdline is initialization by calling rpi_init_cmdline() at the early boot. The we can get the full cmdline by rpi_cmdline()or a substring started with a given string rpi_cmdline_arg(const char* arg). These functions are decalred in header file <bsp.h>.

void rpi_init_cmdline(void);
const char* rpi_cmdline(void);
const char* rpi_cmdline_arg(const char* arg);

It would be great if we can have the support for ATAG and devicetree for arm cpu.

frame buffer

The frame buffer is allocated by GPU and in order to get the frame buffer base pointer, we will use the mailbox channel to allocate memory. Since the mailbox0 channel 1, which is reserved for frame buffer is reported to be unstable, we determined to use the property tags channel instead. So the allocate process can be referred to the use of video core interface, precised in the section before. Once we get the base pointer and the size of memory for frame buffer, we can simply write the pixels' data in a proper format into the right position and normally something will be shown on display. With the frame buffer driver, we can access the fb device in user space and connect it with other graphic applications.

As the cmdline is supported , we are allowed to pass arguments to choose the resolution of framebuffer for display. The command line argument format is

--video=<resX>x<resY>[-<bpp>]

where numbers <resX>, <resY> and <bpp> are decadic. For the moment I've only implemented for bpp=32, (32BIT_X888RGB pixel format) which is the most commonly used, so the <bpp> will be ignored. --video=auto can be used for an auto-detection and auto-configuration.

What is complicated here is the memory management. On raspberry pi, arm cpu and video core gnu share the same memory. The gnu memory size can be setup in config.txt, with a minimum 16M with a step of 8M. The rest of memory will be used for arm cpu. In order to make use of the full memory on board, we have to configure the physic memory size in file /c/src/lib/libbsp/arm/raspberrypi/startup/linkcmds.

MEMORY {
        VECTOR_RAM     (AIW) : ORIGIN = 0x0       , LENGTH = 0x8000
        GPU_RAM        (AIW) : ORIGIN = 256M - 64M, LENGTH = 64M
        RAM            (AIW) : ORIGIN = 0x00008000, LENGTH = 256M - 64M - 48K
	RAM_MMU        (AIW) : ORIGIN = 256M - 64M - 16k, LENGTH = 16k
}

Here is a setup for a rip with maximum 256M memory and 64M will be used for GPU (by default). If you are using 512M board, please replace all 256M by 512M so that the memory can be fully used.

With the help of video core interface, we can get the range of memory for arm cpu and video core gpu. In the early stage of boot, we will query the start point of video core gpu, and limit the range of work area so that we won't access the memory of gpu. The frame buffer area is allocated inside of gpu memory. In order to have access we should also add an additional entry in mmu table.

video output char

At the early stage of boot, _RPI_initVideo() will be called for the initialization of framebuffer and prepare for drawing ASCII characters on the screen. After the init is done, we can have the access to maxCol and maxRow, which stand for the number of characters in a row and the number of rows in the display.

_RPI_outch(char) can be used to output a character on the screen. The scroll of the screen and the ESC sequences are handled automatically by a state machine. The drawing algorithm is implemented with the help of pre-defined table video_font_draw_table32 (32bit X888 ARGB format). A binary font file is used to print ASCII characters. The front ground color and the back ground color can be adjusted by macros : CONSOLE_FG_COL and CONSOLE_BG_COL which is used only for graphic text console.

framebuffer console

Instead of using serial port, we are now able to redirect all to a framebuffer console (fbcons) and use a graphic text console. To choose the fb console, use cmdline argument --console=fbcons, otherwise we will use the UART serial console by default.

Add-on graphic libraries

To build add-on graphic libraries, you just need to build the bsets you want.

Attention: If you are using cross-compiling, no installation will be performed. You may need --pkg-tar-files to creat tar packages and install them manually. Since microwindows and nxlib depend on previous libraries, they should be installed afterward.

Remarks

• If you encounter problems, you may update the firmware to the latest or report on mailing list.

References

• ​Blog
• ​rtems Github Repo
• ​rtems-source-builder Github Repo
• ​rtems-tool Github Repo