Changes between Version 3 and Version 4 of TBR/UserManual/DriverManager

Timestamp:

06/11/09 18:16:26 (15 years ago)

Author:

Hellstrom

Comment:

TBR/UserManual/DriverManager

@@ -6 +6 @@
 This document describes the Driver Manager patch for RTEMS-4.10. The LEON2 and the LEON3 BSP have been used
 to test the Driver Manager, the two Hardware platforms are different when it comes to Plug and Play. The LEON2 
-systems traditionally use a hardcoded register location, whereas in LEON3 systems Plug & Play is used to find 
-all cores such a Memory controller, Timer, IRQ Controller, Ethernet MAC.
+systems traditionally use a hardcoded register location and IRQ assignment, whereas in LEON3 systems 
+Plug & Play is used to find all cores such a Memory controller, Timer, IRQ Controller, Ethernet MAC, etc. The
+Plug & Play information in a LEON3 system contain IRQ number, register base addresses, memory space regions
+etc.
 =  Purpose  =
 
@@ -13 +15 @@
  *  more easily reuse driver code
  *  enable drivers to be written independent of how certain services are implemented.
-  *  interrupt handling (registration, masking, unmasking, acknowledging)
+  *  interrupt handling (registration, masking, unmasking, clearing/acknowledging when level sensitive)
   *  memory space translation
-  *  Bus information (Bus frequency, Snooping/Cache options)
- *  Unify the way drivers are configured. The driver registration string is replaced.
- *  build images suitable for multiple targets with different hardware present.
- *  Hardware topology awareness.
- *  Easier to access which drivers are loaded and what hardware is present.
- *  An initial step towards an architecture independent PCI layer.
+  *  Bus information (Bus/Core frequency, Snooping/Cache options, Location in system)
+ *  Unify the way drivers are configured. The driver registration string is replaced with "driver resources" managed per bus.
+ *  build images suitable for multiple targets with different hardware present, for example different PCI Host controllers.
+ *  Hardware bus topology awareness in software, this makes it easier to access which drivers are loaded and what hardware is present.
 =  Overview  =
 
@@ -30 +30 @@
 addresses, get configuration for a certain hardware device, etc.
 
+A device is situated on a bus, a driver for a bus is called a bus driver. The bus driver tells the driver
+manager about new devices by a procedure called device registration, basically the bus driver fills in a
+data structure describing a specific hardware device and gives it to the manager.
+
 Once a device has been registered by a bus driver the manager tries to find a suitable driver by comparing
-the information about the hardware (provided by the bus driver) to the driver hardware support information
-(provided by the driver). When a driver is found that can handle the device the device and driver are 
-"united". The device is put into a list later processed by the driver manager.
-
-All drivers are registered before starting scanning for devices.
+the information about the hardware (provided by the bus driver from Plug & Play) to the driver hardware 
+support information (provided by the driver). When a driver is found that can handle the hardware device 
+the device and driver are "united". The device is put into a list later processed by the driver manager.
+Each device in the list is picked out and initialized in a two step process (init1 and init2) by calling
+the device driver's functions, init1() and init2(). If the device driver happens to be a bus driver more
+devices will be registered and put at the end of the device list. When there is no more devices left in the
+list the manager will complete.
+
+All device drivers and bus drivers are registered before starting scanning for devices.
 
 The driver manager uses the concepts bus, device, driver and driver resources. A bus represents
-a device with child devices. A device represents a certain hardware. The topmost device is a bus
-device, the root bus device. All devices is connected to the root bus device.
+a device with child devices. A device represents a certain hardware situated on a bus. A driver
+handles one or more devices of the same hardware type. Driver resources are configuration parameters
+targeted one specific driver and one specific device, the resouces consists of an array with name, type 
+and value. The resources are hardcoded at compile time by the user.
+
+The topmost device is a bus device, the root bus device. The driver for that device is called the root
+bus driver. All devices are connected to the root bus device.
+=  Initialization steps  =
+
 
 The root bus device is the first device created during driver manager initialization. It is assigned
 a driver, the root bus driver, manually. The root bus driver must be configured by the user or
-BSP prior to driver manager initialization. The root bus device is initialized and reports the
-available hardware on the bus., once the root bus device is done initializing the driver manager
-proceeds to initialize the found devices in a two step process.
+BSP prior to driver manager initialization.
+
+After all drivers have been registered to the driver manager the device initialization begins. The root 
+bus device is initialized and reports the available hardware devices on the bus. Once the root bus device
+is done initializing, the driver manager proceeds to initialize the found devices in a two step process.
 
 Once a device has been registered by a bus driver the manager tries to find a suitable driver by comparing
@@ -55 +72 @@
 
 All devices are initialized in a two step process, called init1 and init2. Each driver provides two
-function pointers init1 and init2 which are called in order. All device's init1 is first called, when
+function pointers init1 and init2 which are called in order. All devices' init1 is first called, when
 completed the driver manager enter stage two where all devices are called once more, this time
 using the init2 function. The initialization process is separated in two stages to make it possible
-for drivers to rely on other drivers being initialized. The order in which the devices are initialized
-on a bus can not be known (the same order as they are registered in which often depends on how the
-hardware is found in the Plug and Play information).
+for drivers to rely on other drivers services being initialized. The order in which the devices are 
+initialized on a bus can not be known, it is the same order as they were registered in which often 
+depends on how the hardware is found in the Plug and Play information.
 
 All drivers that export some functionality to other drivers, for example memory controller drivers,
@@ -70 +87 @@
 
 Below are some concepts described that are used by the manager.
+=  Driver Interface  =
+
+The driver manager provides the drivers with a common interface independent of bus type. In order for the
+driver manager to implement one interface it relies on the bus driver to make bus dependent operations. The
+bus drivers are the core of the functionality provided by the driver manager.
+
+It may be bus dependent how to find the bus frequency, manage interrupts (level sensitive?), Plug & Play
+information, translate addresses, find a driver for a device etc.
+
+When the driver manager interface is not enough for a bus type, the bus driver may implement functions of
+its own extending the driver manager interface but still taking advantage of the device and driver structure
+introduced by the manager. Perhaps a stupid example but anyway, the example below shows how a PCI bus 
+driver could make PCI Read Configuration Space function simpler by removing the knowledge of Bus,Slot,Func
+in a PCI device driver, instead of calling pci_cfg_read_word directly:
+
+{{{
+int pci_cfg_read_word(int bus, int slot, int func, int offset, unsigned int *buf);
+
+int pcimgr_cfg_read_word(rtems_drvmgr_dev_info *dev, int offset, unsigned int *buf)
+{
+        /* Get Bus specific information previously prepared by bus driver */
+        struct pci_dev_info *pciinfo = (struct pci_dev_info *)dev->businfo;
+
+        return pci_cfg_read_word(pciinfo->bus, pciinfo->slot, pciinfo->func, offset, buf);
+}
+}}}
 =  Bus Driver  =
 
 The Driver Manager needs to be informed about hardware devices present in the system, that is what the bus
 driver does. The bus driver reads a hardcoded set up or Plug & Play information and registers devices to
-the manager. The bus driver must provide some services in order for the drivers to the found devices to 
-function properly. The services are implemented by functions that are pointed to by a operations struct 
-making it possible for the manager driver interface to call the functions.
-
-The bus driver must be able to unite a hardware device with a driver.
+the manager. The bus driver must also provide function pointers to a couple of services in order for 
+the driver manager to present a common driver interface to device drivers.
+
+One of the most important services is the ability to unite a hardware device with a driver, for this to happen
+a driver must provide information about what hardware is supported so that the bus driver can compare that with
+the found hardware.
 
 {{{
@@ -93 +137 @@
         struct rtems_drvmgr_mmap_entry  *mmaps;         /*!< Memory Map Translation, array of address spaces */
 };
-
 }}}
 =  Root bus driver  =
@@ -114 +157 @@
 #define DEV_STATE_UNITED        0x00000100      /* Device United with Device Driver */
 #define DEV_STATE_DELETED       0x00000200      /* Device has been deleted (unregistered) */
-
+#define DEV_STATE_IGNORED       0x00000400      /* Device was ignored according to user's request, the device
+                                                 * was never reported to it's driver (as expected).
+                                                 */
 /*! Device information */
 struct rtems_drvmgr_dev_info {
@@ -130 +175 @@
         void                            *businfo;       /*!< Host bus specific information */
         struct rtems_drvmgr_bus_info    *bus;           /*!< Pointer to bus, set only if this is a bus */
-};
+        int                             error;          /*!< Error state returned by driver */
+};
+}}}
+==  Device State  ==
+
+During the initialization of a device the initialization level (init1 or init2)
+changes, to hold that information a device state integer is maintained for each
+device. The device state is updated when,
+ ¤ Assigning a driver
+ ¤ Init1 is completed or erroneous
+ ¤ Init2 is completed or erroneous
+ ¤ A device is deleted by the user
+ ¤ A device must be ignored by user's request
+
+The manager uses the state internaly but it may also be of improtance to the
+user when a device failed.
+==  Device Error Handling  ==
+
+During the initialization steps of a device the driver manager may not complete
+the initialization because of,
+ ¤ No suitable driver found
+ ¤ Device was requested to be ignored by user
+ ¤ Driver failed to take device into init1 or init2 dur to an error
+
+The driver manager removes the device from further initialization and instead of
+freeing the device structure the device is put into a separate list (the inactive
+list) and the device state is updated accordingly. When initialization fails the
+error field in the device structure is updated indicating the error more in
+detail.
+
+A user may process all failed devices on the inactive list to determined if the
+error was acceptable or not.
+==  Unregistering a device  ==
+
+A device may also be unregistered due to device error (parity error on the PCI
+bus for example) or hardware was removed (hot-plug system). A device is 
+unregistered by calling rtems_drvmgr_dev_unregister. The device is moved from
+the current list to the inactive list (delete=0) or it may be removed completely 
+to avoid memory exhaustion (delete=1).
+
+{{{
+/*! Remove a device, and all its children devices if device is a bus device. The device 
+ *  driver will be requested to remove the device and once gone from bus, device and 
+ *  driver list the device is put into a inactive list for debugging (this is optional
+ *  by using delete argument).
+ *
+ * Removing the Root Bus Device is not supported.
+ *
+ * \param delete If non-zero the device will be deallocated, and not put into the 
+ *               inacitve list.
+ */
+extern int rtems_drvmgr_dev_unregister(struct rtems_drvmgr_dev_info *dev, int delete);
 }}}
 =  Driver   =
@@ -140 +236 @@
 
 Drivers are called twice to initialize a hardware device. Init1 and Init2 stages. A delete function may
-optionally be defined if the driver can handle removing of hardware.
+optionally be defined if the driver can handle removing of hardware Also a info function to request information
+such as device name, type of driver, statistcs, current register content etc for interactive information
+printing from for example the shell.
 
 {{{
@@ -148 +246 @@
         rtems_status_code       (*init2)(struct rtems_drvmgr_dev_info *);       /*! Function doing Init Stage 2 of a hardware device */
         rtems_status_code       (*delete)(struct rtems_drvmgr_dev_info *);      /*! Function called when device instance is to be removed */
+        rtems_status_code       (*info)(struct rtems_drvmgr_dev_info *, int, int);/*! Function called to request information about a device or driver */
 };
 
@@ -162 +261 @@
 };
 }}}
+==  Driver ID  ==
+
+All drivers must have a unique ID, it may be created by using one of the
+supported devices's ID. Often a device have a VENDOR:DEVICE unique for the bus.
+A driver ID is created by combining a bus dependent ID and the bus type number:
+
+From drvmgr.h:
+{{{
+/*** Bus indentification ***/
+#define DRVMGR_BUS_TYPE_NONE 0          /* Not a valid bus */
+#define DRVMGR_BUS_TYPE_ROOT 1          /* Hard coded bus */
+#define DRVMGR_BUS_TYPE_PCI 2           /* PCI bus */
+#define DRVMGR_BUS_TYPE_AMBAPP 3        /* AMBA Plug & Play bus */
+
+/* 64-bit identification integer definition
+ *  ¤ Bus ID 8-bit [7..0]
+ *  ¤ Reserved 8-bit field [63..56]
+ *  ¤ Device ID specific for bus type 48-bit [55..8]  (Different buses have different unique identifications for hardware/driver.)
+ *
+ * ID Rules
+ *  ¤ A root bus driver must always have device ID set to 0. There can only by one root bus driver
+ *    for a certain bus type.
+ *  ¤ A Driver ID must identify a unique hardware core
+ *
+ */
+
+/* Bus ID Mask */
+#define DRIVER_ID_BUS_MASK 0x00000000000000FFULL
+
+/* Reserved Mask for future use */
+#define DRIVER_ID_RSV_MASK 0xFF00000000000000ULL
+
+/* Reserved Mask for future use */
+#define DRIVER_ID_DEV_MASK 0x00FFFFFFFFFFFF00ULL
+
+/* Set Bus ID Mask. */
+#define DRIVER_ID(busid, devid) \
+        (unsigned long long)( (((unsigned long long)(devid) << 8) & DRIVER_ID_DEV_MASK) | ((unsigned long long)(busid) & DRIVER_ID_BUS_MASK) )
+}}}
+
+From AMBA Plug & Play bus, ambapp_bus.h:
+{{{
+/* GRLIB AMBA Plug&Play Driver ID generation */
+#define DRIVER_AMBAPP_ID(vendor, device) \
+        DRIVER_ID(DRVMGR_BUS_TYPE_AMBAPP, ((((vendor) & 0xff) << 16) | ((device) & 0xfff)))
+
+/*** Gaisler Hardware Device Driver IDs ***/
+#define DRIVER_AMBAPP_GAISLER_GRETH_ID          DRIVER_AMBAPP_ID(VENDOR_GAISLER, GAISLER_ETHMAC)
+}}}
+
+In the above example VENDOR_GAISLER and GAISLER_GRETH are IDs assigned by the
+GRLIB Bus domain.
+
+The Driver ID and a device unit number is used when assigning driver resources
+to a certain device. The driver resources must be in a format the driver can
+understand, which the driver ID makes sure of, and the driver must not use
+another device's resources, which the unit/core number makes sure of.
 =  Driver Resource  =
 
 A driver resource is a resource used by a driver for a certain device instance. The resource may
-be integer with value 65 called "numberTxDescriptors". The resources are grouped together in
-arrays targeting one device instance. The resources are assigned to one or multiple buses making
-it possible for the drivers to find whose hardware device is situated on that very bus. Multiple
-hardware devices of the same type is separated by their bus unit number which is always the
-same between starts. The number usually comes from the order the device if found in the plug
-and play information. Below is a typical bus resource definition grlib_drv_resource configuring
-two GRSPW device drivers. The second GRSPW core will have double the amount of descriptors
-than the first.
+be an integer with value 65 called "numberTxDescriptors". The resources are grouped together in
+arrays targeting one device instance. The resources are assigned to a bus making it possible for
+a driver to find the resources for a specific device, since the Driver Manager knows which bus
+the device is situated on. A resource is identified by BUS, DRIVER ID and Unit number. Multiple
+hardware devices of the same type are separated by their bus unit number which is always the
+same between runs. The number usually comes from the order the device if found in the plug
+and play information.
+
+Below is a typical bus resource definition in a LEON3-GRLIB  system. The grlib_drv_resource 
+configures two GRSPW device drivers. The second GRSPW core will have double the amount of 
+descriptors than the first.
 
 {{{
@@ -178 +337 @@
  * 
  * Overview of structures:
- *  All bus resources entries (bus_res_node) are linked together for a bus (bus_info->reslist).
- *  One bus resource entry has a pointer to an array of driver resources (drv_res). One driver 
- *  resouces is made out of an array of keys (drv_res_key). All keys belongs to the same driver 
- *  and harwdare device. Each key has a Name, Type ID and Data interpreted differently 
- *  depending on the Type ID (union key_value).
+ *  All bus resources entries (_bus_res) are linked together per bus (bus_info->reslist).
+ *  One bus resource entry has a pointer to an array of driver resources (_drv_res). One driver 
+ *  resouces is made out of an array of keys (rtems_drvmgr_key). All keys belongs to the 
+ *  same driver and harwdare device. Each key has a Name, Type ID and Data interpreted 
+ *  differently depending on the Type ID (union rtems_drvmgr_key_value).
  *
  */
@@ -198 +357 @@
 /*! Union of different values */
 union rtems_drvmgr_key_value {
-        unsigned int            i;              /*!< Key data type UNIGNED INTEGER */
+        unsigned int            i;              /*!< Key data type UNSIGNED INTEGER */
         char                    *str;           /*!< Key data type STRING */
         void                    *ptr;           /*!< Key data type ADDRESS/POINTER */
@@ -215 +374 @@
 struct rtems_drvmgr_drv_res {
         unsigned long long      drv_id;         /*!< Identifies the driver this resource is aiming */
-        int                     minor_bus;      /*!< Indentifies a specfic device */
+        int                     minor_bus;      /*!< Indentifies a specific device */
         struct rtems_drvmgr_key *keys;          /*!< First key in key array, ended with KEY_EMPTY */
 };
@@ -277 +436 @@
 The root bus device driver is registered by calling root_drv_register, the root bus driver may
 provide a function doing this, in that case one must call that function instead. For example
-drv_mgr_grlib_init register the GRLIB-AMBA Plug & Play Bus as the root bus driver and also
+drv_mgr_grlib_init register the LEON3 GRLIB-AMBA Plug & Play Bus as the root bus driver and also
 assigns the driver resources for the root bus.
 
@@ -283 +442 @@
 per driver that is responsible to register one or more drivers. The drv_mgr_drivers can be set up
 by defining CONFIGURE_INIT, selecting the appropriate drivers and including
-drv_mgr/drvmgr_confdefs.h. This approach is similar to configuring a standard RTEMS project
+drvmgr/drvmgr_confdefs.h. This approach is similar to configuring a standard RTEMS project
 using rtems/confdefs.h. Below is an example how to select drivers.
 
@@ -312 +471 @@
 #define CONFIGURE_DRIVER_PCI_GR_RASTA_TMTC
 #define CONFIGURE_DRIVER_PCI_GR_701
-#include <drv_mgr/drv_manager_confdefs.h>
+#include <drvmgr/drvmgr_confdefs.h>
 }}}
 =  Initialization  =
@@ -442 +601 @@
 /*  General part of a AMBA Plug & Play bus driver. */
 
-#define DRIVER_AMBA_ID(vendor, device, subid) \
-        DRIVER_SUBID_ADD((((unsigned long long)(vendor)<<32) | ((unsigned long long)(device))), subid)
-
-/*** Gaisler Hardware Device Driver definitions ***/
-#define DRIVER_AMBAPP_GAISLER_GRETH_ID          DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_ETHMAC, 0)
-#define DRIVER_AMBAPP_GAISLER_GRETH_ID_ALL      DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_ETHMAC, -1)
-
-#define DRIVER_AMBAPP_GAISLER_GRSPW_ID          DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_SPW, 0)
-#define DRIVER_AMBAPP_GAISLER_GRSPW_ID_ALL      DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_SPW, -1)
-
-#define DRIVER_AMBAPP_GAISLER_GRCAN_ID          DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_GRCAN, 0)
-#define DRIVER_AMBAPP_GAISLER_GRCAN_ID_ALL      DRIVER_AMBA_ID(VENDOR_GAISLER, GAISLER_GRCAN, -1)
-
-/*** ESA Hardware Device Driver definitions ***/
-#define DRIVER_AMBAPP_ESA_MCTRL_ID              DRIVER_AMBA_ID(VENDOR_ESA, ESA_MCTRL, 0)
-#define DRIVER_AMBAPP_ESA_MCTRL_ID_ALL          DRIVER_AMBA_ID(VENDOR_ESA, ESA_MCTRL, -1)
-#define DRIVER_AMBAPP_MCTRL_ID                  DRIVER_AMBAPP_ESA_MCTRL_ID
-#define DRIVER_AMBAPP_MCTRL_ID_ALL              DRIVER_AMBAPP_ESA_MCTRL_ID_ALL
+/* GRLIB AMBA Plug&Play Driver ID generation */
+#define DRIVER_AMBAPP_ID(vendor, device) \
+        DRIVER_ID(DRVMGR_BUS_TYPE_AMBAPP, ((((vendor) & 0xff) << 16) | ((device) & 0xfff)))
+
+/*** Gaisler Hardware Device Driver IDs ***/
+#define DRIVER_AMBAPP_GAISLER_GRETH_ID          DRIVER_AMBAPP_ID(VENDOR_GAISLER, GAISLER_ETHMAC)
+#define DRIVER_AMBAPP_GAISLER_GRSPW_ID          DRIVER_AMBAPP_ID(VENDOR_GAISLER, GAISLER_SPW)
+
+/*** ESA Hardware Device Driver IDs ***/
+#define DRIVER_AMBAPP_ESA_MCTRL_ID              DRIVER_AMBAPP_ID(VENDOR_ESA, ESA_MCTRL)
 
 struct amba_dev_id {
@@ -506 +657 @@
         grspw_init1,
         grspw_init2,
-        NULL            /* Driver does not support device unregister (delete) */
+        NULL,           /* Driver does not support device unregister (delete) */
+        NULL            /* No info function */
 };
 
@@ -676 +828 @@
         }
 
-        /* initialize the code with some resonable values,
+        /* initialize the code with some reasonable values,
          * actual initialization is done later using ioctl(fd)
          * on the opened device */
@@ -830 +982 @@
 }
 }}}
+=  Multi-processor system notes  =
+
+Ina Multi-processor system sharing the RTEMS instances often share the resources, in a non
+Plug & Play system it might be easy to avoid conflicts however in a Plug & Play system it
+may be tougher. Say that four UARTs are found, usually the first one is used a console and the
+other ones are initialized by the driver into some kind of reset state and registered in the
+file system to /dev/ttyS[1..3]. This may not work in a MP system, to solve this problem the
+driver manager provides means to stop a device from being given to the it's device driver.
+This, of course, must be configured by the user. A device may be ignored by the driver manager
+by defining an entry in the driver resource array targeting a specific device, but setting the
+pointer to resource array to NULL.
+
+After a new device has been registered by the bus driver the device is united with a driver, then
+a NULL pointer resource array is searched for, if found the device is put on an inactive list rather
+than entering stage1 later on. The device's state mask is set to DEV_STATE_IGNORED.
 =  TODO  =
 
-=  BSP  =
+=  == BSP ===
 
 Changes to BSPs:
  *  Add support to BSP to register root bus
  *  Make driver resources on the root bus weak so that the user can override them.
-=  == RTEMS ===
+
+=== RTEMS ===
 
  *  Make I/O Manager handle registering I/O Drivers before I/O Manager is initialized.