1 | @c |
---|
2 | @c COPYRIGHT (c) 2011 |
---|
3 | @c Aeroflex Gaisler AB |
---|
4 | @c All rights reserved. |
---|
5 | @c |
---|
6 | |
---|
7 | @chapter PCI Library |
---|
8 | |
---|
9 | @cindex libpci |
---|
10 | |
---|
11 | @section Introduction |
---|
12 | |
---|
13 | The Peripheral Component Interconnect (PCI) bus is a very common computer |
---|
14 | bus architecture that is found in almost every PC today. The PCI bus is |
---|
15 | normally located at the motherboard where some PCI devices are soldered |
---|
16 | directly onto the PCB and expansion slots allows the user to add custom |
---|
17 | devices easily. There is a wide range of PCI hardware available implementing |
---|
18 | all sorts of interfaces and functions. |
---|
19 | |
---|
20 | This section describes the PCI Library available in RTEMS used to access the |
---|
21 | PCI bus in a portable way across computer architectures supported by RTEMS. |
---|
22 | |
---|
23 | The PCI Library aims to be compatible with PCI 2.3 with a couple of |
---|
24 | limitations, for example there is no support for hot-plugging, 64-bit |
---|
25 | memory space and cardbus bridges. |
---|
26 | |
---|
27 | In order to support different architectures and with small foot-print embedded |
---|
28 | systems in mind the PCI Library offers four different configuration options |
---|
29 | listed below. It is selected during compile time by defining the appropriate |
---|
30 | macros in confdefs.h. It is also possible to enable PCI_LIB_NONE (No |
---|
31 | Configuration) which can be used for debuging PCI access functions. |
---|
32 | @itemize @bullet |
---|
33 | @item Auto Configuration (do Plug & Play) |
---|
34 | @item Read Configuration (read BIOS or boot loader configuration) |
---|
35 | @item Static Configuration (write user defined configuration) |
---|
36 | @item Peripheral Configuration (no access to cfg-space) |
---|
37 | @end itemize |
---|
38 | |
---|
39 | @section Background |
---|
40 | |
---|
41 | The PCI bus is constructed in a way where on-board devices and devices |
---|
42 | in expansion slots can be automatically found (probed) and configured |
---|
43 | using Plug & Play completely implemented in software. The bus is set up once |
---|
44 | during boot up. The Plug & Play information can be read and written from |
---|
45 | PCI configuration space. A PCI device is identified in configuration space by |
---|
46 | a unique bus, slot and function number. Each PCI slot can have up to 8 |
---|
47 | functions and interface to another PCI sub-bus by implementing a PCI-to-PCI |
---|
48 | bridge according to the PCI Bridge Architecture specification. |
---|
49 | |
---|
50 | Using the unique [bus:slot:func] any device can be configured regardless of how |
---|
51 | PCI is currently set up as long as all PCI buses are enumerated correctly. The |
---|
52 | enumeration is done during probing, all bridges are given a bus number in |
---|
53 | order for the bridges to respond to accesses from both directions. The PCI |
---|
54 | library can assign address ranges to which a PCI device should respond using |
---|
55 | Plug & Play technique or a static user defined configuration. After the |
---|
56 | configuration has been performed the PCI device drivers can find devices by |
---|
57 | the read-only PCI Class type, Vendor ID and Device ID information found in |
---|
58 | configuration space for each device. |
---|
59 | |
---|
60 | In some systems there is a boot loader or BIOS which have already configured |
---|
61 | all PCI devices, but on embedded targets it is quite common that there is no |
---|
62 | BIOS or boot loader, thus RTEMS must configure the PCI bus. Only the PCI host |
---|
63 | may do configuration space access, the host driver or BSP is responsible to |
---|
64 | translate the [bus:slot:func] into a valid PCI configuration space access. |
---|
65 | |
---|
66 | If the target is not a host, but a peripheral, configuration space can not be |
---|
67 | accessed, the peripheral is set up by the host during start up. In complex |
---|
68 | embedded PCI systems the peripheral may need to access other PCI boards than |
---|
69 | the host. In such systems a custom (static) configuration of both the host |
---|
70 | and peripheral may be a convenient solution. |
---|
71 | |
---|
72 | The PCI bus defines four interrupt signals INTA#..INTD#. The interrupt signals |
---|
73 | must be mapped into a system interrupt/vector, it is up to the BSP or host |
---|
74 | driver to know the mapping, however the BIOS or boot loader may use the |
---|
75 | 8-bit read/write "Interrupt Line" register to pass the knowledge along to the |
---|
76 | OS. |
---|
77 | |
---|
78 | The PCI standard defines and recommends that the backplane route the interupt |
---|
79 | lines in a systematic way, however in standard there is no such requirement. |
---|
80 | The PCI Auto Configuration Library implements the recommended way of routing |
---|
81 | which is very common but it is also supported to some extent to override the |
---|
82 | interrupt routing from the BSP or Host Bridge driver using the configuration |
---|
83 | structure. |
---|
84 | |
---|
85 | @subsection Software Components |
---|
86 | |
---|
87 | The PCI library is located in cpukit/libpci, it consists of different parts: |
---|
88 | @itemize @bullet |
---|
89 | @item PCI Host bridge driver interface |
---|
90 | @item Configuration routines |
---|
91 | @item Access (Configuration, I/O and Memory space) routines |
---|
92 | @item Interrupt routines (implemented by BSP) |
---|
93 | @item Print routines |
---|
94 | @item Static/peripheral configuration creation |
---|
95 | @item PCI shell command |
---|
96 | @end itemize |
---|
97 | |
---|
98 | @subsection PCI Configuration |
---|
99 | |
---|
100 | During start up the PCI bus must be configured in order for host and peripherals |
---|
101 | to access one another using Memory or I/O accesses and that interrupts are |
---|
102 | properly handled. Three different spaces are defined and mapped separately: |
---|
103 | @enumerate |
---|
104 | @item I/O space (IO) |
---|
105 | @item non-prefetchable Memory space (MEMIO) |
---|
106 | @item prefetchable Memory space (MEM) |
---|
107 | @end enumerate |
---|
108 | |
---|
109 | Regions of the same type (I/O or Memory) may not overlap which is guaranteed |
---|
110 | by the software. MEM regions may be mapped into MEMIO regions, but MEMIO |
---|
111 | regions can not be mapped into MEM, for that could lead to prefetching of |
---|
112 | registers. The interrupt pin which a board is driving can be read out from |
---|
113 | PCI configuration space, however it is up to software to know how interrupt |
---|
114 | signals are routed between PCI-to-PCI bridges and how PCI INT[A..D]# pins are |
---|
115 | mapped to system IRQ. In systems where previous software (boot loader or BIOS) |
---|
116 | has already set up this the configuration is overwritten or simply read out. |
---|
117 | |
---|
118 | In order to support different configuration methods the following configuration |
---|
119 | libraries are selectable by the user: |
---|
120 | @itemize @bullet |
---|
121 | @item Auto Configuration (run Plug & Play software) |
---|
122 | @item Read Configuration (relies on a boot loader or BIOS) |
---|
123 | @item Static Configuration (write user defined setup, no Plug & Play) |
---|
124 | @item Peripheral Configuration (user defined setup, no access to configuration space) |
---|
125 | @end itemize |
---|
126 | |
---|
127 | A host driver can be made to support all three configuration methods, or any |
---|
128 | combination. It may be defined by the BSP which approach is used. |
---|
129 | |
---|
130 | The configuration software is called from the PCI driver (pci_config_init()). |
---|
131 | |
---|
132 | Regardless of configuration method a PCI device tree is created in RAM during |
---|
133 | initialization, the tree can be accessed to find devices and resources without |
---|
134 | accessing configuration space later on. The user is responsible to create the |
---|
135 | device tree at compile time when using the static/peripheral method. |
---|
136 | |
---|
137 | |
---|
138 | @subsubsection RTEMS Configuration selection |
---|
139 | |
---|
140 | The active configuration method can be selected at compile time in the same |
---|
141 | way as other project parameters by including rtems/confdefs.h and setting |
---|
142 | @itemize @bullet |
---|
143 | @item CONFIGURE_INIT |
---|
144 | @item RTEMS_PCI_CONFIG_LIB |
---|
145 | @item CONFIGURE_PCI_LIB = PCI_LIB_(AUTO,STATIC,READ,PERIPHERAL) |
---|
146 | @end itemize |
---|
147 | |
---|
148 | See the RTEMS configuration section how to setup the PCI library. |
---|
149 | |
---|
150 | |
---|
151 | @subsubsection Auto Configuration |
---|
152 | |
---|
153 | The auto configuration software enumerates PCI buses and initializes all PCI |
---|
154 | devices found using Plug & Play. The auto configuration software requires |
---|
155 | that a configuration setup has been registered by the driver or BSP in order |
---|
156 | to setup the I/O and Memory regions at the correct address ranges. PCI |
---|
157 | interrupt pins can optionally be routed over PCI-to-PCI bridges and mapped |
---|
158 | to a system interrupt number. BAR resources are sorted by size and required |
---|
159 | alignment, unused "dead" space may be created when PCI bridges are present |
---|
160 | due to the PCI bridge window size does not equal the alignment. To cope with |
---|
161 | that resources are reordered to fit smaller BARs into the dead space to minimize |
---|
162 | the PCI space required. If a BAR or ROM register can not be allocated a PCI |
---|
163 | address region (due to too few resources available) the register will be given |
---|
164 | the value of pci_invalid_address which defaults to 0. |
---|
165 | |
---|
166 | The auto configuration routines support: |
---|
167 | @itemize @bullet |
---|
168 | @item PCI 2.3 |
---|
169 | @item Little and big endian PCI bus |
---|
170 | @item one I/O 16 or 32-bit range (IO) |
---|
171 | @item memory space (MEMIO) |
---|
172 | @item prefetchable memory space (MEM), if not present MEM will be mapped into |
---|
173 | MEMIO |
---|
174 | @item multiple PCI buses - PCI-to-PCI bridges |
---|
175 | @item standard BARs, PCI-to-PCI bridge BARs, ROM BARs |
---|
176 | @item Interrupt routing over bridges |
---|
177 | @item Interrupt pin to system interrupt mapping |
---|
178 | @end itemize |
---|
179 | |
---|
180 | Not supported: |
---|
181 | @itemize @bullet |
---|
182 | @item hot-pluggable devices |
---|
183 | @item Cardbus bridges |
---|
184 | @item 64-bit memory space |
---|
185 | @item 16-bit and 32-bit I/O address ranges at the same time |
---|
186 | @end itemize |
---|
187 | |
---|
188 | In PCI 2.3 there may exist I/O BARs that must be located at the low 64kBytes |
---|
189 | address range, in order to support this the host driver or BSP must make sure |
---|
190 | that I/O addresses region is within this region. |
---|
191 | |
---|
192 | |
---|
193 | @subsubsection Read Configuration |
---|
194 | |
---|
195 | When a BIOS or boot loader already has setup the PCI bus the configuration can |
---|
196 | be read directly from the PCI resource registers and buses are already |
---|
197 | enumerated, this is a much simpler approach than configuring PCI ourselves. The |
---|
198 | PCI device tree is automatically created based on the current configuration and |
---|
199 | devices present. After initialization is done there is no difference between |
---|
200 | the auto or read configuration approaches. |
---|
201 | |
---|
202 | |
---|
203 | @subsubsection Static Configuration |
---|
204 | |
---|
205 | To support custom configurations and small-footprint PCI systems, the user may |
---|
206 | provide the PCI device tree which contains the current configuration. The |
---|
207 | PCI buses are enumerated and all resources are written to PCI devices during |
---|
208 | initialization. When this approach is selected PCI boards must be located at |
---|
209 | the same slots every time and devices can not be removed or added, Plug & Play |
---|
210 | is not performed. Boards of the same type may of course be exchanged. |
---|
211 | |
---|
212 | The user can create a configuration by calling pci_cfg_print() on a running |
---|
213 | system that has had PCI setup by the auto or read configuration routines, it |
---|
214 | can be called from the PCI shell command. The user must provide the PCI device |
---|
215 | tree named pci_hb. |
---|
216 | |
---|
217 | |
---|
218 | @subsubsection Peripheral Configuration |
---|
219 | |
---|
220 | On systems where a peripheral PCI device needs to access other PCI devices than |
---|
221 | the host the peripheral configuration approach may be handy. Most PCI devices |
---|
222 | answers on the PCI host's requests and start DMA accesses into the Hosts memory, |
---|
223 | however in some complex systems PCI devices may want to access other devices |
---|
224 | on the same bus or at another PCI bus. |
---|
225 | |
---|
226 | A PCI peripheral is not allowed to do PCI configuration cycles, which means that |
---|
227 | it must either rely on the host to give it the addresses it needs, or that the |
---|
228 | addresses are predefined. |
---|
229 | |
---|
230 | This configuration approach is very similar to the static option, however the |
---|
231 | configuration is never written to PCI bus, instead it is only used for drivers |
---|
232 | to find PCI devices and resources using the same PCI API as for the host |
---|
233 | |
---|
234 | |
---|
235 | @subsection PCI Access |
---|
236 | |
---|
237 | The PCI access routines are low-level routines provided for drivers, |
---|
238 | configuration software, etc. in order to access different regions in a way |
---|
239 | not dependent upon the host driver, BSP or platform. |
---|
240 | @itemize @bullet |
---|
241 | @item PCI configuration space |
---|
242 | @item PCI I/O space |
---|
243 | @item Registers over PCI memory space |
---|
244 | @item Translate PCI address into CPU accessible address and vice versa |
---|
245 | @end itemize |
---|
246 | |
---|
247 | By using the access routines drivers can be made portable over different |
---|
248 | architectures. The access routines take the architecture endianness into |
---|
249 | consideration and let the host driver or BSP implement I/O space and |
---|
250 | configuration space access. |
---|
251 | |
---|
252 | Some non-standard hardware may also define the PCI bus big-endian, for example |
---|
253 | the LEON2 AT697 PCI host bridge and some LEON3 systems may be configured that |
---|
254 | way. It is up to the BSP to set the appropriate PCI endianness on compile time |
---|
255 | (BSP_PCI_BIG_ENDIAN) in order for inline macros to be correctly defined. |
---|
256 | Another possibility is to use the function pointers defined by the access |
---|
257 | layer to implement drivers that support "run-time endianness detection". |
---|
258 | |
---|
259 | |
---|
260 | @subsubsection Configuration space |
---|
261 | |
---|
262 | Configuration space is accessed using the routines listed below. The |
---|
263 | pci_dev_t type is used to specify a specific PCI bus, device and function. It |
---|
264 | is up to the host driver or BSP to create a valid access to the requested |
---|
265 | PCI slot. Requests made to slots that are not supported by hardware should |
---|
266 | result in PCISTS_MSTABRT and/or data must be ignored (writes) or 0xffffffff |
---|
267 | is always returned (reads). |
---|
268 | |
---|
269 | @example |
---|
270 | /* Configuration Space Access Read Routines */ |
---|
271 | extern int pci_cfg_r8(pci_dev_t dev, int ofs, uint8_t *data); |
---|
272 | extern int pci_cfg_r16(pci_dev_t dev, int ofs, uint16_t *data); |
---|
273 | extern int pci_cfg_r32(pci_dev_t dev, int ofs, uint32_t *data); |
---|
274 | |
---|
275 | /* Configuration Space Access Write Routines */ |
---|
276 | extern int pci_cfg_w8(pci_dev_t dev, int ofs, uint8_t data); |
---|
277 | extern int pci_cfg_w16(pci_dev_t dev, int ofs, uint16_t data); |
---|
278 | extern int pci_cfg_w32(pci_dev_t dev, int ofs, uint32_t data); |
---|
279 | @end example |
---|
280 | |
---|
281 | |
---|
282 | @subsubsection I/O space |
---|
283 | |
---|
284 | The BSP or driver provide special routines in order to access I/O space. Some |
---|
285 | architectures have a special instruction accessing I/O space, others have it |
---|
286 | mapped into a "PCI I/O window" in the standard address space accessed by the |
---|
287 | CPU. The window size may vary and must be taken into consideration by the |
---|
288 | host driver. The below routines must be used to access I/O space. The address |
---|
289 | given to the functions is not the PCI I/O addresses, the caller must have |
---|
290 | translated PCI I/O addresses (available in the PCI BARs) into a BSP or host |
---|
291 | driver custom address, see @ref{Access functions} for how addresses are |
---|
292 | translated. |
---|
293 | |
---|
294 | @example |
---|
295 | /* Read a register over PCI I/O Space */ |
---|
296 | extern uint8_t pci_io_r8(uint32_t adr); |
---|
297 | extern uint16_t pci_io_r16(uint32_t adr); |
---|
298 | extern uint32_t pci_io_r32(uint32_t adr); |
---|
299 | |
---|
300 | /* Write a register over PCI I/O Space */ |
---|
301 | extern void pci_io_w8(uint32_t adr, uint8_t data); |
---|
302 | extern void pci_io_w16(uint32_t adr, uint16_t data); |
---|
303 | extern void pci_io_w32(uint32_t adr, uint32_t data); |
---|
304 | @end example |
---|
305 | |
---|
306 | |
---|
307 | @subsubsection Registers over Memory space |
---|
308 | |
---|
309 | PCI host bridge hardware normally swap data accesses into the endianness of the |
---|
310 | host architecture in order to lower the load of the CPU, peripherals can do DMA |
---|
311 | without swapping. However, the host controller can not separate a standard |
---|
312 | memory access from a memory access to a register, registers may be mapped into |
---|
313 | memory space. This leads to register content being swapped, which must be |
---|
314 | swapped back. The below routines makes it possible to access registers over PCI |
---|
315 | memory space in a portable way on different architectures, the BSP or |
---|
316 | architecture must provide necessary functions in order to implement this. |
---|
317 | |
---|
318 | @example |
---|
319 | static inline uint16_t pci_ld_le16(volatile uint16_t *addr); |
---|
320 | static inline void pci_st_le16(volatile uint16_t *addr, uint16_t val); |
---|
321 | static inline uint32_t pci_ld_le32(volatile uint32_t *addr); |
---|
322 | static inline void pci_st_le32(volatile uint32_t *addr, uint32_t val); |
---|
323 | static inline uint16_t pci_ld_be16(volatile uint16_t *addr); |
---|
324 | static inline void pci_st_be16(volatile uint16_t *addr, uint16_t val); |
---|
325 | static inline uint32_t pci_ld_be32(volatile uint32_t *addr); |
---|
326 | static inline void pci_st_be32(volatile uint32_t *addr, uint32_t val); |
---|
327 | @end example |
---|
328 | |
---|
329 | In order to support non-standard big-endian PCI bus the above pci_* functions |
---|
330 | is required, pci_ld_le16 != ld_le16 on big endian PCI buses. |
---|
331 | |
---|
332 | |
---|
333 | @subsubsection Access functions |
---|
334 | |
---|
335 | The PCI Access Library can provide device drivers with function pointers |
---|
336 | executing the above Configuration, I/O and Memory space accesses. The |
---|
337 | functions have the same arguments and return values as the above |
---|
338 | functions. |
---|
339 | |
---|
340 | The pci_access_func() function defined below can be used to get a function |
---|
341 | pointer of a specific access type. |
---|
342 | |
---|
343 | @example |
---|
344 | /* Get Read/Write function for accessing a register over PCI Memory Space |
---|
345 | * (non-inline functions). |
---|
346 | * |
---|
347 | * Arguments |
---|
348 | * wr 0(Read), 1(Write) |
---|
349 | * size 1(Byte), 2(Word), 4(Double Word) |
---|
350 | * func Where function pointer will be stored |
---|
351 | * endian PCI_LITTLE_ENDIAN or PCI_BIG_ENDIAN |
---|
352 | * type 1(I/O), 3(REG over MEM), 4(CFG) |
---|
353 | * |
---|
354 | * Return |
---|
355 | * 0 Found function |
---|
356 | * others No such function defined by host driver or BSP |
---|
357 | */ |
---|
358 | int pci_access_func(int wr, int size, void **func, int endian, int type); |
---|
359 | @end example |
---|
360 | |
---|
361 | PCI device drivers may be written to support run-time detection of endianess, |
---|
362 | this is mosly for debugging or for development systems. When the product is |
---|
363 | finally deployed macros switch to using the inline functions instead which |
---|
364 | have been configured for the correct endianness. |
---|
365 | |
---|
366 | |
---|
367 | @subsubsection PCI address translation |
---|
368 | |
---|
369 | When PCI addresses, both I/O and memory space, is not mapped 1:1 address |
---|
370 | translation before access is needed. If drivers read the PCI resources directly |
---|
371 | using configuration space routines or in the device tree, the addresses given |
---|
372 | are PCI addresses. The below functions can be used to translate PCI addresses |
---|
373 | into CPU accessible addresses or vice versa, translation may be different for |
---|
374 | different PCI spaces/regions. |
---|
375 | |
---|
376 | @example |
---|
377 | /* Translate PCI address into CPU accessible address */ |
---|
378 | static inline int pci_pci2cpu(uint32_t *address, int type); |
---|
379 | |
---|
380 | /* Translate CPU accessible address into PCI address (for DMA) */ |
---|
381 | static inline int pci_cpu2pci(uint32_t *address, int type); |
---|
382 | @end example |
---|
383 | |
---|
384 | |
---|
385 | @subsection PCI Interrupt |
---|
386 | |
---|
387 | The PCI specification defines four different interrupt lines INTA#..INTD#, |
---|
388 | the interrupts are low level sensitive which make it possible to support |
---|
389 | multiple interrupt sources on the same interrupt line. Since the lines are |
---|
390 | level sensitive the interrupt sources must be acknowledged before clearing the |
---|
391 | interrupt contoller, or the interrupt controller must be masked. The BSP must |
---|
392 | provide a routine for clearing/acknowledging the interrupt controller, it is |
---|
393 | up to the interrupt service routine to acknowledge the interrupt source. |
---|
394 | |
---|
395 | The PCI Library relies on the BSP for implementing shared interrupt handling |
---|
396 | through the BSP_PCI_shared_interrupt_* functions/macros, they must be defined |
---|
397 | when including bsp.h. |
---|
398 | |
---|
399 | PCI device drivers may use the pci_interrupt_ routines in order to call the |
---|
400 | BSP specific functions in a platform independent way. The PCI interrupt |
---|
401 | interface has been made similar to the RTEMS IRQ extension so that a BSP can |
---|
402 | use the standard RTEMS interrupt functions directly. |
---|
403 | |
---|
404 | |
---|
405 | @subsection PCI Shell command |
---|
406 | |
---|
407 | The RTEMS shell has a PCI command 'pci' which makes it possible to read/write |
---|
408 | configuration space, print the current PCI configuration and print out a |
---|
409 | configuration C-file for the static or peripheral library. |
---|