1 | PCI Library |
---|
2 | ########### |
---|
3 | |
---|
4 | .. index:: libpci |
---|
5 | |
---|
6 | Introduction |
---|
7 | ============ |
---|
8 | |
---|
9 | The Peripheral Component Interconnect (PCI) bus is a very common computer |
---|
10 | bus architecture that is found in almost every PC today. The PCI bus is |
---|
11 | normally located at the motherboard where some PCI devices are soldered |
---|
12 | directly onto the PCB and expansion slots allows the user to add custom |
---|
13 | devices easily. There is a wide range of PCI hardware available implementing |
---|
14 | all sorts of interfaces and functions. |
---|
15 | |
---|
16 | This section describes the PCI Library available in RTEMS used to access the |
---|
17 | PCI bus in a portable way across computer architectures supported by RTEMS. |
---|
18 | |
---|
19 | The PCI Library aims to be compatible with PCI 2.3 with a couple of |
---|
20 | limitations, for example there is no support for hot-plugging, 64-bit |
---|
21 | memory space and cardbus bridges. |
---|
22 | |
---|
23 | In order to support different architectures and with small foot-print embedded |
---|
24 | systems in mind the PCI Library offers four different configuration options |
---|
25 | listed below. It is selected during compile time by defining the appropriate |
---|
26 | macros in confdefs.h. It is also possible to enable PCI_LIB_NONE (No |
---|
27 | Configuration) which can be used for debuging PCI access functions. |
---|
28 | |
---|
29 | - Auto Configuration (do Plug & Play) |
---|
30 | |
---|
31 | - Read Configuration (read BIOS or boot loader configuration) |
---|
32 | |
---|
33 | - Static Configuration (write user defined configuration) |
---|
34 | |
---|
35 | - Peripheral Configuration (no access to cfg-space) |
---|
36 | |
---|
37 | Background |
---|
38 | ========== |
---|
39 | |
---|
40 | The PCI bus is constructed in a way where on-board devices and devices |
---|
41 | in expansion slots can be automatically found (probed) and configured |
---|
42 | using Plug & Play completely implemented in software. The bus is set up once |
---|
43 | during boot up. The Plug & Play information can be read and written from |
---|
44 | PCI configuration space. A PCI device is identified in configuration space by |
---|
45 | a unique bus, slot and function number. Each PCI slot can have up to 8 |
---|
46 | functions and interface to another PCI sub-bus by implementing a PCI-to-PCI |
---|
47 | bridge according to the PCI Bridge Architecture specification. |
---|
48 | |
---|
49 | Using the unique \[bus:slot:func] any device can be configured regardless of how |
---|
50 | PCI is currently set up as long as all PCI buses are enumerated correctly. The |
---|
51 | enumeration is done during probing, all bridges are given a bus number in |
---|
52 | order for the bridges to respond to accesses from both directions. The PCI |
---|
53 | library can assign address ranges to which a PCI device should respond using |
---|
54 | Plug & Play technique or a static user defined configuration. After the |
---|
55 | configuration has been performed the PCI device drivers can find devices by |
---|
56 | the read-only PCI Class type, Vendor ID and Device ID information found in |
---|
57 | configuration space for each device. |
---|
58 | |
---|
59 | In some systems there is a boot loader or BIOS which have already configured |
---|
60 | all PCI devices, but on embedded targets it is quite common that there is no |
---|
61 | BIOS or boot loader, thus RTEMS must configure the PCI bus. Only the PCI host |
---|
62 | may do configuration space access, the host driver or BSP is responsible to |
---|
63 | translate the \[bus:slot:func] into a valid PCI configuration space access. |
---|
64 | |
---|
65 | If the target is not a host, but a peripheral, configuration space can not be |
---|
66 | accessed, the peripheral is set up by the host during start up. In complex |
---|
67 | embedded PCI systems the peripheral may need to access other PCI boards than |
---|
68 | the host. In such systems a custom (static) configuration of both the host |
---|
69 | and peripheral may be a convenient solution. |
---|
70 | |
---|
71 | The PCI bus defines four interrupt signals INTA#..INTD#. The interrupt signals |
---|
72 | must be mapped into a system interrupt/vector, it is up to the BSP or host |
---|
73 | driver to know the mapping, however the BIOS or boot loader may use the |
---|
74 | 8-bit read/write "Interrupt Line" register to pass the knowledge along to the |
---|
75 | OS. |
---|
76 | |
---|
77 | The PCI standard defines and recommends that the backplane route the interupt |
---|
78 | lines in a systematic way, however in standard there is no such requirement. |
---|
79 | The PCI Auto Configuration Library implements the recommended way of routing |
---|
80 | which is very common but it is also supported to some extent to override the |
---|
81 | interrupt routing from the BSP or Host Bridge driver using the configuration |
---|
82 | structure. |
---|
83 | |
---|
84 | Software Components |
---|
85 | ------------------- |
---|
86 | |
---|
87 | The PCI library is located in cpukit/libpci, it consists of different parts: |
---|
88 | |
---|
89 | - PCI Host bridge driver interface |
---|
90 | |
---|
91 | - Configuration routines |
---|
92 | |
---|
93 | - Access (Configuration, I/O and Memory space) routines |
---|
94 | |
---|
95 | - Interrupt routines (implemented by BSP) |
---|
96 | |
---|
97 | - Print routines |
---|
98 | |
---|
99 | - Static/peripheral configuration creation |
---|
100 | |
---|
101 | - PCI shell command |
---|
102 | |
---|
103 | PCI Configuration |
---|
104 | ----------------- |
---|
105 | |
---|
106 | During start up the PCI bus must be configured in order for host and |
---|
107 | peripherals to access one another using Memory or I/O accesses and that |
---|
108 | interrupts are properly handled. Three different spaces are defined and |
---|
109 | mapped separately: |
---|
110 | |
---|
111 | # I/O space (IO) |
---|
112 | |
---|
113 | # non-prefetchable Memory space (MEMIO) |
---|
114 | |
---|
115 | # prefetchable Memory space (MEM) |
---|
116 | |
---|
117 | Regions of the same type (I/O or Memory) may not overlap which is guaranteed |
---|
118 | by the software. MEM regions may be mapped into MEMIO regions, but MEMIO |
---|
119 | regions can not be mapped into MEM, for that could lead to prefetching of |
---|
120 | registers. The interrupt pin which a board is driving can be read out from |
---|
121 | PCI configuration space, however it is up to software to know how interrupt |
---|
122 | signals are routed between PCI-to-PCI bridges and how PCI INT[A..D]# pins are |
---|
123 | mapped to system IRQ. In systems where previous software (boot loader or BIOS) |
---|
124 | has already set up this the configuration is overwritten or simply read out. |
---|
125 | |
---|
126 | In order to support different configuration methods the following configuration |
---|
127 | libraries are selectable by the user: |
---|
128 | |
---|
129 | - Auto Configuration (run Plug & Play software) |
---|
130 | |
---|
131 | - Read Configuration (relies on a boot loader or BIOS) |
---|
132 | |
---|
133 | - Static Configuration (write user defined setup, no Plug & Play) |
---|
134 | |
---|
135 | - Peripheral Configuration (user defined setup, no access to |
---|
136 | configuration space) |
---|
137 | |
---|
138 | A host driver can be made to support all three configuration methods, or any |
---|
139 | combination. It may be defined by the BSP which approach is used. |
---|
140 | |
---|
141 | The configuration software is called from the PCI driver (pci_config_init()). |
---|
142 | |
---|
143 | Regardless of configuration method a PCI device tree is created in RAM during |
---|
144 | initialization, the tree can be accessed to find devices and resources without |
---|
145 | accessing configuration space later on. The user is responsible to create the |
---|
146 | device tree at compile time when using the static/peripheral method. |
---|
147 | |
---|
148 | RTEMS Configuration selection |
---|
149 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
150 | |
---|
151 | The active configuration method can be selected at compile time in the same |
---|
152 | way as other project parameters by including rtems/confdefs.h and setting |
---|
153 | |
---|
154 | - CONFIGURE_INIT |
---|
155 | |
---|
156 | - RTEMS_PCI_CONFIG_LIB |
---|
157 | |
---|
158 | - CONFIGURE_PCI_LIB = PCI_LIB_(AUTO,STATIC,READ,PERIPHERAL) |
---|
159 | |
---|
160 | See the RTEMS configuration section how to setup the PCI library. |
---|
161 | |
---|
162 | Auto Configuration |
---|
163 | ~~~~~~~~~~~~~~~~~~ |
---|
164 | |
---|
165 | The auto configuration software enumerates PCI buses and initializes all PCI |
---|
166 | devices found using Plug & Play. The auto configuration software requires |
---|
167 | that a configuration setup has been registered by the driver or BSP in order |
---|
168 | to setup the I/O and Memory regions at the correct address ranges. PCI |
---|
169 | interrupt pins can optionally be routed over PCI-to-PCI bridges and mapped |
---|
170 | to a system interrupt number. BAR resources are sorted by size and required |
---|
171 | alignment, unused "dead" space may be created when PCI bridges are present |
---|
172 | due to the PCI bridge window size does not equal the alignment. To cope with |
---|
173 | that resources are reordered to fit smaller BARs into the dead space to minimize |
---|
174 | the PCI space required. If a BAR or ROM register can not be allocated a PCI |
---|
175 | address region (due to too few resources available) the register will be given |
---|
176 | the value of pci_invalid_address which defaults to 0. |
---|
177 | |
---|
178 | The auto configuration routines support: |
---|
179 | |
---|
180 | - PCI 2.3 |
---|
181 | |
---|
182 | - Little and big endian PCI bus |
---|
183 | |
---|
184 | - one I/O 16 or 32-bit range (IO) |
---|
185 | |
---|
186 | - memory space (MEMIO) |
---|
187 | |
---|
188 | - prefetchable memory space (MEM), if not present MEM will be mapped into |
---|
189 | MEMIO |
---|
190 | |
---|
191 | - multiple PCI buses - PCI-to-PCI bridges |
---|
192 | |
---|
193 | - standard BARs, PCI-to-PCI bridge BARs, ROM BARs |
---|
194 | |
---|
195 | - Interrupt routing over bridges |
---|
196 | |
---|
197 | - Interrupt pin to system interrupt mapping |
---|
198 | |
---|
199 | Not supported: |
---|
200 | |
---|
201 | - hot-pluggable devices |
---|
202 | |
---|
203 | - Cardbus bridges |
---|
204 | |
---|
205 | - 64-bit memory space |
---|
206 | |
---|
207 | - 16-bit and 32-bit I/O address ranges at the same time |
---|
208 | |
---|
209 | In PCI 2.3 there may exist I/O BARs that must be located at the low 64kBytes |
---|
210 | address range, in order to support this the host driver or BSP must make sure |
---|
211 | that I/O addresses region is within this region. |
---|
212 | |
---|
213 | Read Configuration |
---|
214 | ~~~~~~~~~~~~~~~~~~ |
---|
215 | |
---|
216 | When a BIOS or boot loader already has setup the PCI bus the configuration can |
---|
217 | be read directly from the PCI resource registers and buses are already |
---|
218 | enumerated, this is a much simpler approach than configuring PCI ourselves. The |
---|
219 | PCI device tree is automatically created based on the current configuration and |
---|
220 | devices present. After initialization is done there is no difference between |
---|
221 | the auto or read configuration approaches. |
---|
222 | |
---|
223 | Static Configuration |
---|
224 | ~~~~~~~~~~~~~~~~~~~~ |
---|
225 | |
---|
226 | To support custom configurations and small-footprint PCI systems, the user may |
---|
227 | provide the PCI device tree which contains the current configuration. The |
---|
228 | PCI buses are enumerated and all resources are written to PCI devices during |
---|
229 | initialization. When this approach is selected PCI boards must be located at |
---|
230 | the same slots every time and devices can not be removed or added, Plug & Play |
---|
231 | is not performed. Boards of the same type may of course be exchanged. |
---|
232 | |
---|
233 | The user can create a configuration by calling pci_cfg_print() on a running |
---|
234 | system that has had PCI setup by the auto or read configuration routines, it |
---|
235 | can be called from the PCI shell command. The user must provide the PCI device |
---|
236 | tree named pci_hb. |
---|
237 | |
---|
238 | Peripheral Configuration |
---|
239 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
240 | |
---|
241 | On systems where a peripheral PCI device needs to access other PCI devices than |
---|
242 | the host the peripheral configuration approach may be handy. Most PCI devices |
---|
243 | answers on the PCI host's requests and start DMA accesses into the Hosts memory, |
---|
244 | however in some complex systems PCI devices may want to access other devices |
---|
245 | on the same bus or at another PCI bus. |
---|
246 | |
---|
247 | A PCI peripheral is not allowed to do PCI configuration cycles, which |
---|
248 | means that it must either rely on the host to give it the addresses it |
---|
249 | needs, or that the addresses are predefined. |
---|
250 | |
---|
251 | This configuration approach is very similar to the static option, however the |
---|
252 | configuration is never written to PCI bus, instead it is only used for drivers |
---|
253 | to find PCI devices and resources using the same PCI API as for the host |
---|
254 | |
---|
255 | PCI Access |
---|
256 | ---------- |
---|
257 | |
---|
258 | The PCI access routines are low-level routines provided for drivers, |
---|
259 | configuration software, etc. in order to access different regions in a way |
---|
260 | not dependent upon the host driver, BSP or platform. |
---|
261 | |
---|
262 | - PCI configuration space |
---|
263 | |
---|
264 | - PCI I/O space |
---|
265 | |
---|
266 | - Registers over PCI memory space |
---|
267 | |
---|
268 | - Translate PCI address into CPU accessible address and vice versa |
---|
269 | |
---|
270 | By using the access routines drivers can be made portable over different |
---|
271 | architectures. The access routines take the architecture endianness into |
---|
272 | consideration and let the host driver or BSP implement I/O space and |
---|
273 | configuration space access. |
---|
274 | |
---|
275 | Some non-standard hardware may also define the PCI bus big-endian, for example |
---|
276 | the LEON2 AT697 PCI host bridge and some LEON3 systems may be configured that |
---|
277 | way. It is up to the BSP to set the appropriate PCI endianness on compile time |
---|
278 | (BSP_PCI_BIG_ENDIAN) in order for inline macros to be correctly defined. |
---|
279 | Another possibility is to use the function pointers defined by the access |
---|
280 | layer to implement drivers that support "run-time endianness detection". |
---|
281 | |
---|
282 | Configuration space |
---|
283 | ~~~~~~~~~~~~~~~~~~~ |
---|
284 | |
---|
285 | Configuration space is accessed using the routines listed below. The |
---|
286 | pci_dev_t type is used to specify a specific PCI bus, device and function. It |
---|
287 | is up to the host driver or BSP to create a valid access to the requested |
---|
288 | PCI slot. Requests made to slots that are not supported by hardware should |
---|
289 | result in PCISTS_MSTABRT and/or data must be ignored (writes) or 0xffffffff |
---|
290 | is always returned (reads). |
---|
291 | .. code:: c |
---|
292 | |
---|
293 | /* Configuration Space Access Read Routines \*/ |
---|
294 | extern int pci_cfg_r8(pci_dev_t dev, int ofs, uint8_t \*data); |
---|
295 | extern int pci_cfg_r16(pci_dev_t dev, int ofs, uint16_t \*data); |
---|
296 | extern int pci_cfg_r32(pci_dev_t dev, int ofs, uint32_t \*data); |
---|
297 | /* Configuration Space Access Write Routines \*/ |
---|
298 | extern int pci_cfg_w8(pci_dev_t dev, int ofs, uint8_t data); |
---|
299 | extern int pci_cfg_w16(pci_dev_t dev, int ofs, uint16_t data); |
---|
300 | extern int pci_cfg_w32(pci_dev_t dev, int ofs, uint32_t data); |
---|
301 | |
---|
302 | I/O space |
---|
303 | ~~~~~~~~~ |
---|
304 | |
---|
305 | The BSP or driver provide special routines in order to access I/O space. Some |
---|
306 | architectures have a special instruction accessing I/O space, others have it |
---|
307 | mapped into a "PCI I/O window" in the standard address space accessed by the |
---|
308 | CPU. The window size may vary and must be taken into consideration by the |
---|
309 | host driver. The below routines must be used to access I/O space. The address |
---|
310 | given to the functions is not the PCI I/O addresses, the caller must have |
---|
311 | translated PCI I/O addresses (available in the PCI BARs) into a BSP or host |
---|
312 | driver custom address, see `Access functions`_ for how |
---|
313 | addresses are translated. |
---|
314 | |
---|
315 | .. code:: c |
---|
316 | |
---|
317 | /* Read a register over PCI I/O Space \*/ |
---|
318 | extern uint8_t pci_io_r8(uint32_t adr); |
---|
319 | extern uint16_t pci_io_r16(uint32_t adr); |
---|
320 | extern uint32_t pci_io_r32(uint32_t adr); |
---|
321 | /* Write a register over PCI I/O Space \*/ |
---|
322 | extern void pci_io_w8(uint32_t adr, uint8_t data); |
---|
323 | extern void pci_io_w16(uint32_t adr, uint16_t data); |
---|
324 | extern void pci_io_w32(uint32_t adr, uint32_t data); |
---|
325 | |
---|
326 | Registers over Memory space |
---|
327 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
328 | |
---|
329 | PCI host bridge hardware normally swap data accesses into the endianness of the |
---|
330 | host architecture in order to lower the load of the CPU, peripherals can do DMA |
---|
331 | without swapping. However, the host controller can not separate a standard |
---|
332 | memory access from a memory access to a register, registers may be mapped into |
---|
333 | memory space. This leads to register content being swapped, which must be |
---|
334 | swapped back. The below routines makes it possible to access registers over PCI |
---|
335 | memory space in a portable way on different architectures, the BSP or |
---|
336 | architecture must provide necessary functions in order to implement this. |
---|
337 | .. code:: c |
---|
338 | |
---|
339 | static inline uint16_t pci_ld_le16(volatile uint16_t \*addr); |
---|
340 | static inline void pci_st_le16(volatile uint16_t \*addr, uint16_t val); |
---|
341 | static inline uint32_t pci_ld_le32(volatile uint32_t \*addr); |
---|
342 | static inline void pci_st_le32(volatile uint32_t \*addr, uint32_t val); |
---|
343 | static inline uint16_t pci_ld_be16(volatile uint16_t \*addr); |
---|
344 | static inline void pci_st_be16(volatile uint16_t \*addr, uint16_t val); |
---|
345 | static inline uint32_t pci_ld_be32(volatile uint32_t \*addr); |
---|
346 | static inline void pci_st_be32(volatile uint32_t \*addr, uint32_t val); |
---|
347 | |
---|
348 | In order to support non-standard big-endian PCI bus the above pci_* functions |
---|
349 | is required, pci_ld_le16 != ld_le16 on big endian PCI buses. |
---|
350 | |
---|
351 | Access functions |
---|
352 | ~~~~~~~~~~~~~~~~ |
---|
353 | |
---|
354 | The PCI Access Library can provide device drivers with function pointers |
---|
355 | executing the above Configuration, I/O and Memory space accesses. The |
---|
356 | functions have the same arguments and return values as the above |
---|
357 | functions. |
---|
358 | |
---|
359 | The pci_access_func() function defined below can be used to get a function |
---|
360 | pointer of a specific access type. |
---|
361 | .. code:: c |
---|
362 | |
---|
363 | /* Get Read/Write function for accessing a register over PCI Memory Space |
---|
364 | * (non-inline functions). |
---|
365 | * |
---|
366 | * Arguments |
---|
367 | * wr 0(Read), 1(Write) |
---|
368 | * size 1(Byte), 2(Word), 4(Double Word) |
---|
369 | * func Where function pointer will be stored |
---|
370 | * endian PCI_LITTLE_ENDIAN or PCI_BIG_ENDIAN |
---|
371 | * type 1(I/O), 3(REG over MEM), 4(CFG) |
---|
372 | * |
---|
373 | * Return |
---|
374 | * 0 Found function |
---|
375 | * others No such function defined by host driver or BSP |
---|
376 | \*/ |
---|
377 | int pci_access_func(int wr, int size, void \**func, int endian, int type); |
---|
378 | |
---|
379 | PCI device drivers may be written to support run-time detection of endianess, |
---|
380 | this is mosly for debugging or for development systems. When the product is |
---|
381 | finally deployed macros switch to using the inline functions instead which |
---|
382 | have been configured for the correct endianness. |
---|
383 | |
---|
384 | PCI address translation |
---|
385 | ~~~~~~~~~~~~~~~~~~~~~~~ |
---|
386 | |
---|
387 | When PCI addresses, both I/O and memory space, is not mapped 1:1 address |
---|
388 | translation before access is needed. If drivers read the PCI resources directly |
---|
389 | using configuration space routines or in the device tree, the addresses given |
---|
390 | are PCI addresses. The below functions can be used to translate PCI addresses |
---|
391 | into CPU accessible addresses or vice versa, translation may be different for |
---|
392 | different PCI spaces/regions. |
---|
393 | .. code:: c |
---|
394 | |
---|
395 | /* Translate PCI address into CPU accessible address \*/ |
---|
396 | static inline int pci_pci2cpu(uint32_t \*address, int type); |
---|
397 | /* Translate CPU accessible address into PCI address (for DMA) \*/ |
---|
398 | static inline int pci_cpu2pci(uint32_t \*address, int type); |
---|
399 | |
---|
400 | PCI Interrupt |
---|
401 | ------------- |
---|
402 | |
---|
403 | The PCI specification defines four different interrupt lines INTA#..INTD#, |
---|
404 | the interrupts are low level sensitive which make it possible to support |
---|
405 | multiple interrupt sources on the same interrupt line. Since the lines are |
---|
406 | level sensitive the interrupt sources must be acknowledged before clearing the |
---|
407 | interrupt contoller, or the interrupt controller must be masked. The BSP must |
---|
408 | provide a routine for clearing/acknowledging the interrupt controller, it is |
---|
409 | up to the interrupt service routine to acknowledge the interrupt source. |
---|
410 | |
---|
411 | The PCI Library relies on the BSP for implementing shared interrupt handling |
---|
412 | through the BSP_PCI_shared_interrupt_* functions/macros, they must be defined |
---|
413 | when including bsp.h. |
---|
414 | |
---|
415 | PCI device drivers may use the pci_interrupt_* routines in order to call the |
---|
416 | BSP specific functions in a platform independent way. The PCI interrupt |
---|
417 | interface has been made similar to the RTEMS IRQ extension so that a BSP can |
---|
418 | use the standard RTEMS interrupt functions directly. |
---|
419 | |
---|
420 | PCI Shell command |
---|
421 | ----------------- |
---|
422 | |
---|
423 | The RTEMS shell has a PCI command 'pci' which makes it possible to read/write |
---|
424 | configuration space, print the current PCI configuration and print out a |
---|
425 | configuration C-file for the static or peripheral library. |
---|
426 | |
---|
427 | .. COMMENT: COPYRIGHT (c) 1988-2007. |
---|
428 | |
---|
429 | .. COMMENT: On-Line Applications Research Corporation (OAR). |
---|
430 | |
---|
431 | .. COMMENT: All rights reserved. |
---|
432 | |
---|