1 | .. SPDX-License-Identifier: CC-BY-SA-4.0 |
---|
2 | |
---|
3 | .. Copyright (C) 2011,2015 Aeroflex Gaisler AB |
---|
4 | |
---|
5 | .. index:: PCI |
---|
6 | .. index:: libpci |
---|
7 | |
---|
8 | PCI Library |
---|
9 | *********** |
---|
10 | |
---|
11 | Introduction |
---|
12 | ============ |
---|
13 | |
---|
14 | The Peripheral Component Interconnect (PCI) bus is a very common computer bus |
---|
15 | architecture that is found in almost every PC today. The PCI bus is normally |
---|
16 | located at the motherboard where some PCI devices are soldered directly onto |
---|
17 | the PCB and expansion slots allows the user to add custom devices easily. There |
---|
18 | is a wide range of PCI hardware available implementing all sorts of interfaces |
---|
19 | and functions. |
---|
20 | |
---|
21 | This section describes the PCI Library available in RTEMS used to access the |
---|
22 | PCI bus in a portable way across computer architectures supported by RTEMS. |
---|
23 | |
---|
24 | The PCI Library aims to be compatible with PCI 2.3 with a couple of |
---|
25 | limitations, for example there is no support for hot-plugging, 64-bit memory |
---|
26 | space and cardbus bridges. |
---|
27 | |
---|
28 | In order to support different architectures and with small foot-print embedded |
---|
29 | systems in mind the PCI Library offers four different configuration options |
---|
30 | listed below. It is selected during compile time by defining the appropriate |
---|
31 | macros in confdefs.h. It is also possible to enable PCI_LIB_NONE (No |
---|
32 | Configuration) which can be used for debuging PCI access functions. |
---|
33 | |
---|
34 | - Auto Configuration (Plug & Play) |
---|
35 | |
---|
36 | - Read Configuration (read BIOS or boot loader configuration) |
---|
37 | |
---|
38 | - Static Configuration (write user defined configuration) |
---|
39 | |
---|
40 | - Peripheral Configuration (no access to cfg-space) |
---|
41 | |
---|
42 | Background |
---|
43 | ========== |
---|
44 | |
---|
45 | The PCI bus is constructed in a way where on-board devices and devices in |
---|
46 | expansion slots can be automatically found (probed) and configured using Plug & |
---|
47 | Play completely implemented in software. The bus is set up once during boot |
---|
48 | up. The Plug & Play information can be read and written from PCI configuration |
---|
49 | space. A PCI device is identified in configuration space by a unique bus, slot |
---|
50 | and function number. Each PCI slot can have up to 8 functions and interface to |
---|
51 | another PCI sub-bus by implementing a PCI-to-PCI bridge according to the PCI |
---|
52 | Bridge Architecture specification. |
---|
53 | |
---|
54 | Using the unique \[bus:slot:func] any device can be configured regardless of |
---|
55 | how PCI is currently set up as long as all PCI buses are enumerated |
---|
56 | correctly. The enumeration is done during probing, all bridges are given a bus |
---|
57 | number in order for the bridges to respond to accesses from both |
---|
58 | directions. The PCI library can assign address ranges to which a PCI device |
---|
59 | should respond using Plug & Play technique or a static user defined |
---|
60 | configuration. After the configuration has been performed the PCI device |
---|
61 | drivers can find devices by the read-only PCI Class type, Vendor ID and Device |
---|
62 | ID information found in configuration space for each device. |
---|
63 | |
---|
64 | In some systems there is a boot loader or BIOS which have already configured |
---|
65 | all PCI devices, but on embedded targets it is quite common that there is no |
---|
66 | BIOS or boot loader, thus RTEMS must configure the PCI bus. Only the PCI host |
---|
67 | may do configuration space access, the host driver or BSP is responsible to |
---|
68 | translate the \[bus:slot:func] into a valid PCI configuration space access. |
---|
69 | |
---|
70 | If the target is not a host, but a peripheral, configuration space can not be |
---|
71 | accessed, the peripheral is set up by the host during start up. In complex |
---|
72 | embedded PCI systems the peripheral may need to access other PCI boards than |
---|
73 | the host. In such systems a custom (static) configuration of both the host and |
---|
74 | peripheral may be a convenient solution. |
---|
75 | |
---|
76 | The PCI bus defines four interrupt signals INTA#..INTD#. The interrupt signals |
---|
77 | must be mapped into a system interrupt/vector, it is up to the BSP or host |
---|
78 | driver to know the mapping, however the BIOS or boot loader may use the 8-bit |
---|
79 | read/write "Interrupt Line" register to pass the knowledge along to the OS. |
---|
80 | |
---|
81 | The PCI standard defines and recommends that the backplane route the interupt |
---|
82 | lines in a systematic way, however in standard there is no such requirement. |
---|
83 | The PCI Auto Configuration Library implements the recommended way of routing |
---|
84 | which is very common but it is also supported to some extent to override the |
---|
85 | interrupt routing from the BSP or Host Bridge driver using the configuration |
---|
86 | structure. |
---|
87 | |
---|
88 | Software Components |
---|
89 | ------------------- |
---|
90 | |
---|
91 | The PCI library is located in cpukit/libpci, it consists of different parts: |
---|
92 | |
---|
93 | - PCI Host bridge driver interface |
---|
94 | |
---|
95 | - Configuration routines |
---|
96 | |
---|
97 | - Access (Configuration, I/O and Memory space) routines |
---|
98 | |
---|
99 | - Interrupt routines (implemented by BSP) |
---|
100 | |
---|
101 | - Print routines |
---|
102 | |
---|
103 | - Static/peripheral configuration creation |
---|
104 | |
---|
105 | - PCI shell command |
---|
106 | |
---|
107 | PCI Configuration |
---|
108 | ----------------- |
---|
109 | |
---|
110 | During start up the PCI bus must be configured in order for host and |
---|
111 | peripherals to access one another using Memory or I/O accesses and that |
---|
112 | interrupts are properly handled. Three different spaces are defined and mapped |
---|
113 | separately: |
---|
114 | |
---|
115 | #. I/O space (IO) |
---|
116 | |
---|
117 | #. non-prefetchable Memory space (MEMIO) |
---|
118 | |
---|
119 | #. prefetchable Memory space (MEM) |
---|
120 | |
---|
121 | Regions of the same type (I/O or Memory) may not overlap which is guaranteed by |
---|
122 | the software. MEM regions may be mapped into MEMIO regions, but MEMIO regions |
---|
123 | can not be mapped into MEM, for that could lead to prefetching of |
---|
124 | registers. The interrupt pin which a board is driving can be read out from PCI |
---|
125 | configuration space, however it is up to software to know how interrupt signals |
---|
126 | are routed between PCI-to-PCI bridges and how PCI INT[A..D]# pins are mapped to |
---|
127 | system IRQ. In systems where previous software (boot loader or BIOS) has |
---|
128 | already set up this the configuration is overwritten or simply read out. |
---|
129 | |
---|
130 | In order to support different configuration methods the following configuration |
---|
131 | libraries are selectable by the user: |
---|
132 | |
---|
133 | - Auto Configuration (run Plug & Play software) |
---|
134 | |
---|
135 | - Read Configuration (relies on a boot loader or BIOS) |
---|
136 | |
---|
137 | - Static Configuration (write user defined setup, no Plug & Play) |
---|
138 | |
---|
139 | - Peripheral Configuration (user defined setup, no access to |
---|
140 | configuration space) |
---|
141 | |
---|
142 | A host driver can be made to support all three configuration methods, or any |
---|
143 | combination. It may be defined by the BSP which approach is used. |
---|
144 | |
---|
145 | The configuration software is called from the PCI driver |
---|
146 | (``pci_config_init()``). |
---|
147 | |
---|
148 | Regardless of configuration method a PCI device tree is created in RAM during |
---|
149 | initialization, the tree can be accessed to find devices and resources without |
---|
150 | accessing configuration space later on. The user is responsible to create the |
---|
151 | device tree at compile time when using the static/peripheral method. |
---|
152 | |
---|
153 | RTEMS Configuration selection |
---|
154 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
155 | |
---|
156 | The active configuration method can be selected at compile time in the same way |
---|
157 | as other project parameters by including rtems/confdefs.h and setting |
---|
158 | |
---|
159 | - ``CONFIGURE_INIT`` |
---|
160 | |
---|
161 | - ``RTEMS_PCI_CONFIG_LIB`` |
---|
162 | |
---|
163 | - ``CONFIGURE_PCI_LIB`` = PCI_LIB_(AUTO,STATIC,READ,PERIPHERAL) |
---|
164 | |
---|
165 | See the RTEMS configuration section how to setup the PCI library. |
---|
166 | |
---|
167 | Auto Configuration |
---|
168 | ~~~~~~~~~~~~~~~~~~ |
---|
169 | |
---|
170 | The auto configuration software enumerates PCI buses and initializes all PCI |
---|
171 | devices found using Plug & Play. The auto configuration software requires that |
---|
172 | a configuration setup has been registered by the driver or BSP in order to |
---|
173 | setup the I/O and Memory regions at the correct address ranges. PCI interrupt |
---|
174 | pins can optionally be routed over PCI-to-PCI bridges and mapped to a system |
---|
175 | interrupt number. BAR resources are sorted by size and required alignment, |
---|
176 | unused "dead" space may be created when PCI bridges are present due to the PCI |
---|
177 | bridge window size does not equal the alignment. To cope with that resources |
---|
178 | are reordered to fit smaller BARs into the dead space to minimize the PCI space |
---|
179 | required. If a BAR or ROM register can not be allocated a PCI address region |
---|
180 | (due to too few resources available) the register will be given the value of |
---|
181 | pci_invalid_address which defaults to 0. |
---|
182 | |
---|
183 | The auto configuration routines support: |
---|
184 | |
---|
185 | - PCI 2.3 |
---|
186 | |
---|
187 | - Little and big endian PCI bus |
---|
188 | |
---|
189 | - one I/O 16 or 32-bit range (IO) |
---|
190 | |
---|
191 | - memory space (MEMIO) |
---|
192 | |
---|
193 | - prefetchable memory space (MEM), if not present MEM will be mapped into MEMIO |
---|
194 | |
---|
195 | - multiple PCI buses - PCI-to-PCI bridges |
---|
196 | |
---|
197 | - standard BARs, PCI-to-PCI bridge BARs, ROM BARs |
---|
198 | |
---|
199 | - Interrupt routing over bridges |
---|
200 | |
---|
201 | - Interrupt pin to system interrupt mapping |
---|
202 | |
---|
203 | Not supported: |
---|
204 | |
---|
205 | - hot-pluggable devices |
---|
206 | |
---|
207 | - Cardbus bridges |
---|
208 | |
---|
209 | - 64-bit memory space |
---|
210 | |
---|
211 | - 16-bit and 32-bit I/O address ranges at the same time |
---|
212 | |
---|
213 | In PCI 2.3 there may exist I/O BARs that must be located at the low 64kBytes |
---|
214 | address range, in order to support this the host driver or BSP must make sure |
---|
215 | that I/O addresses region is within this region. |
---|
216 | |
---|
217 | Read Configuration |
---|
218 | ~~~~~~~~~~~~~~~~~~ |
---|
219 | |
---|
220 | When a BIOS or boot loader already has setup the PCI bus the configuration can |
---|
221 | be read directly from the PCI resource registers and buses are already |
---|
222 | enumerated, this is a much simpler approach than configuring PCI ourselves. The |
---|
223 | PCI device tree is automatically created based on the current configuration and |
---|
224 | devices present. After initialization is done there is no difference between |
---|
225 | the auto or read configuration approaches. |
---|
226 | |
---|
227 | Static Configuration |
---|
228 | ~~~~~~~~~~~~~~~~~~~~ |
---|
229 | |
---|
230 | To support custom configurations and small-footprint PCI systems, the user may |
---|
231 | provide the PCI device tree which contains the current configuration. The PCI |
---|
232 | buses are enumerated and all resources are written to PCI devices during |
---|
233 | initialization. When this approach is selected PCI boards must be located at |
---|
234 | the same slots every time and devices can not be removed or added, Plug & Play |
---|
235 | is not performed. Boards of the same type may of course be exchanged. |
---|
236 | |
---|
237 | The user can create a configuration by calling pci_cfg_print() on a running |
---|
238 | system that has had PCI setup by the auto or read configuration routines, it |
---|
239 | can be called from the PCI shell command. The user must provide the PCI device |
---|
240 | tree named pci_hb. |
---|
241 | |
---|
242 | Peripheral Configuration |
---|
243 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
244 | |
---|
245 | On systems where a peripheral PCI device needs to access other PCI devices than |
---|
246 | the host the peripheral configuration approach may be handy. Most PCI devices |
---|
247 | answers on the PCI host's requests and start DMA accesses into the Hosts |
---|
248 | memory, however in some complex systems PCI devices may want to access other |
---|
249 | devices on the same bus or at another PCI bus. |
---|
250 | |
---|
251 | A PCI peripheral is not allowed to do PCI configuration cycles, which means |
---|
252 | that it must either rely on the host to give it the addresses it needs, or that |
---|
253 | the addresses are predefined. |
---|
254 | |
---|
255 | This configuration approach is very similar to the static option, however the |
---|
256 | configuration is never written to PCI bus, instead it is only used for drivers |
---|
257 | to find PCI devices and resources using the same PCI API as for the host |
---|
258 | |
---|
259 | PCI Access |
---|
260 | ---------- |
---|
261 | |
---|
262 | The PCI access routines are low-level routines provided for drivers, |
---|
263 | configuration software, etc. in order to access different regions in a way not |
---|
264 | dependent upon the host driver, BSP or platform. |
---|
265 | |
---|
266 | - PCI configuration space |
---|
267 | |
---|
268 | - PCI I/O space |
---|
269 | |
---|
270 | - Registers over PCI memory space |
---|
271 | |
---|
272 | - Translate PCI address into CPU accessible address and vice versa |
---|
273 | |
---|
274 | By using the access routines drivers can be made portable over different |
---|
275 | architectures. The access routines take the architecture endianness into |
---|
276 | consideration and let the host driver or BSP implement I/O space and |
---|
277 | configuration space access. |
---|
278 | |
---|
279 | Some non-standard hardware may also define the PCI bus big-endian, for example |
---|
280 | the LEON2 AT697 PCI host bridge and some LEON3 systems may be configured that |
---|
281 | way. It is up to the BSP to set the appropriate PCI endianness on compile time |
---|
282 | (``BSP_PCI_BIG_ENDIAN``) in order for inline macros to be correctly defined. |
---|
283 | Another possibility is to use the function pointers defined by the access layer |
---|
284 | to implement drivers that support "run-time endianness detection". |
---|
285 | |
---|
286 | Configuration space |
---|
287 | ~~~~~~~~~~~~~~~~~~~ |
---|
288 | |
---|
289 | Configuration space is accessed using the routines listed below. The pci_dev_t |
---|
290 | type is used to specify a specific PCI bus, device and function. It is up to |
---|
291 | the host driver or BSP to create a valid access to the requested PCI |
---|
292 | slot. Requests made to slots that are not supported by hardware should result |
---|
293 | in ``PCISTS_MSTABRT`` and/or data must be ignored (writes) or ``0xFFFFFFFF`` is |
---|
294 | always returned (reads). |
---|
295 | |
---|
296 | .. code-block:: c |
---|
297 | |
---|
298 | /* Configuration Space Access Read Routines */ |
---|
299 | extern int pci_cfg_r8(pci_dev_t dev, int ofs, uint8_t *data); |
---|
300 | extern int pci_cfg_r16(pci_dev_t dev, int ofs, uint16_t *data); |
---|
301 | extern int pci_cfg_r32(pci_dev_t dev, int ofs, uint32_t *data); |
---|
302 | |
---|
303 | /* Configuration Space Access Write Routines */ |
---|
304 | extern int pci_cfg_w8(pci_dev_t dev, int ofs, uint8_t data); |
---|
305 | extern int pci_cfg_w16(pci_dev_t dev, int ofs, uint16_t data); |
---|
306 | extern int pci_cfg_w32(pci_dev_t dev, int ofs, uint32_t data); |
---|
307 | |
---|
308 | I/O space |
---|
309 | ~~~~~~~~~ |
---|
310 | |
---|
311 | The BSP or driver provide special routines in order to access I/O space. Some |
---|
312 | architectures have a special instruction accessing I/O space, others have it |
---|
313 | mapped into a "PCI I/O window" in the standard address space accessed by the |
---|
314 | CPU. The window size may vary and must be taken into consideration by the host |
---|
315 | driver. The below routines must be used to access I/O space. The address given |
---|
316 | to the functions is not the PCI I/O addresses, the caller must have translated |
---|
317 | PCI I/O addresses (available in the PCI BARs) into a BSP or host driver custom |
---|
318 | address, see `Access functions`_ for how addresses are translated. |
---|
319 | |
---|
320 | .. code-block:: c |
---|
321 | |
---|
322 | /* Read a register over PCI I/O Space */ |
---|
323 | extern uint8_t pci_io_r8(uint32_t adr); |
---|
324 | extern uint16_t pci_io_r16(uint32_t adr); |
---|
325 | extern uint32_t pci_io_r32(uint32_t adr); |
---|
326 | |
---|
327 | /* Write a register over PCI I/O Space */ |
---|
328 | extern void pci_io_w8(uint32_t adr, uint8_t data); |
---|
329 | extern void pci_io_w16(uint32_t adr, uint16_t data); |
---|
330 | extern void pci_io_w32(uint32_t adr, uint32_t data); |
---|
331 | |
---|
332 | Registers over Memory space |
---|
333 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
334 | |
---|
335 | PCI host bridge hardware normally swap data accesses into the endianness of the |
---|
336 | host architecture in order to lower the load of the CPU, peripherals can do DMA |
---|
337 | without swapping. However, the host controller can not separate a standard |
---|
338 | memory access from a memory access to a register, registers may be mapped into |
---|
339 | memory space. This leads to register content being swapped, which must be |
---|
340 | swapped back. The below routines makes it possible to access registers over PCI |
---|
341 | memory space in a portable way on different architectures, the BSP or |
---|
342 | architecture must provide necessary functions in order to implement this. |
---|
343 | |
---|
344 | .. code-block:: c |
---|
345 | |
---|
346 | static inline uint16_t pci_ld_le16(volatile uint16_t *addr); |
---|
347 | static inline void pci_st_le16(volatile uint16_t *addr, uint16_t val); |
---|
348 | static inline uint32_t pci_ld_le32(volatile uint32_t *addr); |
---|
349 | static inline void pci_st_le32(volatile uint32_t *addr, uint32_t val); |
---|
350 | static inline uint16_t pci_ld_be16(volatile uint16_t *addr); |
---|
351 | static inline void pci_st_be16(volatile uint16_t *addr, uint16_t val); |
---|
352 | static inline uint32_t pci_ld_be32(volatile uint32_t *addr); |
---|
353 | static inline void pci_st_be32(volatile uint32_t *addr, uint32_t val); |
---|
354 | |
---|
355 | In order to support non-standard big-endian PCI bus the above ``pci_*`` |
---|
356 | functions is required, ``pci_ld_le16 != ld_le16`` on big endian PCI buses. |
---|
357 | |
---|
358 | Access functions |
---|
359 | ~~~~~~~~~~~~~~~~ |
---|
360 | |
---|
361 | The PCI Access Library can provide device drivers with function pointers |
---|
362 | executing the above Configuration, I/O and Memory space accesses. The functions |
---|
363 | have the same arguments and return values as the above functions. |
---|
364 | |
---|
365 | The pci_access_func() function defined below can be used to get a function |
---|
366 | pointer of a specific access type. |
---|
367 | |
---|
368 | .. code-block:: c |
---|
369 | |
---|
370 | /* Get Read/Write function for accessing a register over PCI Memory Space |
---|
371 | * (non-inline functions). |
---|
372 | * |
---|
373 | * Arguments |
---|
374 | * wr 0(Read), 1(Write) |
---|
375 | * size 1(Byte), 2(Word), 4(Double Word) |
---|
376 | * func Where function pointer will be stored |
---|
377 | * endian PCI_LITTLE_ENDIAN or PCI_BIG_ENDIAN |
---|
378 | * type 1(I/O), 3(REG over MEM), 4(CFG) |
---|
379 | * |
---|
380 | * Return |
---|
381 | * 0 Found function |
---|
382 | * others No such function defined by host driver or BSP |
---|
383 | */ |
---|
384 | int pci_access_func(int wr, int size, void **func, int endian, int type); |
---|
385 | |
---|
386 | PCI device drivers may be written to support run-time detection of endianess, |
---|
387 | this is mosly for debugging or for development systems. When the product is |
---|
388 | finally deployed macros switch to using the inline functions instead which have |
---|
389 | been configured for the correct endianness. |
---|
390 | |
---|
391 | .. index:: PCI address translation |
---|
392 | |
---|
393 | PCI address translation |
---|
394 | ~~~~~~~~~~~~~~~~~~~~~~~ |
---|
395 | |
---|
396 | When PCI addresses, both I/O and memory space, is not mapped 1:1 address |
---|
397 | translation before access is needed. If drivers read the PCI resources directly |
---|
398 | using configuration space routines or in the device tree, the addresses given |
---|
399 | are PCI addresses. The below functions can be used to translate PCI addresses |
---|
400 | into CPU accessible addresses or vice versa, translation may be different for |
---|
401 | different PCI spaces/regions. |
---|
402 | |
---|
403 | .. code-block:: c |
---|
404 | |
---|
405 | /* Translate PCI address into CPU accessible address */ |
---|
406 | static inline int pci_pci2cpu(uint32_t *address, int type); |
---|
407 | |
---|
408 | /* Translate CPU accessible address into PCI address (for DMA) */ |
---|
409 | static inline int pci_cpu2pci(uint32_t *address, int type); |
---|
410 | |
---|
411 | .. index:: PCI Interrupt |
---|
412 | |
---|
413 | PCI Interrupt |
---|
414 | ------------- |
---|
415 | |
---|
416 | The PCI specification defines four different interrupt lines INTA#..INTD#, the |
---|
417 | interrupts are low level sensitive which make it possible to support multiple |
---|
418 | interrupt sources on the same interrupt line. Since the lines are level |
---|
419 | sensitive the interrupt sources must be acknowledged before clearing the |
---|
420 | interrupt contoller, or the interrupt controller must be masked. The BSP must |
---|
421 | provide a routine for clearing/acknowledging the interrupt controller, it is up |
---|
422 | to the interrupt service routine to acknowledge the interrupt source. |
---|
423 | |
---|
424 | The PCI Library relies on the BSP for implementing shared interrupt handling |
---|
425 | through the BSP_PCI_shared_interrupt_* functions/macros, they must be defined |
---|
426 | when including bsp.h. |
---|
427 | |
---|
428 | PCI device drivers may use the pci_interrupt_* routines in order to call the |
---|
429 | BSP specific functions in a platform independent way. The PCI interrupt |
---|
430 | interface has been made similar to the RTEMS IRQ extension so that a BSP can |
---|
431 | use the standard RTEMS interrupt functions directly. |
---|
432 | |
---|
433 | PCI Shell command |
---|
434 | ----------------- |
---|
435 | |
---|
436 | The RTEMS shell has a PCI command 'pci' which makes it possible to read/write |
---|
437 | configuration space, print the current PCI configuration and print out a |
---|
438 | configuration C-file for the static or peripheral library. |
---|