1 | .. comment SPDX-License-Identifier: CC-BY-SA-4.0 |
---|
2 | |
---|
3 | .. COMMENT: COPYRIGHT (c) 2011,2015 |
---|
4 | .. COMMENT: Aeroflex Gaisler AB |
---|
5 | .. COMMENT: All rights reserved. |
---|
6 | |
---|
7 | .. index:: PCI |
---|
8 | .. index:: libpci |
---|
9 | |
---|
10 | PCI Library |
---|
11 | *********** |
---|
12 | |
---|
13 | Introduction |
---|
14 | ============ |
---|
15 | |
---|
16 | The Peripheral Component Interconnect (PCI) bus is a very common computer bus |
---|
17 | architecture that is found in almost every PC today. The PCI bus is normally |
---|
18 | located at the motherboard where some PCI devices are soldered directly onto |
---|
19 | the PCB and expansion slots allows the user to add custom devices easily. There |
---|
20 | is a wide range of PCI hardware available implementing all sorts of interfaces |
---|
21 | and functions. |
---|
22 | |
---|
23 | This section describes the PCI Library available in RTEMS used to access the |
---|
24 | PCI bus in a portable way across computer architectures supported by RTEMS. |
---|
25 | |
---|
26 | The PCI Library aims to be compatible with PCI 2.3 with a couple of |
---|
27 | limitations, for example there is no support for hot-plugging, 64-bit memory |
---|
28 | space and cardbus bridges. |
---|
29 | |
---|
30 | In order to support different architectures and with small foot-print embedded |
---|
31 | systems in mind the PCI Library offers four different configuration options |
---|
32 | listed below. It is selected during compile time by defining the appropriate |
---|
33 | macros in confdefs.h. It is also possible to enable PCI_LIB_NONE (No |
---|
34 | Configuration) which can be used for debuging PCI access functions. |
---|
35 | |
---|
36 | - Auto Configuration (Plug & Play) |
---|
37 | |
---|
38 | - Read Configuration (read BIOS or boot loader configuration) |
---|
39 | |
---|
40 | - Static Configuration (write user defined configuration) |
---|
41 | |
---|
42 | - Peripheral Configuration (no access to cfg-space) |
---|
43 | |
---|
44 | Background |
---|
45 | ========== |
---|
46 | |
---|
47 | The PCI bus is constructed in a way where on-board devices and devices in |
---|
48 | expansion slots can be automatically found (probed) and configured using Plug & |
---|
49 | Play completely implemented in software. The bus is set up once during boot |
---|
50 | up. The Plug & Play information can be read and written from PCI configuration |
---|
51 | space. A PCI device is identified in configuration space by a unique bus, slot |
---|
52 | and function number. Each PCI slot can have up to 8 functions and interface to |
---|
53 | another PCI sub-bus by implementing a PCI-to-PCI bridge according to the PCI |
---|
54 | Bridge Architecture specification. |
---|
55 | |
---|
56 | Using the unique \[bus:slot:func] any device can be configured regardless of |
---|
57 | how PCI is currently set up as long as all PCI buses are enumerated |
---|
58 | correctly. The enumeration is done during probing, all bridges are given a bus |
---|
59 | number in order for the bridges to respond to accesses from both |
---|
60 | directions. The PCI library can assign address ranges to which a PCI device |
---|
61 | should respond using Plug & Play technique or a static user defined |
---|
62 | configuration. After the configuration has been performed the PCI device |
---|
63 | drivers can find devices by the read-only PCI Class type, Vendor ID and Device |
---|
64 | ID information found in configuration space for each device. |
---|
65 | |
---|
66 | In some systems there is a boot loader or BIOS which have already configured |
---|
67 | all PCI devices, but on embedded targets it is quite common that there is no |
---|
68 | BIOS or boot loader, thus RTEMS must configure the PCI bus. Only the PCI host |
---|
69 | may do configuration space access, the host driver or BSP is responsible to |
---|
70 | translate the \[bus:slot:func] into a valid PCI configuration space access. |
---|
71 | |
---|
72 | If the target is not a host, but a peripheral, configuration space can not be |
---|
73 | accessed, the peripheral is set up by the host during start up. In complex |
---|
74 | embedded PCI systems the peripheral may need to access other PCI boards than |
---|
75 | the host. In such systems a custom (static) configuration of both the host and |
---|
76 | peripheral may be a convenient solution. |
---|
77 | |
---|
78 | The PCI bus defines four interrupt signals INTA#..INTD#. The interrupt signals |
---|
79 | must be mapped into a system interrupt/vector, it is up to the BSP or host |
---|
80 | driver to know the mapping, however the BIOS or boot loader may use the 8-bit |
---|
81 | read/write "Interrupt Line" register to pass the knowledge along to the OS. |
---|
82 | |
---|
83 | The PCI standard defines and recommends that the backplane route the interupt |
---|
84 | lines in a systematic way, however in standard there is no such requirement. |
---|
85 | The PCI Auto Configuration Library implements the recommended way of routing |
---|
86 | which is very common but it is also supported to some extent to override the |
---|
87 | interrupt routing from the BSP or Host Bridge driver using the configuration |
---|
88 | structure. |
---|
89 | |
---|
90 | Software Components |
---|
91 | ------------------- |
---|
92 | |
---|
93 | The PCI library is located in cpukit/libpci, it consists of different parts: |
---|
94 | |
---|
95 | - PCI Host bridge driver interface |
---|
96 | |
---|
97 | - Configuration routines |
---|
98 | |
---|
99 | - Access (Configuration, I/O and Memory space) routines |
---|
100 | |
---|
101 | - Interrupt routines (implemented by BSP) |
---|
102 | |
---|
103 | - Print routines |
---|
104 | |
---|
105 | - Static/peripheral configuration creation |
---|
106 | |
---|
107 | - PCI shell command |
---|
108 | |
---|
109 | PCI Configuration |
---|
110 | ----------------- |
---|
111 | |
---|
112 | During start up the PCI bus must be configured in order for host and |
---|
113 | peripherals to access one another using Memory or I/O accesses and that |
---|
114 | interrupts are properly handled. Three different spaces are defined and mapped |
---|
115 | separately: |
---|
116 | |
---|
117 | #. I/O space (IO) |
---|
118 | |
---|
119 | #. non-prefetchable Memory space (MEMIO) |
---|
120 | |
---|
121 | #. prefetchable Memory space (MEM) |
---|
122 | |
---|
123 | Regions of the same type (I/O or Memory) may not overlap which is guaranteed by |
---|
124 | the software. MEM regions may be mapped into MEMIO regions, but MEMIO regions |
---|
125 | can not be mapped into MEM, for that could lead to prefetching of |
---|
126 | registers. The interrupt pin which a board is driving can be read out from PCI |
---|
127 | configuration space, however it is up to software to know how interrupt signals |
---|
128 | are routed between PCI-to-PCI bridges and how PCI INT[A..D]# pins are mapped to |
---|
129 | system IRQ. In systems where previous software (boot loader or BIOS) has |
---|
130 | already set up this the configuration is overwritten or simply read out. |
---|
131 | |
---|
132 | In order to support different configuration methods the following configuration |
---|
133 | libraries are selectable by the user: |
---|
134 | |
---|
135 | - Auto Configuration (run Plug & Play software) |
---|
136 | |
---|
137 | - Read Configuration (relies on a boot loader or BIOS) |
---|
138 | |
---|
139 | - Static Configuration (write user defined setup, no Plug & Play) |
---|
140 | |
---|
141 | - Peripheral Configuration (user defined setup, no access to |
---|
142 | configuration space) |
---|
143 | |
---|
144 | A host driver can be made to support all three configuration methods, or any |
---|
145 | combination. It may be defined by the BSP which approach is used. |
---|
146 | |
---|
147 | The configuration software is called from the PCI driver |
---|
148 | (``pci_config_init()``). |
---|
149 | |
---|
150 | Regardless of configuration method a PCI device tree is created in RAM during |
---|
151 | initialization, the tree can be accessed to find devices and resources without |
---|
152 | accessing configuration space later on. The user is responsible to create the |
---|
153 | device tree at compile time when using the static/peripheral method. |
---|
154 | |
---|
155 | RTEMS Configuration selection |
---|
156 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
157 | |
---|
158 | The active configuration method can be selected at compile time in the same way |
---|
159 | as other project parameters by including rtems/confdefs.h and setting |
---|
160 | |
---|
161 | - ``CONFIGURE_INIT`` |
---|
162 | |
---|
163 | - ``RTEMS_PCI_CONFIG_LIB`` |
---|
164 | |
---|
165 | - ``CONFIGURE_PCI_LIB`` = PCI_LIB_(AUTO,STATIC,READ,PERIPHERAL) |
---|
166 | |
---|
167 | See the RTEMS configuration section how to setup the PCI library. |
---|
168 | |
---|
169 | Auto Configuration |
---|
170 | ~~~~~~~~~~~~~~~~~~ |
---|
171 | |
---|
172 | The auto configuration software enumerates PCI buses and initializes all PCI |
---|
173 | devices found using Plug & Play. The auto configuration software requires that |
---|
174 | a configuration setup has been registered by the driver or BSP in order to |
---|
175 | setup the I/O and Memory regions at the correct address ranges. PCI interrupt |
---|
176 | pins can optionally be routed over PCI-to-PCI bridges and mapped to a system |
---|
177 | interrupt number. BAR resources are sorted by size and required alignment, |
---|
178 | unused "dead" space may be created when PCI bridges are present due to the PCI |
---|
179 | bridge window size does not equal the alignment. To cope with that resources |
---|
180 | are reordered to fit smaller BARs into the dead space to minimize the PCI space |
---|
181 | required. If a BAR or ROM register can not be allocated a PCI address region |
---|
182 | (due to too few resources available) the register will be given the value of |
---|
183 | pci_invalid_address which defaults to 0. |
---|
184 | |
---|
185 | The auto configuration routines support: |
---|
186 | |
---|
187 | - PCI 2.3 |
---|
188 | |
---|
189 | - Little and big endian PCI bus |
---|
190 | |
---|
191 | - one I/O 16 or 32-bit range (IO) |
---|
192 | |
---|
193 | - memory space (MEMIO) |
---|
194 | |
---|
195 | - prefetchable memory space (MEM), if not present MEM will be mapped into MEMIO |
---|
196 | |
---|
197 | - multiple PCI buses - PCI-to-PCI bridges |
---|
198 | |
---|
199 | - standard BARs, PCI-to-PCI bridge BARs, ROM BARs |
---|
200 | |
---|
201 | - Interrupt routing over bridges |
---|
202 | |
---|
203 | - Interrupt pin to system interrupt mapping |
---|
204 | |
---|
205 | Not supported: |
---|
206 | |
---|
207 | - hot-pluggable devices |
---|
208 | |
---|
209 | - Cardbus bridges |
---|
210 | |
---|
211 | - 64-bit memory space |
---|
212 | |
---|
213 | - 16-bit and 32-bit I/O address ranges at the same time |
---|
214 | |
---|
215 | In PCI 2.3 there may exist I/O BARs that must be located at the low 64kBytes |
---|
216 | address range, in order to support this the host driver or BSP must make sure |
---|
217 | that I/O addresses region is within this region. |
---|
218 | |
---|
219 | Read Configuration |
---|
220 | ~~~~~~~~~~~~~~~~~~ |
---|
221 | |
---|
222 | When a BIOS or boot loader already has setup the PCI bus the configuration can |
---|
223 | be read directly from the PCI resource registers and buses are already |
---|
224 | enumerated, this is a much simpler approach than configuring PCI ourselves. The |
---|
225 | PCI device tree is automatically created based on the current configuration and |
---|
226 | devices present. After initialization is done there is no difference between |
---|
227 | the auto or read configuration approaches. |
---|
228 | |
---|
229 | Static Configuration |
---|
230 | ~~~~~~~~~~~~~~~~~~~~ |
---|
231 | |
---|
232 | To support custom configurations and small-footprint PCI systems, the user may |
---|
233 | provide the PCI device tree which contains the current configuration. The PCI |
---|
234 | buses are enumerated and all resources are written to PCI devices during |
---|
235 | initialization. When this approach is selected PCI boards must be located at |
---|
236 | the same slots every time and devices can not be removed or added, Plug & Play |
---|
237 | is not performed. Boards of the same type may of course be exchanged. |
---|
238 | |
---|
239 | The user can create a configuration by calling pci_cfg_print() on a running |
---|
240 | system that has had PCI setup by the auto or read configuration routines, it |
---|
241 | can be called from the PCI shell command. The user must provide the PCI device |
---|
242 | tree named pci_hb. |
---|
243 | |
---|
244 | Peripheral Configuration |
---|
245 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
246 | |
---|
247 | On systems where a peripheral PCI device needs to access other PCI devices than |
---|
248 | the host the peripheral configuration approach may be handy. Most PCI devices |
---|
249 | answers on the PCI host's requests and start DMA accesses into the Hosts |
---|
250 | memory, however in some complex systems PCI devices may want to access other |
---|
251 | devices on the same bus or at another PCI bus. |
---|
252 | |
---|
253 | A PCI peripheral is not allowed to do PCI configuration cycles, which means |
---|
254 | that it must either rely on the host to give it the addresses it needs, or that |
---|
255 | the addresses are predefined. |
---|
256 | |
---|
257 | This configuration approach is very similar to the static option, however the |
---|
258 | configuration is never written to PCI bus, instead it is only used for drivers |
---|
259 | to find PCI devices and resources using the same PCI API as for the host |
---|
260 | |
---|
261 | PCI Access |
---|
262 | ---------- |
---|
263 | |
---|
264 | The PCI access routines are low-level routines provided for drivers, |
---|
265 | configuration software, etc. in order to access different regions in a way not |
---|
266 | dependent upon the host driver, BSP or platform. |
---|
267 | |
---|
268 | - PCI configuration space |
---|
269 | |
---|
270 | - PCI I/O space |
---|
271 | |
---|
272 | - Registers over PCI memory space |
---|
273 | |
---|
274 | - Translate PCI address into CPU accessible address and vice versa |
---|
275 | |
---|
276 | By using the access routines drivers can be made portable over different |
---|
277 | architectures. The access routines take the architecture endianness into |
---|
278 | consideration and let the host driver or BSP implement I/O space and |
---|
279 | configuration space access. |
---|
280 | |
---|
281 | Some non-standard hardware may also define the PCI bus big-endian, for example |
---|
282 | the LEON2 AT697 PCI host bridge and some LEON3 systems may be configured that |
---|
283 | way. It is up to the BSP to set the appropriate PCI endianness on compile time |
---|
284 | (``BSP_PCI_BIG_ENDIAN``) in order for inline macros to be correctly defined. |
---|
285 | Another possibility is to use the function pointers defined by the access layer |
---|
286 | to implement drivers that support "run-time endianness detection". |
---|
287 | |
---|
288 | Configuration space |
---|
289 | ~~~~~~~~~~~~~~~~~~~ |
---|
290 | |
---|
291 | Configuration space is accessed using the routines listed below. The pci_dev_t |
---|
292 | type is used to specify a specific PCI bus, device and function. It is up to |
---|
293 | the host driver or BSP to create a valid access to the requested PCI |
---|
294 | slot. Requests made to slots that are not supported by hardware should result |
---|
295 | in ``PCISTS_MSTABRT`` and/or data must be ignored (writes) or ``0xFFFFFFFF`` is |
---|
296 | always returned (reads). |
---|
297 | |
---|
298 | .. code-block:: c |
---|
299 | |
---|
300 | /* Configuration Space Access Read Routines */ |
---|
301 | extern int pci_cfg_r8(pci_dev_t dev, int ofs, uint8_t *data); |
---|
302 | extern int pci_cfg_r16(pci_dev_t dev, int ofs, uint16_t *data); |
---|
303 | extern int pci_cfg_r32(pci_dev_t dev, int ofs, uint32_t *data); |
---|
304 | |
---|
305 | /* Configuration Space Access Write Routines */ |
---|
306 | extern int pci_cfg_w8(pci_dev_t dev, int ofs, uint8_t data); |
---|
307 | extern int pci_cfg_w16(pci_dev_t dev, int ofs, uint16_t data); |
---|
308 | extern int pci_cfg_w32(pci_dev_t dev, int ofs, uint32_t data); |
---|
309 | |
---|
310 | I/O space |
---|
311 | ~~~~~~~~~ |
---|
312 | |
---|
313 | The BSP or driver provide special routines in order to access I/O space. Some |
---|
314 | architectures have a special instruction accessing I/O space, others have it |
---|
315 | mapped into a "PCI I/O window" in the standard address space accessed by the |
---|
316 | CPU. The window size may vary and must be taken into consideration by the host |
---|
317 | driver. The below routines must be used to access I/O space. The address given |
---|
318 | to the functions is not the PCI I/O addresses, the caller must have translated |
---|
319 | PCI I/O addresses (available in the PCI BARs) into a BSP or host driver custom |
---|
320 | address, see `Access functions`_ for how addresses are translated. |
---|
321 | |
---|
322 | .. code-block:: c |
---|
323 | |
---|
324 | /* Read a register over PCI I/O Space */ |
---|
325 | extern uint8_t pci_io_r8(uint32_t adr); |
---|
326 | extern uint16_t pci_io_r16(uint32_t adr); |
---|
327 | extern uint32_t pci_io_r32(uint32_t adr); |
---|
328 | |
---|
329 | /* Write a register over PCI I/O Space */ |
---|
330 | extern void pci_io_w8(uint32_t adr, uint8_t data); |
---|
331 | extern void pci_io_w16(uint32_t adr, uint16_t data); |
---|
332 | extern void pci_io_w32(uint32_t adr, uint32_t data); |
---|
333 | |
---|
334 | Registers over Memory space |
---|
335 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
336 | |
---|
337 | PCI host bridge hardware normally swap data accesses into the endianness of the |
---|
338 | host architecture in order to lower the load of the CPU, peripherals can do DMA |
---|
339 | without swapping. However, the host controller can not separate a standard |
---|
340 | memory access from a memory access to a register, registers may be mapped into |
---|
341 | memory space. This leads to register content being swapped, which must be |
---|
342 | swapped back. The below routines makes it possible to access registers over PCI |
---|
343 | memory space in a portable way on different architectures, the BSP or |
---|
344 | architecture must provide necessary functions in order to implement this. |
---|
345 | |
---|
346 | .. code-block:: c |
---|
347 | |
---|
348 | static inline uint16_t pci_ld_le16(volatile uint16_t *addr); |
---|
349 | static inline void pci_st_le16(volatile uint16_t *addr, uint16_t val); |
---|
350 | static inline uint32_t pci_ld_le32(volatile uint32_t *addr); |
---|
351 | static inline void pci_st_le32(volatile uint32_t *addr, uint32_t val); |
---|
352 | static inline uint16_t pci_ld_be16(volatile uint16_t *addr); |
---|
353 | static inline void pci_st_be16(volatile uint16_t *addr, uint16_t val); |
---|
354 | static inline uint32_t pci_ld_be32(volatile uint32_t *addr); |
---|
355 | static inline void pci_st_be32(volatile uint32_t *addr, uint32_t val); |
---|
356 | |
---|
357 | In order to support non-standard big-endian PCI bus the above ``pci_*`` |
---|
358 | functions is required, ``pci_ld_le16 != ld_le16`` on big endian PCI buses. |
---|
359 | |
---|
360 | Access functions |
---|
361 | ~~~~~~~~~~~~~~~~ |
---|
362 | |
---|
363 | The PCI Access Library can provide device drivers with function pointers |
---|
364 | executing the above Configuration, I/O and Memory space accesses. The functions |
---|
365 | have the same arguments and return values as the above functions. |
---|
366 | |
---|
367 | The pci_access_func() function defined below can be used to get a function |
---|
368 | pointer of a specific access type. |
---|
369 | |
---|
370 | .. code-block:: c |
---|
371 | |
---|
372 | /* Get Read/Write function for accessing a register over PCI Memory Space |
---|
373 | * (non-inline functions). |
---|
374 | * |
---|
375 | * Arguments |
---|
376 | * wr 0(Read), 1(Write) |
---|
377 | * size 1(Byte), 2(Word), 4(Double Word) |
---|
378 | * func Where function pointer will be stored |
---|
379 | * endian PCI_LITTLE_ENDIAN or PCI_BIG_ENDIAN |
---|
380 | * type 1(I/O), 3(REG over MEM), 4(CFG) |
---|
381 | * |
---|
382 | * Return |
---|
383 | * 0 Found function |
---|
384 | * others No such function defined by host driver or BSP |
---|
385 | */ |
---|
386 | int pci_access_func(int wr, int size, void **func, int endian, int type); |
---|
387 | |
---|
388 | PCI device drivers may be written to support run-time detection of endianess, |
---|
389 | this is mosly for debugging or for development systems. When the product is |
---|
390 | finally deployed macros switch to using the inline functions instead which have |
---|
391 | been configured for the correct endianness. |
---|
392 | |
---|
393 | .. index:: PCI address translation |
---|
394 | |
---|
395 | PCI address translation |
---|
396 | ~~~~~~~~~~~~~~~~~~~~~~~ |
---|
397 | |
---|
398 | When PCI addresses, both I/O and memory space, is not mapped 1:1 address |
---|
399 | translation before access is needed. If drivers read the PCI resources directly |
---|
400 | using configuration space routines or in the device tree, the addresses given |
---|
401 | are PCI addresses. The below functions can be used to translate PCI addresses |
---|
402 | into CPU accessible addresses or vice versa, translation may be different for |
---|
403 | different PCI spaces/regions. |
---|
404 | |
---|
405 | .. code-block:: c |
---|
406 | |
---|
407 | /* Translate PCI address into CPU accessible address */ |
---|
408 | static inline int pci_pci2cpu(uint32_t *address, int type); |
---|
409 | |
---|
410 | /* Translate CPU accessible address into PCI address (for DMA) */ |
---|
411 | static inline int pci_cpu2pci(uint32_t *address, int type); |
---|
412 | |
---|
413 | .. index:: PCI Interrupt |
---|
414 | |
---|
415 | PCI Interrupt |
---|
416 | ------------- |
---|
417 | |
---|
418 | The PCI specification defines four different interrupt lines INTA#..INTD#, the |
---|
419 | interrupts are low level sensitive which make it possible to support multiple |
---|
420 | interrupt sources on the same interrupt line. Since the lines are level |
---|
421 | sensitive the interrupt sources must be acknowledged before clearing the |
---|
422 | interrupt contoller, or the interrupt controller must be masked. The BSP must |
---|
423 | provide a routine for clearing/acknowledging the interrupt controller, it is up |
---|
424 | to the interrupt service routine to acknowledge the interrupt source. |
---|
425 | |
---|
426 | The PCI Library relies on the BSP for implementing shared interrupt handling |
---|
427 | through the BSP_PCI_shared_interrupt_* functions/macros, they must be defined |
---|
428 | when including bsp.h. |
---|
429 | |
---|
430 | PCI device drivers may use the pci_interrupt_* routines in order to call the |
---|
431 | BSP specific functions in a platform independent way. The PCI interrupt |
---|
432 | interface has been made similar to the RTEMS IRQ extension so that a BSP can |
---|
433 | use the standard RTEMS interrupt functions directly. |
---|
434 | |
---|
435 | PCI Shell command |
---|
436 | ----------------- |
---|
437 | |
---|
438 | The RTEMS shell has a PCI command 'pci' which makes it possible to read/write |
---|
439 | configuration space, print the current PCI configuration and print out a |
---|
440 | configuration C-file for the static or peripheral library. |
---|