Components of PCIe Architecture and Their Roles in Processor Systems
This article explains the main components of PCIe architecture—including Root Complex (RC), switches, and PCIe‑to‑PCI bridges—how they are implemented in different processor systems such as x86 and PowerPC, and the mechanisms for QoS, port arbitration, and extended configuration space.
PCIe serves as a local bus in processor systems, similar to the legacy PCI bus, connecting external devices and sometimes other processors. Most implementations use a Root Complex (RC), switches, and PCIe‑to‑PCI bridges to link PCIe and PCI devices, with endpoint devices (EP) residing on the PCIe bus.
Processor System A (x86‑based) : In many x86 systems the RC consists of memory controllers and two FSB‑to‑PCIe bridges. A virtual PCI bridge separates the memory domain from the PCI domain, and the PCI bus 0 hosts a memory controller and two PCIe links (×16 for graphics, ×8 for a switch). Switches expand PCIe links, creating new PCI bus numbers, while the RC also contains registers (RCRB) and an Event Collector for error handling.
PowerPC Processor : PowerPC designs differ; PCIe is less central and many internal devices connect directly to the SoC bus. The upcoming Freescale P4080 (E500‑mc core) uses a CoreNet interconnect with three PCIe controllers, inbound/outbound address translation registers, and a PAMU to separate peripheral and memory address spaces.
Generic PCIe‑Based Processor Structure : RC implementations vary; some treat RC as a multi‑port PCIe bridge, while others separate the PCIe controller from other functions. A typical generic system integrates RC, multiple PCIe ports, switches, and bridges, allowing endpoint devices, switches, and PCIe‑to‑PCI bridges to connect.
RC Composition : In x86 systems RC includes memory controllers, PCIe ports, RCRB registers, and an Event Collector. PowerPC systems lack a true RC, using only PCIe controllers. RC performs address translation between memory and PCI domains and, with virtualization, handles complex functions such as MR‑IOV.
Switch Architecture : Switches have one upstream port and multiple downstream ports, each downstream port representing a virtual PCI bridge. Switches manage QoS using virtual channels (VC) and traffic classes (TC), supporting up to eight VCs per link and arbitration algorithms like Strict Priority, Round Robin, and Weighted Round Robin.
VC and Port Arbitration : When multiple ingress ports target the same egress port, a port arbiter decides the order, followed by VC arbitration. Arbitration can be hardware‑fixed, WRR, or time‑based WRR, and is defined in the device’s Capability registers.
PCIe‑to‑PCI/PCI‑X Bridges : These bridges convert PCIe to PCI (or vice‑versa) to connect legacy PCI devices. Multi‑port bridges provide one upstream PCIe port and several downstream PCI ports, enabling integration of older designs into modern PCIe systems.
PCIe Device Extended Configuration Space : PCIe devices have a 64‑byte base configuration space (0x00‑0x3F) and extended spaces up to 4 KB (0x100‑0xFFF) for capabilities such as MSI/MSI‑X, power management, and PCIe‑specific features. Access methods differ between x86 (CONFIG_ADDRESS/CONFIG_DATA or ECAM) and PowerPC (CFG_ADDR/CFG_DATA).
Architects' Tech Alliance
Sharing project experiences, insights into cutting-edge architectures, focusing on cloud computing, microservices, big data, hyper-convergence, storage, data protection, artificial intelligence, industry practices and solutions.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.