GPU Interconnect Architecture: PCIe, NVLink, and NVSwitch

GPU interconnect architecture determines how accelerators exchange data with CPUs, memory, storage, and each other. For AI infrastructure engineers and machine-learning developers, the links between GPUs can matter as much as their compute capacity: model-parallel workloads move activations and gradients frequently, while data-parallel training synchronizes updates across devices. This explainer distinguishes PCI Express (PCIe), NVIDIA NVLink, and NVSwitch, and shows how to inspect and benchmark the paths a system actually provides.

What Is GPU Interconnect Architecture?

An interconnect is the hardware and signaling path that carries data between components. In a GPU server, the path may run from a GPU through a PCIe switch or CPU root complex, across a direct NVLink connection to another GPU, or through an NVSwitch fabric. A multi-node job adds a network interface and an external fabric such as Ethernet or InfiniBand.

These paths form different bandwidth and latency domains. A device can be fast in isolation but spend time waiting for another GPU if their communication path is constrained. NVIDIA’s NVLink overview describes its GPU links and NVLink Switch systems. The Linux kernel’s PCI driver guide provides background on how PCI devices are discovered and managed.

Why GPU Interconnects Matter

An accelerator usually has far more local memory bandwidth than the host can provide through a general-purpose device link. If every step of a workload moves substantial data between GPUs or between host and device, communication can limit throughput before the compute units are saturated.

The effect depends on the algorithm. Data-parallel training commonly exchanges gradients or model updates; tensor-parallel inference exchanges intermediate activations within a model. A large batch of independent jobs may need little GPU-to-GPU traffic, so paying for a more capable fabric would not necessarily improve results. The right question is not simply how fast a link is, but how much data the workload sends, how often it sends it, and whether transfers overlap with computation.

Topology adds another variable. Two GPUs in the same server may reach each other over different numbers of PCIe switches, through separate CPU sockets, over direct NVLinks, or through a switched NVLink fabric. A server’s product name or GPU count does not by itself identify the path available to a specific pair of devices.

How PCIe, NVLink, and NVSwitch Work

PCIe is a general-purpose, packet-based expansion interconnect used by GPUs, network adapters, storage devices, and other peripherals. A GPU normally connects to a CPU’s PCIe root complex, potentially through one or more PCIe switches. Link generation, lane width, switch arrangement, and CPU attachment all affect available bandwidth and sharing. Two GPUs that both report an x16 link to the host can still have different peer-to-peer routes.

NVLink is NVIDIA’s high-speed GPU interconnect. Depending on the supported GPU and platform, NVLink links can provide direct GPU-to-GPU paths or connect GPUs into a switched system. Link counts, bandwidth, and supported features vary by product generation, so figures for one system should not be generalized to all devices. An NVLink connection also does not mean that every memory access is automatically faster: the software, workload, and route must use the connection.

NVSwitch is a switch fabric for NVLink-connected GPUs. Rather than relying only on a limited set of direct GPU pairs, a platform can use NVSwitch devices to connect multiple GPUs through a fabric. Its purpose is to provide more flexible GPU-to-GPU communication within a scale-up system. The resulting paths, aggregate capacity, and contention depend on the particular platform and fabric design; the name alone is not a guarantee of uniform per-GPU bandwidth.

These scale-up links are distinct from scale-out networking between servers. A multi-node training job sends data through network adapters and an external network, often using RDMA-capable transports. The IETF’s RFC 5040 specifies one RDMA protocol model; it is a network protocol specification, not a definition of PCIe or NVLink. NVIDIA’s NCCL overview describes a communication library that selects and uses available transports for collective operations such as all-reduce.

Bandwidth figures need context. A vendor may report a per-link, per-direction, or aggregate figure; adding link rates can describe theoretical capacity without showing application throughput. Effective transfer rates also depend on message size, synchronization, concurrency, protocol overhead, memory placement, and whether transfers overlap useful computation.

Components and Key Concepts

• GPU and host attachment: A GPU’s PCIe connection leads to a CPU root complex, sometimes through a PCIe switch. CPU socket affinity and the root complex can affect the route to system memory, a NIC, storage, or another GPU.
• Switches and hops: PCIe switches fan out host connectivity; NVSwitch devices connect NVLink peers in supported systems. More hops do not always mean a fixed penalty, but shared links and oversubscribed uplinks can become bottlenecks under concurrent traffic.
• Peer-to-peer access: A GPU may be able to read or write another GPU’s memory without staging the payload through host memory, subject to GPU, driver, platform, and topology support. Visibility of multiple GPUs alone does not prove that peer access is available or selected.
• Topology labels: NVIDIA’s nvidia-smi topo -m output reports relative paths. Labels such as PIX, PXB, PHB, NODE, and SYS distinguish PCIe-switch and host/NUMA relationships; NV# indicates an NVLink connection count. Interpret labels alongside the command’s legend and platform documentation.
• Collectives: Operations such as all-reduce, all-gather, and reduce-scatter exchange data among a group of GPUs. Libraries such as NCCL choose algorithms and transports based on the topology and operation; the fastest route for one collective or message size may not be best for another.
• Local and remote memory paths: Host-to-GPU transfers, GPU peer copies, and storage-to-GPU transfers are related but different paths. For example, GPUDirect Storage concerns supported storage I/O into GPU memory, not a general GPU-to-GPU fabric.

Real-World Use Cases

Data-parallel training can benefit from fast GPU collectives when gradient synchronization takes a meaningful share of each training step. If compute time dominates or the workload is small, a premium interconnect may have little effect. Compare step time and scaling efficiency, not only a link’s advertised rate.

Tensor-parallel model training and inference divide model operations across GPUs and exchange intermediate data during execution. These frequent transfers make communication latency and aggregate fabric capacity important. The placement of GPU ranks should match the actual topology where the framework allows it.

GPU servers that share devices across workloads need predictable communication as well as compute and memory resources. GPU partitioning changes how device capacity is assigned; it does not create a different physical interconnect. See GPU partitioning with MIG and MPS for the distinction between hardware partitioning and concurrent processes.

Multi-node clusters combine intra-server scale-up links with an inter-server scale-out network. A job can be limited by either layer: fast GPU-to-GPU traffic inside a node cannot compensate for a saturated network between nodes, and a fast external network cannot fix poor placement within a server. The practical multi-GPU machine-learning setup guide covers the broader training stack.

Getting Started: Inspect and Benchmark the Topology

Interconnect hardware is platform-specific. There is no package that adds NVLink to a GPU or turns an unsupported PCIe peer path into a supported one. First install a compatible GPU driver and compute stack using the vendor’s instructions for the exact GPU and operating system. On a Linux host with NVIDIA drivers, inspect the devices and their reported connections:

# List the GPUs visible to the NVIDIA driver.
nvidia-smi -L

# Show the relative connection path between GPUs and other devices.
nvidia-smi topo -m

# Inspect NVLink link state on supported systems.
nvidia-smi nvlink --status

Read the topology matrix before interpreting a benchmark. Check whether the GPUs share a PCIe switch or host bridge, whether their routes cross CPU sockets, and whether NVLink links are active. A link-status command can show hardware state, but not prove that an application is using the path efficiently.

For a multi-GPU baseline, build NVIDIA’s nccl-tests on a machine with a compatible CUDA Toolkit and NCCL installation. The following example runs an all-reduce test on four local GPUs; change the GPU count to one actually available on the host:

git clone --depth 1 https://github.com/NVIDIA/nccl-tests.git
make -C nccl-tests -j
cd nccl-tests
NCCL_DEBUG=INFO ./build/all_reduce_perf -b 8M -e 1G -f 2 -g 4

Compare results across message sizes and GPU counts, and read the NCCL log for the selected topology and transport. An all-reduce benchmark is a diagnostic, not a prediction of application speed: it does not reproduce every model’s communication pattern, data placement, or compute overlap. For an application, profile communication alongside compute and measure end-to-end step time. If performance is unexpectedly low, verify link state, GPU-to-GPU peer access, process placement, driver and NCCL versions, and whether the tested GPUs share a congested path. For a broader compute-stack comparison, see CUDA, ROCm, and Vulkan GPU compute.

Common Misconceptions

“Every GPU in one server can communicate over NVLink.” NVLink support and wiring depend on the GPU and system. Devices may instead communicate through PCIe, or may have no usable peer path for a particular configuration.

“NVSwitch means every transfer gets the same full bandwidth.” A switch fabric connects devices; its realized throughput still depends on the system design, traffic pattern, and simultaneous users. Measure the communication that matters to the workload.

“The fastest advertised link guarantees the fastest training.” Compute, memory capacity, host input, synchronization, software versions, and inter-node networking can all be limiting factors. Benchmark complete workloads and compare time-to-result.

Related Articles

• Multi-GPU Setup for Machine Learning covers hardware planning, software, and distributed training.
• CUDA, ROCm, and Vulkan GPU Compute compares programming and runtime stacks, which sit above the physical interconnect.
• NVIDIA GPUDirect Storage Architecture explains storage-to-GPU data movement.
• NVIDIA GPU Partitioning with MIG and MPS explains GPU sharing and hardware partitioning.

Changelog

• Initial publication; NVIDIA, Linux kernel, and IETF source pages checked.