Linux NVMe Multipathing and ANA Explained

Linux NVMe multipathing lets a host access one NVMe namespace through multiple controller paths. Storage and Linux administrators use it to avoid relying on a single route and, where the subsystem supports it, distribute I/O across eligible paths. This guide explains how the kernel groups paths, how Asymmetric Namespace Access (ANA) informs path selection, and what to verify before relying on multipathing for availability or performance.

What Is Linux NVMe Multipathing?

NVMe multipathing is a host feature that recognizes multiple routes to the same namespace and presents them as one logical block device. Applications continue issuing reads and writes to that device while the Linux NVMe host driver chooses an underlying controller path for each I/O. The Linux kernel NVMe multipath documentation describes the supported path-selection policies and their relationship to ANA.

This is distinct from connecting several unrelated disks: the paths must lead to the same namespace identity within an NVMe subsystem. One path might use a different controller, network interface, switch, or fabric from another. Whether those paths are truly redundant depends on the physical and logical topology, not just their count.

NVMe is standardized by NVM Express. The NVM Express specifications define the protocol and transport specifications that allow a host to communicate with NVMe subsystems over interfaces such as PCIe, RDMA, and TCP. Multipathing is the host-side behavior that makes more than one route to a namespace usable as a coordinated device.

The Problem Multipathing Solves

A host with only one route to a storage namespace depends on every component along that route. A failed controller port, cable, network interface, switch, or transport connection can make the namespace temporarily unreachable even if the storage media and other parts of the subsystem remain healthy.

Multiple paths give the host alternatives. If the subsystem reports a route as unavailable, the kernel may direct new I/O through another eligible route. Depending on the subsystem and policy, it can also spread I/O across paths to use available bandwidth. The goal is to improve path resilience and utilization without requiring applications to manage individual connections.

Multipathing has boundaries. It cannot protect against loss of the entire storage subsystem, namespace deletion, corruption, or a failure shared by every route. Redundant paths that traverse the same switch or controller still share that failure domain. Failover may also involve transport recovery and queued I/O, so it is not necessarily instantaneous or invisible to the application. Snapshots, replication, backups, and application-level recovery solve different problems.

How NVMe Multipathing and ANA Work

The host discovers controllers and namespaces, then groups paths that refer to the same namespace. Instead of asking an application to select a controller connection, the kernel exposes a multipath block device. Each I/O submitted to that device is routed through one of the namespace’s paths according to the configured policy and the access state reported by the subsystem.

ANA is the NVMe mechanism that communicates a namespace’s accessibility through a controller. It allows a subsystem to distinguish paths that are preferred from those that remain usable but are less desirable, and from paths that cannot currently serve I/O. The kernel considers ANA state when making path choices; a policy does not make an inaccessible path usable.

The current Linux documentation describes three policies, all of which follow ANA: NUMA (the default documented policy) favors a path close to the CPU’s NUMA node; round-robin cycles among eligible paths; and queue-depth favors the path with the fewest I/Os in flight. The policy determines selection among usable options, not which ANA state the subsystem reports.

Linux native NVMe multipathing is not the same mechanism as SCSI device-mapper multipath. For iSCSI LUNs, Linux commonly uses SCSI devices managed by DM-Multipath and multipathd. Native NVMe multipathing is implemented in the NVMe host stack and uses NVMe namespace identity and ANA information. These mechanisms should be diagnosed and configured using the tools for their respective protocols.

Components and Key Concepts

• Namespace: The logical block storage unit presented by an NVMe subsystem. Paths are grouped only when they identify the same namespace.
• Controller path: A host’s connection to a controller that can provide access to a namespace. Different controllers can provide paths with different ANA states.
• ANA: The subsystem-reported accessibility state for a namespace through a controller. It helps the host distinguish preferred, alternate, transitioning, and unavailable routes.
• Multipath block device: The device applications and filesystems use while the kernel selects an underlying path.
• I/O policy: The Linux path-selection strategy, such as NUMA, round-robin, or queue-depth. It operates within the constraints of path availability and ANA.
• Transport and topology: PCIe, RDMA, TCP, and other supported connections determine how a path reaches the subsystem. Independent paths require independent components where failures are meant to be tolerated.

For remote storage, NVMe over Fabrics (NVMe-oF) explains how NVMe commands travel over network transports. Multipathing addresses the host’s choice among multiple routes to a namespace; it does not replace the transport, discovery, or target configuration.

Real-World Use Cases

NVMe-oF storage networks can expose the same namespace through multiple target ports or network routes. A host can continue I/O through a remaining accessible path when one route is lost, provided that the subsystem, host, and network are configured for that behavior.

Enterprise storage arrays may expose optimized and non-optimized paths because different controllers own or serve different namespaces. ANA lets the host prefer the route the array identifies as optimized while retaining alternate routes where the array supports them.

High-throughput systems may benefit from distributing eligible I/O across paths. The benefit depends on the target’s capabilities, network capacity, path policy, workload, and controller resources; additional paths do not guarantee a proportional increase in throughput.

Planned maintenance and fault testing can use alternate routes to maintain access when a single path component is taken offline. Tests should be designed around the real failure domains and run under an approved maintenance procedure, with the storage and application owners aware of the expected impact.

Getting Started: Inspect and Configure on Linux

Install the NVMe command-line tools using the package manager for the host. For Debian or Ubuntu, the package is commonly named nvme-cli; Fedora-family distributions also provide an nvme-cli package. The upstream nvme-cli project provides the command-line utilities for managing NVMe devices.

Start by listing devices and subsystem paths. These commands inspect the host; they do not change the storage configuration:

sudo nvme list
sudo nvme list-subsys
sudo nvme list-subsys -o json

Review the output for the expected subsystem, namespaces, controllers, and transport addresses. Confirm with the storage administrator that each reported path is intended to reach the same namespace and that supposedly redundant routes do not share components that should be protected. The kernel and distribution must also include native NVMe multipath support; availability and defaults can differ by kernel build and distribution.

Check the active I/O policy with:

cat /sys/module/nvme_core/parameters/iopolicy

Only change a policy when there is a measured reason and the storage design supports the alternative. The kernel documentation shows how to change it at runtime:

printf '%s\n' round-robin | sudo tee /sys/module/nvme_core/parameters/iopolicy
cat /sys/module/nvme_core/parameters/iopolicy

This changes the active kernel parameter and may not persist after reboot. To request a persistent policy, the kernel documentation describes the nvme_core.iopolicy=round-robin kernel command-line parameter; configure it using the bootloader procedure for the distribution and validate it after a maintenance reboot. Do not copy a bootloader command from a different distribution without checking its documentation.

Before relying on multipathing, verify the expected controller paths with nvme list-subsys, review kernel messages for controller or ANA transitions, and test path loss only in a lab or approved maintenance window. Test one known path at a time while monitoring representative I/O and application behavior. Avoid blindly disconnecting controllers or disabling interfaces: a shared dependency can remove more paths than intended.

For diagnosis, compare the current subsystem listing with recent kernel messages:

sudo nvme list-subsys
journalctl -k -b | grep -i nvme

A missing path can indicate a discovery, transport, target-port, or subsystem configuration problem rather than an I/O policy issue. If paths are present but throughput does not increase, measure with a workload representative of the application and inspect network, controller, and queue limits. The guide to block storage performance and benchmarking covers IOPS, throughput, latency, and safe test design. For a different protocol’s host path management, see iSCSI multipathing; the concepts overlap, but the Linux mechanisms are not interchangeable.

Common Misconceptions

• “More paths automatically make storage faster.” Multiple routes provide opportunities for resilience or distribution. Actual throughput depends on the subsystem, topology, workload, and policy; a bottleneck shared by all paths remains a bottleneck.
• “Every reachable path is equally good.” ANA distinguishes optimized and non-optimized access, and a non-optimized path is not necessarily failed. The host’s selection policy follows the access state rather than treating all routes identically.
• “Multipathing is a backup.” It can maintain access through another eligible route after some path failures. It does not preserve a separate copy of data or protect against failures that affect the entire subsystem.
• “NVMe uses the same multipath tools as iSCSI.” Native NVMe paths are managed by the NVMe host stack and ANA. SCSI multipath tools such as DM-Multipath address a different protocol path.

Related Articles

• NVMe over Fabrics networking explains NVMe transports for remote storage.
• iSCSI multipathing explained covers SCSI path grouping, failover, and DM-Multipath.
• Block storage performance and benchmarking explains storage measurements and workload trade-offs.
• NVMe Zoned Namespace storage introduces a specialized NVMe namespace model.
• Linux storage management basics covers Linux block devices and storage administration.

Changelog

• Initial publication.

Last updated: 2026-10-26