Linux blk-mq Explained: Multi-Queue Block I/O

Modern storage devices can process many I/O requests in parallel, but a kernel that funnels every request through one shared queue can become a bottleneck before the device is busy. Linux blk-mq, or the multi-queue block I/O layer, organizes block requests so multicore systems can submit work through multiple software queues and dispatch it through driver-facing hardware contexts. This explainer is for Linux administrators, storage engineers, and developers who want to understand the request path rather than treat queue depth as a tuning shortcut.

What Is Linux blk-mq?

Linux blk-mq is the kernel’s multi-queue block layer: the part of the I/O stack that manages requests between filesystems or raw block-device users and storage drivers. Its queue model supports parallel request handling on multicore systems and devices that can accept multiple outstanding commands. The Linux kernel documentation for blk-mq describes its software staging queues, hardware dispatch queues, request merging, scheduling, and completion tags.

The name can be confusing because “hardware queue” is a Linux block-layer term for a driver-facing hardware context. It may map to a device submission queue, a virtual queue, or another driver resource; it is not always a separate physical queue built into a disk. The Linux kernel project publishes kernel source and releases, while the exact facilities available on a host depend on its running kernel, driver, and distribution configuration.

Why the Multi-Queue Model Exists

Older block-layer designs relied more heavily on a single shared request queue and lock. As processors and storage devices became more parallel, CPUs could contend for that shared state even when the device itself was capable of accepting concurrent operations. This is especially wasteful for flash and non-volatile memory, where mechanical disk-head movement is not the central constraint.

blk-mq reduces that software serialization by giving CPUs or NUMA nodes local software contexts and mapping them to driver-facing hardware contexts. Multiple CPUs can prepare and submit I/O without every request passing through one global software queue. The kernel can still merge requests or apply a scheduler where that makes sense; multi-queue does not mean “skip all software processing.”

The distinction matters in a real system. A low-latency NVMe device may expose substantial parallelism, but a workload with only one outstanding request cannot use much of it. Conversely, a high-concurrency workload can still be limited by CPU work, a driver, a virtual machine, a storage network, or a remote array. blk-mq addresses the Linux block layer’s request-management path, not every bottleneck in the end-to-end data path.

How blk-mq Works

A filesystem or application submits block I/O, commonly represented initially as a bio. The block layer turns one or more bios into a request that can be presented to a device driver. Depending on the situation, the request may go directly to a hardware context or first pass through a software staging queue, where the kernel can batch, merge, or schedule it.

Software contexts (blk_mq_ctx) are associated with CPUs or nodes. Hardware contexts (blk_mq_hw_ctx) are associated with queues the driver can service. A mapping connects the software contexts to one or more hardware contexts. The actual mapping depends on the device driver, available queue resources, and system topology; it is not necessarily a simple one-CPU-to-one-device-queue arrangement.

When a request is ready, blk-mq dispatches it to the driver. If the driver or device cannot accept more work, the block layer can retain requests for later dispatch instead of losing them. The driver uses tags to track outstanding requests, and it reports completions back to the block layer using those identifiers. Completion order is not a general guarantee that higher layers can rely on; filesystems and applications maintain the ordering semantics they need.

The queue at the application interface is not necessarily the same queue as the block layer’s dispatch context. For example, NVM Express specifications define the NVMe protocol and its controller interfaces. Linux’s NVMe driver connects its block-layer work to NVMe controller queues, but NVMe submission and completion queues belong to the protocol/device layer, while blk-mq contexts belong to Linux’s block layer.

Components and Key Concepts

Request merging and plugging can combine adjacent or compatible operations before dispatch. Fewer, larger requests may reduce per-request overhead, even when a flash device has no disk-head seek cost. Requests are not always mergeable: workload access patterns, timing, boundaries, and queue mappings matter.

I/O schedulers can reorder or prioritize requests that pass through a scheduler. The available choices depend on the kernel, driver, and device. Depending on the distribution and workload, a host might expose options such as mq-deadline, kyber, bfq, or none; the last means no block I/O elevator is selected, not that the whole I/O stack has no queues. A scheduler trades among concerns such as latency, fairness, and throughput, and should not be selected by name alone.

Queue depth and tags describe capacity for outstanding operations at a particular layer. Increasing concurrency can help keep a capable device busy, but can also create longer waits and higher tail latency. Queue limits in an application, filesystem, block layer, driver, transport, hypervisor, or storage service are separate constraints; one displayed value does not describe the whole path.

NUMA and CPU mapping affect where request processing happens. Keeping work near the CPU or device resources that serve it can reduce cross-node traffic, but the useful mapping depends on a host’s topology and driver. A system with many software queues is not automatically more efficient if the workload is small or the storage path is elsewhere.

Real-World Uses

• NVMe storage: Multiple CPU cores can submit requests through driver-facing contexts that make use of supported device parallelism. The benefit depends on queue availability, request concurrency, and the rest of the system.
• Virtual machines and cloud volumes: A guest’s block requests pass through virtual storage and host layers. Guest blk-mq settings cannot by themselves remove host scheduling, network, or provider limits.
• Multicore database and storage servers: Concurrent workloads may avoid contention around a single software submission path, while schedulers and merge behavior still influence latency and fairness.
• SCSI and virtualized devices: blk-mq also supports modern drivers beyond local NVMe. The driver determines how its hardware contexts correspond to transport or virtual-device queues.

The common thread is parallel request handling, not a guarantee of higher benchmark numbers. An idle device, a serialized application, a saturated CPU, or a downstream bottleneck can keep performance unchanged after queueing has been optimized.

Getting Started: Inspecting blk-mq

Begin with read-only checks. uname identifies the running kernel, lsblk shows the devices visible to the host, and the scheduler file shows the scheduler options and the active choice in brackets. Replace nvme0n1 with a block-device name shown by lsblk; the relevant sysfs files vary by device and kernel.

uname -r
lsblk -o NAME,TYPE,TRAN,SIZE,MODEL
cat /sys/block/nvme0n1/queue/scheduler

For example, output like [none] mq-deadline means none is selected, while mq-deadline is another option shown by that device. The result is device-specific, and distributions may backport kernel features or use different defaults. Check your distribution’s kernel documentation as well as the kernel project’s release information before comparing two hosts.

To see whether a storage device is busy and how its I/O behaves over time, install sysstat if iostat is not already available. On Debian or Ubuntu, use sudo apt install sysstat; on Fedora, use sudo dnf install sysstat. Then sample extended device statistics:

iostat -xz 1

Interpret utilization, request size, queue length, and wait-time fields in the context of the device type. A virtual or cloud device may not report the same physical-device busy time as a directly attached drive. Compare measurements with application latency, CPU use, and network or storage-service metrics. For a careful fio workload and benchmark safety guidance, see block storage performance and benchmarking.

Do not change scheduler or queue settings just because a value looks small or a benchmark result is lower than a vendor maximum. First establish a reproducible workload and a baseline, then change one variable at a time in a test environment and measure both throughput and latency. Avoid writing benchmarks to a raw production device; a write test can destroy data. Keep kernel and driver versions, device type, filesystem, workload, and queue settings with each measurement so the comparison is meaningful.

Common Misconceptions

• “blk-mq makes every storage device parallel.” It enables scalable request handling in the Linux block layer, but a device or transport may offer limited concurrency, and the application may submit only one operation at a time.
• “A hardware context is always a physical hardware queue.” It is a block-layer abstraction used by a driver. Its relationship to the device’s native queues depends on that driver and interface.
• “More queues or a higher queue depth always improve performance.” Additional concurrency can use available capacity, but it can also add contention and waiting. Measure the actual workload and tail latency.
• “The scheduler alone determines I/O order.” Filesystems, applications, device protocols, drivers, and remote storage can also queue or reorder work. Completion ordering is not generally guaranteed by blk-mq.
• “Selecting none removes queues.” It disables a block I/O scheduler for that device; software contexts, driver dispatch, and device queues still exist.

Related Articles

• Block storage performance and benchmarking explains IOPS, throughput, latency, queue depth, and safe measurement.
• Linux NVMe multipathing and ANA covers path selection for NVMe namespaces, a separate layer from blk-mq request dispatch.
• Linux io_uring asynchronous I/O architecture explains the application-facing submission and completion interface.
• Linux storage management basics introduces block devices, filesystems, LVM, RAID, and inspection tools.

Changelog

• Initial publication.

Last updated: 2026-11-13