IotTechnology

MQTT Implementation Patterns for Reliable IoT Messaging

Updated on
10 min read

MQTT is a lightweight publish/subscribe protocol for exchanging messages through a broker. It is common in IoT systems because devices and services can exchange telemetry and commands without maintaining direct connections to each other. This guide explains the protocol’s delivery limits, session and topic patterns, and the operational choices needed to make an MQTT deployment reliable. For a broader comparison with CoAP, HTTP, and other options, see the IoT network protocols guide.

What Is MQTT?

MQTT defines how clients connect to a server (usually called a broker), publish application messages to named topics, and subscribe to topic filters. The broker routes each publication to clients whose subscriptions match. A client can publish, subscribe, or do both.

The protocol is intentionally small, but a working system includes more than MQTT: client identity, a broker, network security, topic permissions, data schemas, and consumer behavior. MQTT standardizes message exchange; it does not prescribe how an application stores or acts on a reading.

The Problem MQTT Solves

Direct device-to-service connections tightly couple producers to consumers. Each device may need to know every destination, retry independently, and handle changes in backend availability. A broker provides an intermediary: devices publish once, and interested consumers subscribe independently.

This decoupling is useful for small telemetry messages and intermittent connections, but it does not make the broker or network infallible. The broker becomes an important service to secure, size, monitor, and recover. MQTT is also not a universal substitute for HTTP APIs, file transfer, or a durable event log. Compare it with other application protocols in the IoT network protocols guide before choosing a design.

How MQTT Works

A typical message path is:

Sensor or gateway --PUBLISH(topic, payload, QoS)--> MQTT broker
                                                    |
Consumer service <--PUBLISH for matching filters---+

Clients establish a connection to a broker, commonly over TCP with TLS; some deployments use WebSockets. A client can then subscribe to a topic filter such as sites/+/sensors/+/telemetry. When a publisher sends to a matching topic, the broker applies its routing and authorization rules and forwards the message to eligible subscribers.

The broker may also keep limited protocol state: subscriptions and queued QoS 1 or QoS 2 messages for an unexpired persistent session, a retained value for a topic, or a will message to publish after an unexpected disconnect. These are different mechanisms with different lifetimes; none should be mistaken for an application database or an unlimited offline queue.

MQTT delivery happens in protocol exchanges between participants. A message acknowledged by the broker has not necessarily been processed or committed by every downstream service. Applications that must not apply an event twice need stable event IDs and idempotent consumers, and commands usually need an application-level acknowledgement describing whether the action actually completed.

Components and Variants

  • Broker: Accepts client connections, authenticates clients, enforces topic permissions, matches subscriptions, and manages supported session state.
  • Client ID: Identifies a connection and, when sessions persist, associates reconnects with broker-held state. Simultaneous clients should not reuse the same ID unless replacing the existing connection is intentional.
  • Topic and topic filter: Publishers send to a concrete topic such as sites/plant-7/sensors/pump-2/telemetry. Subscribers use filters; + matches one level and # matches the remaining levels. Wildcards are for subscriptions, not publish topic names.
  • Payload and properties: A payload can be JSON, a compact binary format, or another application-defined encoding. MQTT 5 adds properties such as content type, message expiry, response topic, and correlation data.
  • Quality of Service (QoS): Selects a protocol delivery flow for each publication. The subscriber’s maximum QoS also constrains delivery.
QoS Protocol behavior Typical fit Main trade-off
0 At most once; no MQTT acknowledgement for delivery Frequent readings where an occasional loss is acceptable Lowest protocol overhead, but no MQTT retry
1 At least once Important telemetry and commands with deduplication A message can arrive more than once
2 Exactly-once delivery within the MQTT exchange between participants Rare cases where the extra handshake is justified More protocol state and round trips; still does not make a business operation exactly once

For QoS 1 and QoS 2, the broker and clients also apply their configured in-flight and queue limits. Pick the lowest QoS that meets the application’s delivery needs, then make the consumer safe against retries. The OASIS MQTT Version 5.0 specification defines the protocol behavior.

Sessions, retained messages, and shared subscriptions

In MQTT 3.1.1, the clean-session choice controls whether the broker keeps a session across connections. MQTT 5 separates the initial clean-start choice from a session-expiry interval. A nonzero expiry can preserve subscriptions and eligible queued QoS 1/2 messages after disconnect, subject to broker limits and policy. Message expiry is separate: it lets an individual publication become stale before a session expires.

A retained message is the broker’s latest retained publication for a topic, delivered to a new matching subscriber. It is useful for current state, such as a device’s last reported status, not for retaining every historical event. Publishing a retained zero-byte payload removes the retained value for that topic.

Where supported by the broker, a shared subscription such as $share/analytics/sites/+/sensors/+/telemetry lets a group of consumers share work: a matching publication is routed to a member of that group rather than copied to every member. This is a consumption pattern, not a replacement for durable downstream processing.

MQTT 3.1.1 and MQTT 5

Both versions are used in deployed systems. MQTT 5 adds richer reason codes, session and message expiry, flow-control properties, and standard request/response properties. It can improve protocol-level visibility and control, but it does not remove application responsibilities such as authorization, retry policy, schema evolution, and deduplication. Check that the clients, broker, and any managed service agree on the version and features; for example, AWS IoT Core documents its supported MQTT features and differences.

Real-World Use Cases

  • Telemetry ingestion: Sensors publish readings to a per-device topic; a rule engine, time-series writer, and alerting service subscribe to the data they need.
  • Device commands and state: A service publishes a command to a device-specific topic. Use authorization and expiry to prevent stale commands from being applied, and have the device publish an acknowledgement or reported state after it acts.
  • Gateway store-and-forward: A gateway translates a local field protocol, buffers data during a WAN outage, and republishes it after reconnection. Its local queue must have a size limit and a defined overflow policy.
  • Request and response: MQTT 5 response-topic and correlation-data properties can associate a reply with a request. Callers still need timeouts, authorization for reply topics, and duplicate-safe handling.

An IoT gateway architecture guide explains where protocol translation and local processing fit. For placement and synchronization choices, see the edge-to-cloud architecture guide and IoT data-processing architectures guide.

Practical Considerations

Define topics and message contracts

Choose a predictable hierarchy that reflects ownership and permissions, for example tenant/site/device/telemetry. Keep personal or secret data out of topic names, which can appear in logs and access policies. Document which identities may publish or subscribe to each branch, and version payload schemas so consumers can evolve safely.

Include application fields such as a stable device ID, event ID or sequence number, schema version, and observation time where they are needed. Do not rely on the broker’s delivery order or receipt time as a substitute for event identity and measurement time.

Try a local broker

Install Eclipse Mosquitto and its command-line clients. With Mosquitto 2.x, this command starts a verbose broker on the local machine’s loopback interface:

mosquitto -p 1883 -v

In a second terminal, subscribe to telemetry:

mosquitto_sub -V mqttv5 -h 127.0.0.1 -p 1883 \
  -t 'lab/sensors/+/telemetry' -q 1 -v

In a third terminal, publish a test message:

mosquitto_pub -V mqttv5 -h 127.0.0.1 -p 1883 \
  -t 'lab/sensors/sensor-01/telemetry' -q 1 \
  -m '{"schema":1,"device_id":"sensor-01","sequence":1,"temperature_c":22.5}'

The local-only listener is suitable for a lab, not a remote deployment. See the Eclipse Mosquitto broker documentation and its configuration reference before exposing a listener to other hosts. Windows users can run the Linux command-line tools in WSL. If a local container is part of the test setup, review container networking basics.

Secure the broker and clients

Bind a development listener to loopback, disable anonymous access when identities are required, and grant each identity only the topic permissions it needs. For example, an illustrative local Mosquitto configuration can reference password and ACL files:

listener 1883 127.0.0.1
allow_anonymous false
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl
persistence true
persistence_location /var/lib/mosquitto/

An ACL can separate a sensor publisher from a dashboard reader:

user sensor-01
topic write devices/sensor-01/telemetry

user dashboard
topic read devices/+/telemetry

These are examples, not a remotely deployable security configuration. For network access, configure TLS with certificate validation, use unique device credentials or certificates, rotate and revoke credentials, and test the broker’s authorization rules. Never expose a listener with anonymous access to an untrusted network. The OWASP security risks guide provides broader security context.

Design for failures and operations

  • Use bounded persistent queues on devices or gateways when data must survive outages; broker sessions alone do not guarantee indefinite storage.
  • Reconnect with exponential backoff and jitter so a site-wide outage does not create a connection storm.
  • Make consumers idempotent using an event ID or sequence number. For commands, publish a distinct application acknowledgement and report the resulting device state.
  • Set session and message expiry to match the business lifetime of the data. A delayed temperature reading may still be useful; an old actuator command usually is not.
  • Monitor connection churn, publish and delivery rates, rejected connections, queue depth, oldest queued-message age, and dropped messages. Test broker restarts, WAN loss, expired credentials, and full queues.
  • Automate repeatable broker configuration and rollout with tools such as Ansible, and verify persistence and recovery on the actual storage layer.

Common Misconceptions

“QoS 2 makes the whole application exactly once”

QoS 2 coordinates delivery in an MQTT protocol exchange. It cannot atomically commit a database transaction or prevent a consumer from repeating a side effect after a crash. Use idempotency keys and application-level transaction handling where duplicate effects matter.

“A persistent session is an unlimited offline queue”

Session expiry, broker queue limits, message expiry, storage, and service policy all affect what survives. Specify the offline period and overflow behavior, and use a durable application queue when the data’s importance exceeds the broker’s guarantees.

“A retained message is message history”

The retained value gives a new subscriber the current retained state for a topic; it is not an event archive. Store history in a database or log designed for that purpose.

“MQTT replaces security and monitoring”

The protocol does not decide which device is allowed to operate a relay or read a tenant’s data. Authenticate clients, authorize topics, protect transport, monitor failures, and plan credential lifecycle as part of the system design.

TBO Editorial

About the Author

TBO Editorial writes about the latest updates about products and services related to Technology, Business, Finance & Lifestyle. Do get in touch if you want to share any useful article with our community.