You are excited, you have just finished your new IoT device, and have many excellent ideas on how you will use it remotely, on your smartphone, or even from your computer at the office. You now start thinking about how you will send and receive data to this new device of yours…
There are many ways that you can do this, but today, I would like to suggest a very easy and lightweight data transfer protocol. It is called MQTT. Some of you may already be using it, or you may at least have heard about it. Read on if you would like to know more about what it is, as well as how it works.
The History of MQTT
MQTT was designed by Andy Stanford-Clark (IBM) and Arlen Nipper (Cirrus Link, then Eurotech) in 1999. It was first used to monitor an oil pipeline running through the desert. The goal was to design and implement a protocol that is bandwidth-efficient, lightweight and uses very little battery power, because the devices were connected via satellite link which, at that time, was extremely expensive.
In 2013, IBM submitted the MQTT v3.1 protocol to the OASIS specification body with a charter that ensured that only minor changes to the specification could be accepted.MQTT-SN is a variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as Zigbee.
Historically, the “MQ” in “MQTT” came from the IBM MQ (then ‘MQSeries’) MQ product line. The protocol, however, provides publish-and-subscribe messaging (no queues, in spite of the name) and was specifically designed for resource-constrained devices and low bandwidth, high latency networks. This makes it an excellent candidate for data transmission on IoT and other low resource devices.
The Protocol Architecture (How does it work)
MQTT uses a client-server architecture, where the server is called the broker, and the client, called a client. The broker typically functions like a post office, in the sense that it doesn’t use the client’s address, but rather the subject of the topic that a client is subscribed to, to determine which client should receive a certain message.
This enables many clients to subscribe to the same subject, with each receiving the same message. Clients can also publish a topic, thus transmitting or sending a message to other clients. This concept make bidirectional communication possible between clients, and it also ensures that it is extremely easy to use.
What do theses topics look like?
An MQTT Topic is a text string, delimited by a /
for example, let us say you have a device in your kitchen, that monitors the temperature, and controls the lights and microwave oven.
This device may use the following topics to publish (send data) or subscribe to (receive data)
@msg/myhouse/kitchen/temp
@msg/myhouse/kitchen/microwave
@msg/myhouse/kitchen/lights
@msg/myhouse/livingroom/lights
@msg/myhouse/bedroom/lights
As you can see, we can easily group topics by their location. It would thus be very easy for a smart home controller, like OpenHab or similar to get status or set a particular state in a certain room or area.
This can be done by using wildcard operators, of which there are two, the + and #
If we want to subscribe to all the lights in the house, we can subscribe to the following topic:
@msg/myhouse/+/lights
This will give us the data from the following topics:
@msg/myhouse/kitchen/lights
@msg/myhouse/livingroom/lights
@msg/myhouse/bedroom/lights
You can also subscribe to a multilevel wildcard topic, for example:
@msg/myhouse/kitchen/#
This will subscribe you to all the topics related to the kitchen.
How do I get access to MQTT
There are quite a few online MQTT brokers available that allow you to apply for a free account. These services have many limits on the amounts of messages you may send or receive, but generally, they are quite useful.
Adafruit.io should be well known. It is stable, and easy to use. They do however have a few limitations on how many devices and messages you can add to the service.
NetPie.io is a fairly unknown provider outside of Thailand. They limit you to 3 Projects, with 10 devices per project, but their messaging limits are extremely liberal and permissive. This service will also always be free (or at least that is according to NetPie themselves). On the negative side, all the configuration is available in English, but all the documentation is in Thai. You can however use Google Translate to translate it into a language that you can understand.
If you are not comfortable with running your data on an online broker, you can also download and install one of the many MQTT brokers that are available for installation on Linux and MS-Windows. This will, however, mean that you should also have permanent Internet access, as well as a public IP address, to enable your devices to connect to your broker.
Microprocessor and other Device support
MQTT Libraries are available for use on Arduino, STM32, ESP32 as well as Raspberry PI and Python. You can also get clients and brokers for the Linux and MS-Windows platforms.
Smartphone Access
Various applications exist on the google playstore, as well as the Apple App Store, that allow you to connect to an MQTT broker, subscribe to topics, and also publish topics. Many of these are free to use, and also provide you with a nice user interface.
You can also write your own smartphone app, if you are skilled enough, to do exactly what you want it do do.