I've been learning a lot about IoT protocols lately. Come learn the wonders of MQTT and CoAP!
The protocols that rule over how a device transmits and receives data can be a critical component behind how the rest of its resources are managed. Arduino and Arduino-like IoT "things" especially, with their limited flash and SRAM, can benefit from specially crafted IoT protocols.
There are a quite a few IoT protocols out there to choose from. XMPP is a popular, proven, XML-based service, but it's not as lightweight as other alternatives. DDS is a powerful device-to-device service that offers high performance data distribution, but still a little bit too resource intensive. There are a handful more, but the two that have caught my eye the most these days are MQTT and CoAP.
MQTT (Message Queue Telemetry Transport) is my IoT protocol of choice right now. It was created nearly 15 years ago with an eye towards remote sensor nodes ("remote" as in space), so it was designed to conserve both power and memory. It's a message queuing protocol that uses a publish/subscribe methodology to allow multiple clients to post messages and receive updates from a central server. For example, using MQTT, a connected device can subscribe to any number of topics hosted by an MQTT broker (a server equipped with MQTT). Any time a different device publishes data to one of those topics, the server sends out a message to any-and-all connected subscribers of those topics alerting them to the new data.
What really makes MQTT perfect for small embedded electronics is its lightweight packet structure - the entire protocol is documented on a single page. Simply set up a session with a MQTT broker, tell it what topics you want to subscribe to, and maintain that session as long as you need.
There's a really solid Arduino library to help implement MQTT -- knolleary's pubsubclient. An Arduino with an Ethernet Shield/WiFi Shield/etc., or just a bare Yun or Edison, can utilize the pubsubclient library for all of its MQTT-publishing-and-subscribing needs.
If you're trying to find an application for MQTT, look no further than our own data.sparkfun.com, which supports the protocol! You can subscribe to specific topics on your Phant data stream, or use MQTT to publish your sensor readings. Check out this GitHub topic for help getting started. Plus, since Phant is open source, you can install the phant-server-mqtt, phant-input-mqtt, and phant-output-mqtt modules to enable MQTT on your own Phant server.
To test MQTT out, I decided to try to use Phant to "control" an Arduino, not just host its sensor readings. I whipped up a quick sketch and threw it on a Arduino/Ethernet Shield combo. Here's an example of how easy the library makes MQTT:
Now whenever data is posted to my stream, the RGB LEDs on my Arduino are updated too. The poster to Phant can be anything from another Arduino (publishing with MQTT?!) to a customized web URL (e.g.: https://data.sparkfun.com/input/roE6z8QZq5iyEdrMGwdG?private_key=jklEy4DG75TY1lzm6nl6&blue=1024&green=42&red=66).
One last note on MQTT: if you're looking to give the protocol a try sans Arduino, I've found Mosquitto to be a great, command-line-based tool to test it out.
CoAP (Constrained Application Protocol) caught my attention because it's billed as something of a lightweight version of HTTP. If you've done any HTTP with an Arduino, you may be familiar with what a memory sink those verbose messages and responses can be. Instead of messing about with strings, CoAP packets are mostly based around bit mapping.
To compare the two, a simple HTTP GET header may look like this:
GET /output/roE6z8QZq5iyEdrMGwdG/latest.txt HTTP/1.1 Host: data.sparkfun.com Connection: close
...very human-readable, but also very "stringy." It may not look like much, but that's almost 100 bytes, and they can grow to be much more than that. On the other hand, CoAP headers take on a bit-mapped structure like this:
As few as 4 bytes (ignoring all of the options)! You can't pass as much information through, but it gets the job done. Also as a bonus, at least as far as resource requirements go, CoAP, by default, operates over UDP instead of TCP.
If you're too stubborn to swap horses, or if the Arduino's just all you have on your desk, CoAP is an interesting alternative to HTTP. There are a few Arduino implementations out there, like microcoap, if you'd like to check it out further.
Both of these protocols, and all of the rest of the "IoT protocols" hold a lot of potential and power in the encroaching IoT world. MQTT is a great choice for multi-device networks, and CoAP is well-suited for point-to-point connections when resources are limited. I'm excited to find some projects to put them to use in 2015!
Thank you for your post. I recently installed phant on a RaspberryPi and it worked without any problems. But: I don't succeed in starting the mqtt server. Could you please explain how to start the phant-mqtt-server ?
Great walkthrough, but could you add two more established protocols: AMQP and ZMTP/0MQ? Thanks.
The thing I don't like about MQTT is that every device must be configured with the address of a broker. This is an extra configuration step, and the broker can become a single point of failure; if it goes down, all devices are unusable. I prefer point-to-point self configuring protocols such as ZeroMQ
Thanks for the indirect reference to codebender.cc - just what I was looking for to publish my sketches!
We made a CoAP product using a Raspberry Pi connected to any old Android using SMS messaging. We had a Kickstarter project recently. We're QAing the code now and should be shipping it soon. It's a remote temperature sensor and we plan to release the code as open source as soon as we are sure there are no bugs. Check it out at http://www.amazon.com/dp/B00RJKPKT0/
Jimb0, which SparkFun Ethernet/WiFi module/break-out-board would you recommend to implement the MQTT protocol with. My desire is to create a home monitoring system around Arduinos.
But it took me to get to the bottom of the post to realize IoT was "Internet of Things"... ;)
I find the term Internet of Stuff to be slightly more accurate.
Thank you for a very timely and pertinent post. Are you reading my mind?