Categories
Eclipse IoT

Using MQTT-SN over BLE with the BBC micro:bit

The micro:bit is one of the best IoT prototyping platforms I’ve come across in the past few months.

The main MCU is a Nordic nRF51822 with 16K RAM and 256K Flash. A Freescale KL26Z is used for conveniently implementing a USB interface as well as a mass storage driver so as deploying code onto the micro:bit is as simple as directly copying a .hex file over USB (if your familiar with the mbed ecosystem, this will sound familiar :-)).

The board is packed with all the typical sensors and actuators you need for prototyping an IoT solution: accelerometer, compass, push buttons, an LED matrix, … What’s really cool, is the built-in BLE support, combined with the battery connector, making it really easy to have a tetherless, low-power [ref]You should keep in mind that the micro:bit, like other similar boards, is meant to be a prototyping platform, and for example having the KL26Z core taking core of the USB controller might not be ideal battery-wise, if you only care about doing tetherless BLE communications.[/ref], IoT testing device.

So how does one take the micro:bit and turn it into an IoT device? Since there is no Internet connectivity, you need to rely on some kind of gateway to bridge the constrained device that is the micro:bit to the Internet. You can of course implement your own protocol to do just that, but then you have to basically reimplement the wheel. That’s the reason why I thought the micro:bit would be ideal to experiment with MQTT-SN.

You can jump directly to the video tutorial at the end of the post, and come back later for more in-depth reading.

What is MQTT-SN and why you should care

If I were to over simplify things, I would just say that MQTT-SN (which stands for “MQTT for Sensor Networks”, by the way) is an adaptation of the MQTT protocol to deal with constrained devices, both from a footprint/complexity standpoint, and to adapt to the fact constrained devices may not have TCP/IP support.

MQTT-SN is designed so as to make the packets as small as possible. An example is the fact that an MQTT-SN client registers the topic(s) it wishes to us against the  server, this way further PUBLISH or SUBSCRIBE exchanges only have to deal with a 2-byte long ID, as opposed to a possibly very long UTF-8 string.

Like I said before, you really don’t want to reimplement your own protocol, and using MQTT-SN just makes lot of sense since it bridges very naturally to good ol’ MQTT.

Setting up an MQTT-SN client on the micro:bit

The MQTT-SN supports the BLE UARTService from Nordic, that essentially mimics a classical UART by means of two BLE characteristics, for RX and TX. This is what we’ll use as our communication channel.

The Eclipse Paho project provides an MQTT-SN embedded library that turns out to be really easy to use. It allows you to serialize and deserialize MQTT-SN packets, the only remaining thing to do is for you to effectively transmit them (send or receive) over your communication channel – BLE UART in our case.

In order to show you how simple the library is to use, here’s an example of how you would issue a CONNECT:

MQTTSNPacket_connectData options = MQTTSNPacket_connectData_initializer;
options.clientID.cstring = microbit_friendly_name();
int len = MQTTSNSerialize_connect(buf, buflen, & options);

int rc = transport_sendPacketBuffer(buf, len); /* wait for connack */

rc = MQTTSNPacket_read(buf, buflen, transport_getdata);
if (rc == MQTTSN_CONNACK) {
  int connack_rc = -1;
  if (MQTTSNDeserialize_connack( & connack_rc, buf, buflen) != 1 || connack_rc != 0) {
    return -1;
  } else { // CONNECTION OK - continue
  }
} else {
  return -1;
}

Now what’s behind the transport_sendPacketBuffer and transport_getdata functions? You’ve guess correctly, this is where either send or read a buffer to/from the BLE UART.
Using the micro:bit UART service API, the code for transport_getdata is indeed very straightforward:

int transport_getdata(unsigned char * buf, int count) {
  int rc = uart->read(buf, count, ASYNC);
  return rc;
}

You can find the complete code for publishing the micro:bit acceloremeter data over BLE on my Github. Note that for the sake of simplifying things, I’ve disabled Bluetooth pairing so as connecting to a BLE/MQTT-SN gateway just works out of the box.

MQTT-SN gateway

There are a few MQTT-SN gateways available out there, and you should feel free to use the one that floats your boat. Some (most?) MQTT-SN gateways will also behave as regular MQTT brokers so you won’t necessarily have to bridge the MQTT-SN devices to MQTT strictly speaking, but rather directly use the gateway as your MQTT broker.
For my tests, I’ve been pretty happy with RSMB, an Eclipse Paho component, that you can get from Github.

The README of the project is pretty complete and you should be able to have your RSMB broker compiled in no time. The default configuration file for RSMB should be named broker.cfg (you can specify a different configuration file on the command line, of course).
Below is an example of the configuration file so as RSMB behaves as both a good ol’ MQTT broker, but also an MQTT-SN gateway, bridged to iot.eclipse.org’s MQTT sandbox broker. Note that in my example I only care about publishing messages, so the bridge is configured in out mode, meaning that messages only flow from my MQTT-SN devices to iot.eclipse.org, and not the other way around. Your mileage may vary if you also want your MQTT-SN devices to be able to subscribe to messages, in which case the bridging mode should be set to both

# will show you packets being sent and received
trace_output protocol

# MQTT listener
listener 1883 INADDR_ANY mqtt

# MQTT-S listener
listener 1884 INADDR_ANY mqtts

# QoS 2 MQTT-S bridge
connection mqtts
  protocol mqtt
  address 198.41.30.241:1883
  topic # out

Bridging the BLE device(s) to the MQTT-SN gateway

Now there is still one missing piece, right? We need some piece of software for forwarding the messages coming from the BLE link, to the MQTT-SN gateway.

I’ve adapted an existing Node.js application that does just that. For each BLE device that attaches to it, it creates a UDP socket to the MQTT-SN gateway, and transparently routes packets back and forth. When the micro:bit “publishes” an MQTT-SN packet, it is just as if it were directly talking to the MQTT-SN gateway.

The overall architecture is as follows:

Note that it would be more elegant (and also avoid some nasty bugs, actually[ref]RSMB expects the first packet received on an incoming UDP connection to be a CONNECT packet. If the bridge forwards everything to the gateway transparently, that may not always be the case. If, instead, it takes care of encapsulating all MQTT-SN packets properly, that means you know need only one UDP socket from your BLE/UDP bridge to the gateway)[/ref]) to leverage MQTT-SN’s encapsulation mechanism so as to make the bridge even more straightforward, and not have to maintain one UDP socket per BLE device. To quote the MQTT-SN specification:

The forwarder simply encapsulates the MQTT-SN frames it receives on the wireless side and forwards them unchanged to the GW; in the opposite direction, it decapsulates the frames it receives from the gateway and sends them to the clients, unchanged too.

Unfortunately RSMB does not support encapsulated packets at this point, but you can rely on this fork if you want to use encapsulation: https://github.com/MichalFoksa/rsmb.

Visualizing the data: mqtt-spy to the rescue!

Like in my previous article about Android Things, I used mqtt-spy to visualize the data coming from the sensors.

Note that publishing sensor data in JSON might not be the best idea in production: the MTU of a BLE packet is just 20 bytes. Those extra curly braces, commas, and double quotes are as many bytes you won’t be able to use for your MQTT payload. You may want to look at something like CBOR for creating small, yet typed, binary payloads.
However, JSON is of course pretty convenient since there’s a plethora of libraries out there that will allow you to easily manipulate the data…

Using mqtt-spy, it’s very easy to visualize the values we’re collecting from the accelerometer of the micro:bit, either in “raw” form, or on a chart, using mqtt-spy’s ability to parse JSON payloads.

Video tutorial and wrap-up

I’ve wanted to give MQTT-SN a try for a long time now, and I’m really happy I took the time to do so. All in all, I would summarize my findings as follow:

  • The Eclipse Paho MQTT-SN embedded client just works! Similarly to the MQTT embedded client, it is very easy to take it and port it to your embedded device, and no matter what actual transport layer you are using (Bluetooth, Zigbee, UDP, …), you essentially just have to provide an implementation of “transport_read” and “transport_write”.
  • You may want to be careful when doing things like “UART over BLE”. The main point of BLE is that it’s been designed to be really low-power, so if you tend to overly communicate or to remain paired with the gateway all the time, you will likely kill your battery in no time!
  • The NRF5x series from Nordic is very widely available on the market, so it would be really interesting to run a similar MQTT-SN stack on other devices than the micro:bit, therefore demonstrating how it truly enables interoperability. If you build something like this, I really want to hear from you!
  • Although it’s true that there are not quite as many MQTT-SN libraries and gateways available out there as there are for MQTT, the protocol is pretty straightforward and that shouldn’t be preventing you from giving it a try!

By Benjamin Cabé

Benjamin Cabé is a technology enthusiast with a passion for empowering developers to build innovative solutions. He has invented an award-winning open source and open hardware artificial nose that he likes to use as an educational platform for people interested in diving into the world of embedded development.
He is currently a Developer Advocate for the Zephyr Project at the Linux Foundation and lives near Toulouse, France, where he enjoys baking sourdough bread with the help of his artificial nose.

20 replies on “Using MQTT-SN over BLE with the BBC micro:bit”

Hi Benjamin. I’m trying this out but the connack seemingly isn’t being sent back to the microbit, which continually tries to reconnect. The output from bbowl.js is:

Received from UART:
will: false
cleanSession: true
duration: 10
clientId: zuput
cmd: connect
Received from UDP:

in a continuous cycle. Debugging the microbit read() call shows it gets nothing. The above trace does not look like the one you show: your packet data from UDP is apparently encapsulated. I tried using the fork of RSMB which supports encapsulation to no effect. Can you tell me what I’m doing wrong? Thanks for the tutorial and code!

Hey Ian – pretty thrilled to see you’re trying this out!

I am pretty sure you need to patch microbit-dal with this fix that I’ve contributed upstream and that’s been merged but not released yet. The read() method has a bug when the buffer contains the null character.

Also, IIRC I’ve made several experiments indeed, and I think the “HEAD” of my Github relies on encapsulation, hence I used the RSMB fork with encapsulation indeed (overall the goal with encapsulation was, as you can imagine, to make BLE-to-UDP bridge much simpler and mostly stateless).

Please let me know if the fix to the microbit-dal library does the trick!

The microbit-dal fix works great, thanks! I wanted to try this scenario out using Tomoaki’s MQTT-SN gateway instead of RSMB, and it’s working great. It means each of the microbit devices automatically get their own MQTT connection – which has its pros and cons of course. What I want to do next is to try to add bluetooth support directly into Tomoaki’s gateway.

Cool! Please keep me posted about next steps, that sounds exciting 😉 FWIW I wanted to use Tomoaki’s gateway for the blog post myself, but it does not work out of the box on Mac/FreeBSD so there’s some hacks I had to do in some socket code, and then although the gateway mostly worked, it was using 100% CPU.

Hi Benjamin,

thanks a lot for your article! You were asking about other possibilities to run MQTT-SN on NRF5x-based devices. I just recently did a new implementation called `emCute` on top of UDP in RIOT (https://riot-os.org/api/group__net__emcute.html). To get started, one can simply run RIOT as a process in Linux (our so called `native` target -> https://github.com/RIOT-OS/RIOT/tree/master/examples/emcute).

I designed the implementation to be very easy to use and optimized for low memory usage (<2kb ROM on 32-bit platforms). It would be very nice to get some opinions on the approach! 🙂

Cheers,
Hauke

Hi,

thanks for your great article!

I have an issue with Bluetooth connectivity to Ubuntu 16. I connect micro:bit via clicking ON button near the device in Ubuntu. Micro:bit connects successfully (I can see “C” in the display), however after 10-40 seconds Micro:bit disconnects (“D”).

How can I achieve stable Bluetooth connectivity to my Ubuntu?

(And of course it would be great to have PIN pairing sometime)

Best,

Marijus Bernotas

Hi Benjamin,
Thank you for the great article. When I try to run it, I get the error below from bbowl.js .Do you have any idea? Many Thanks.
Onder

bluetooth state: [poweredOn]
scanning for bluetooth le devices…
found BBC micro:bit [pegoz] ca:ab:d7:84:5f:b3
trying to connect to BBC micro:bit [pegoz][ca:ab:d7:84:5f:b3]
noble warning: unknown peripheral caabd7845fb3
noble: unknown peripheral null connected!
noble: unknown peripheral null connected!
noble: unknown peripheral null connected!

Hello Benjamin,
I trying to do this , Send data to Cloud over MQTT-SN , i have question, what is the maximum number of node , or nrf52832 module that i can connect to gatway (Broker)?

So here you are essentially using MQTT-SN like a container or serialization handler. I couldn’t figure out the architectural position of MQTT-SN here!
Sure MQTT-SN works well to bridge these messages. But if there is a bridge device and if it can talk both BLE and TCP/UDP over IP, then it can get the BLE messages in whatever format/protocol it wants southbound, directly pump it to an MQTT topic northbound! Why would you need a layering or indirection via UDP in between?
If MQTT-SN directly worked over BLE like IPv6 or something then it would have made more sense. Your thoughts?

Hi Benjamin,
thanks for sharing this aproach! I’ve also seen many advantages using the ‘Caliope mini’ https://calliope.cc/en which is an advancement of the micro:bit where all components are placed on one side. It is very robust and used for education in elementary schools for a ‘get started’ in coding.
For a rapid prototyping of an IoT application and to feed our backend solution continously with data, I’m simply reading out its serial interface and sending the sensor data from the gateway (my laptop using paho) to the mqtt-broker. The coding of the calliope mini can be done very easily by js or a provided graphical building block environment. I’ going to setup the mqtt-sn client and sending via BLE on the calliope soon …
Best, Nicole

Hi Nicole

found it very helpful you pointed to the calliope device.
Residing in Austria am educating Youngsters 15+ hoping to get them interessted in understanding IOT in my vacation times.

Professionally I try to find new approaches is decentralizing Event-documentation in medical informatics.

Like: How to provide microdevices which are able to do document very small parts of a medical documentation Event, like coding an ICD Code in an Ambulance while treating the Patient on the fly.

So please feel free to contact me if something interresting is going on with your experiences regarding calliope

Hi Benjamin,
I want to do the similar demo but on a different platform. The same MQTT-SN code will be ported to a Nordic 5 serial base dev board, not microbit. The transparent bridge and gateway will run on Windows cli.
I see that the bridge you used is bbowl.js which works great on your setup. I am looking for the similar tool which is Windows based. I see from GitHub, there is project provide the similar features, but it run on POSIX system.
Can I have you suggestion of what/where should I look for?
By the way, your demo is very interesting!

Thanks a lot Benjamin!!! thanks-a-lot!!!!

After a lot of hours looking for the connection establishment problem vía uart I was very happy when I found your comment with this fix in “MicroBitUARTService.cpp”:

while(i = 0)

A simple and beautiful line of code!

Hi, great project. Respect.

I have one question. When I have two deviceses with BLE (master and slave) it is possible to create mqtt sn between them? maybe you have also code to master ble device?

Leave a Reply

Your email address will not be published. Required fields are marked *