Category Archives: IoT

Monetizing IoT Data using IOTA

There is a lot of buzz around cryptocurrencies  – will BTC be over $20,000 by the time this blog post goes live? 🙂 – these days, but beyond the hype, one thing is pretty clear: it enables a decentralized economy that is particularly interesting to look at from an IoT perspective.

For the Internet of Things, I believe the role of cryptocurrencies and distributed ledgers is threefold:

  • monetization of IoT data – cryptocurrencies make it really easy to implement scenarios where data producers can get a financial compensation in exchange for the value provided by the data they expose. Think of a weather station you would put on your balcony: people or corporations could have access to the feed of sensor data in exchange for some tokens.
  • smart contracts – taking monetization a step further, smart contracts can enable higher-order, secured, transactions to automatically take place on the network. If you take the example of the weather station, you could sign a smart contract with a 3rd party whereby you get paid for giving access to your sensor data if and only if you can provide an uptime of at least 99%. Note that this would probably require a trusted third party responsible for checking your uptime and storing it in the ledger.
  • security – manufacturers can use a distributed ledger to store information that can be used to help securely bootstrap  IoT devices, by e.g allowing customers to check the authenticity of a chip.

In this blog post, we will focus on the first topic: monetizing IoT data.

For that, we will look at leveraging IOTA’s distributed ledger. We will be showing how anyone can be given access to “simple” IoT sensor data on an MQTT broker, while only people who are paying for it will be able to access the live, raw, data stream.

Use case

We will be basing this article on the use case of someone, let’s call her Jane, willing to give access to a live sensor data stream coming from a smart plug, granted that she gets some money (iotas) in return 🙂 .

The instant power consumption metrics will be published to an MQTT broker twice per second, making for a pretty accurate sampling that people might be interested in buying. On the same MQTT broker, Jane will also be making available the 1-min moving average of the power consumption for anyone to access, even if they don’t necessarily want to pay extra money.

When John, who is maybe a data scientist interested in getting his hands on “real” data wants to get the power consumption from Jane’s smart plug, he has two options:

  • access the MQTT broker and subscribe to data published on the ‘public’ sensor/1m topic, for free,
  • send some IOTAs to Jane (she’s decided that 500 iotas is what people should pay to access the data for one hour), for his MQTT client to be granted access to sensor/live topic where the live, and more valuable to him, data stream is being published.

Technology being used

  • Eclipse Mosquitto as our MQTT broker of choice.
  • Two IOTA wallets, one where the payments to Jane shall be sent to, and one for John to pay for accessing sensor data
  • mosquitto-auth-plug, a flexible authentication plug-in for Mosquitto. We will use it for implementing a flexible access-control model, based on the information available on the IOTA tangle (i.e payments)
    • The ACLs (access-control lists) will be stored in a MongoDB database, used as the authentication backend for mosquitto-auth-plug
  • A Belkin WeMo Insight smart plug.

Demo

Enough with the words! I’ve put together a video (click here if the video doesn’t show up below) that shows the whole scenario in action. While you will quickly realize that I do a bad job impersonating Jane and John, what I hope you notice is that this is a real demo, and anyone can implement something similar today.

The technology is here, and while it has some limitations, I do think it’s worth exploring what new scenarios it enables.

Check out the code

I have published all the code behind this proof-of-concept on my Github repository, please check it out and open pull requests or issues if you happen to give it a try!

Next steps?

This is obviously a simple proof-of-concept, and I am almost certain it is not free of security issues. I would certainly be happy to hear from you if you have feedback :smile:.

I see at least two things that would be worth improving:

  • IOTA transactions, just like with many other distributed ledgers, can take a significant amount of time before they get confirmed. Having to wait for a couple hours after you’ve made a payment to effectively get access to the data can seriously impact the user experience. A nice improvement would be to rely on IOTA Flash Channels,  as they allow to perform instantaneous, off-tangle transactions.
  • In the current PoC, the Mosquitto server only accepts connections from a list of users already agreed upon. This wouldn’t be difficult, and much smarter, to allow anonymous access to anyone, and let clients interested in having full access to the data indicate in their payment message the username/password they plan on using when reconnecting as authenticated clients. This information should, of course, be encrypted with a key only known to the client and the server – this key, unique for each client, could be broadcasted to anonymous clients on the $SYS topic, for example…

Do you see other areas for improvement? Have you seen similar solutions implemented with other cryptocurrencies?

Note that the IOTA folks have recently announced an IoT data market that seems to implement a similar idea, but I haven’t had time to look into it closely.

Cloud Native IoT Development in Practice

With Kubecon happening this week in Austin,  it is probably a good time to write an article on the role of containers and having a cloud native strategy for IoT, don’t you think?

Over the past years, Docker and its ecosystem have been instrumental in modernizing our approach to writing and shipping software. Today, more and more applications are becoming cloud native, meaning that not only core functionalities are being isolated as (micro)services, but also that applications are evolving to be first-class citizens in cloud environments (e.g exposing health metrics, acting as stateless processes, etc.).

In this blog post, we will be looking at how to approach cloud native IoT development. We will be deploying an end-to-end IoT solution for power consumption monitoring on OpenShift. The deployed services include:

  • IoT connectivity layer – getting telemetry data into a backend system is a challenge in itself, and we’ll see how Eclipse Hono can help with IoT connectivity ;
  • Device data simulator – as a way to illustrate how thinking cloud native for IoT can help make your application scale, we will actually have device simulators running on our cluster ;
  • Monitoring dashboards – we’ll see how we can leverage Grafana to visualize the data coming into our cluster, and its overall health ;
  • End-user application – getting IoT data into our backend is one thing, but we’ll also see how to develop a simple web application to visualize our instant power consumption ;
  • Cloud IDE – we will be using Eclipse Che to develop the web application mentioned just before.

So, let’s break this down!

Firing up a single-node OpenShift cluster with Minishift

The best way to get an OpenShift cluster setup is to use Minishift, which helps you deploy a single-node cluster on your local machine.

You can download the latest Minishift releases, and find install instructions on the project’s Github repository.

Once you have the Minishift command installed, firing up the cluster is actually pretty easy. Here’s the command I use on my quad-core Intel i7 MacBook Pro:

minishift start --cpus 4 --memory 12GB --metrics --disk-size 40GB

Obviously, your mileage will vary depending on the number of CPUs, memory, or disk space you want to allocate to your cluster, but no matter what your operating system is, soon enough you should be able to log into the OpenShift web console.

Scalable IoT Messaging with Eclipse Hono

Eclipse Hono enables scalable and secure ingestion of large volumes of sensor data into backend systems.

Eclipse Hono Overview
Eclipse Hono Overview

The different building blocks of Hono (protocol adapters, device registry, …) can be deployed as microservices.

Deploying Hono on OpenShift is really easy: just follow the instructions available in the documentation! In my setup I’ve deployed a Hono environment where the core messaging functionality is taken care of by EnMasse, but you should also be fine with the regular, Qpid Dispatch-based, distro.

Feeding data into our system

Now that our IoT connectivity layer is deployed, with Hono running within our cluster, we want to ingest data into our system, and consume this data to e.g store it in a database.

Jens Reimann put together a nice setup that uses a public dataset of the energy consumption of a residential house to simulate “real” IoT devices. The application essentially deploys two services on our cluster:

  • A data simulator that sends energy consumption information to Hono using MQTT. The producer can be configured to simulate 1, 10… 10,000 of devices. And of course, you can also scale up the number of pods for the simulator to simulate even more devices.
  • A data consumer that taps into Hono’s telemetry API to retrieve data coming from all our virtual houses, and dump it into an InfluxDB time-series database.

If you follow the install instructions provided in Jens’ repo, you should have your simulator and consumer running in your OpenShift cluster, and data will start showing up in your InfluxDB database.

Here’s an example of how my Grafana dashboard looks like:

Running Eclipse Che on OpenShift

So we now have an IoT messaging infrastructure deployed in our OpenShift cluster, as well as an IoT app effectively pumping business data into our backend. Wouldn’t it be cool if we could also have the developer tools needed to write our user-facing application running in the same cluster?

Eclipse Che is a developer workspace server and cloud IDE that we will be deploying in our cluster, and using to write some Javascript code right from our browser. Deploying Eclipse Che on OpenShift is pretty straightforward and you can refer to the instructions on the Eclipse Che website to deploy it into your OpenShift project.

In my case, here’s how I would get the nightly build of Che 5.x deployed into my OpenShift project:

export CHE_IMAGE_TAG="nightly-centos"
export CHE_MULTIUSER="false"
export CHE_OPENSHIFT_PROJECT="hono"    

DEPLOY_SCRIPT_URL=https://raw.githubusercontent.com/eclipse/che/master/dockerfiles/init/modules/openshift/files/scripts/deploy_che.sh
WAIT_SCRIPT_URL=https://raw.githubusercontent.com/eclipse/che/master/dockerfiles/init/modules/openshift/files/scripts/wait_until_che_is_available.sh
STACKS_SCRIPT_URL=https://raw.githubusercontent.com/eclipse/che/master/dockerfiles/init/modules/openshift/files/scripts/replace_stacks.sh

curl -fsSL ${DEPLOY_SCRIPT_URL} -o ./get-che.sh
curl -fsSL ${WAIT_SCRIPT_URL} -o ./wait-che.sh
curl -fsSL ${STACKS_SCRIPT_URL} -o ./stacks-che.sh

bash ./get-che.sh ; bash ./wait-che.sh ; bash ./stacks-che.sh

And that’s it! Depending on your Internet speed it may take a few minutes for everything to get deployed, but Eclipse Che now just is a click away, accessible through a URL such as http://che-hono.192.168.64.2.nip.io/.

Writing our user-facing ExpressJS app from Eclipse Che

However quick this all was to set up, we’ve essentially worked on the infrastructure of IoT application: messaging, development environment, …

Arguably, the most interesting part is to actually make use of the data we’ve been collecting! For this, we will be developing a Node.js application that will be getting the overall electricity consumption metrics from  InfluxDB and displaying them on a fancy gauge.

The final version of the app is available here on my Github account. It uses Express, the InfluxDB client for Node.js, as well as gaugeJS for the gauge widget.

There are at least two interesting things to note here:

  • Thanks to Eclipse Che, we can not only easily set up a stack for Node.js development in no time, but we really have a full-blown IDE that includes advanced content assist features – not something you get that often when developing Javascript code. I can tell you that not being an expert in using the InfluxDB Javascript API, having code completion available in the IDE has been a pretty useful thing 🙂

  • Since Eclipse Che runs on the very same OpenShift cluster that holds our IoT backend, we can easily test our code against it. From within our Che workspace, all our environment variables are set up, and we can e.g access Hono, InfluxDB, etc.

Closing the loop

One last thing… We now have a Node.js application built from Che, that lives in its own Github repo. Wouldn’t it be great to have it run in our cluster, alongside the rest of our microservices?

From the OpenShift console, you are just a couple clicks away from deploying the Node.js app into the cluster. You can use the template for Node.JS applications to automatically build a Docker image from the Github repository that contains our app. It will automatically detect that the repository contains a Node application, install all its dependencies, build an image, and then deploy it to a pod with a route properly configured to expose our app outside of the cluster.

You could also set up a hook so that whenever there is a new commit in the upstream repository, the image gets rebuilt and redeployed.

Takeaways

Hopefully, this blog post helped you understand the importance of thinking cloud native when it comes to IoT development.

If you use Eclipse Hono for your IoT connectivity layer, for example, you automagically get a piece of infrastructure that is already instrumented to autoscale, should the number of devices connected to your backend require it.

Thanks to Eclipse Che, you can develop your IoT services in a controlled environment, that is already part of the same cluster where the rest of your IoT infrastructure and applications is already running.

Final words: don’t push it!

Now, I cannot conclude this blog post without a personal observation, and something I hope others have in mind as well.

Many moons ago, I used to teach people how to develop plugins for the Eclipse RCP platform – a truly great, highly extensible, framework. However, the platform being so modular, soon enough, you could end up turning everything into a plugin, just for the sake of having an “elegant” design. And when you think about it, microservices are very similar to Eclipse plugins…

Does it really make sense to isolate really tiny microservices in their own containers? For each microservice, what’s the overhead gonna be like to be maintaining its build system, access rights to the corresponding Git repository, configuration files, …?

You should absolutely have a cloud native strategy when it comes to building your IoT solution, but don’t overthink it! Your microservice architecture will likely emerge over time, and starting with a too small service granularity will just make things unnecessarily complex.

Please use the comments section below to share your thoughts on cloud native and IoT. I think this will be a hot topic for the near future, and I’m interested in hearing your views!


Final note: Shout out to Jens Reimann and Dejan Bosanac from Red Hat who’ve put a lot of work into the Hono part of the demo (running Hono on OpenShift, and putting together the demo app publishing electricity consumption information). Thanks also to Eugene Ivantsov for helping out with getting a proper Eclipse Che stack for JavaScript set up.

Enter the Open IoT Challenge 4.0: some ideas for your submission

There are only a couple weeks left (deadline is Nov. 13) for entering the fourth edition of our Open IoT Challenge!

The Open IoT Challenge encourages IoT enthusiasts and developers to build innovative solutions for the Internet of Things using open standards and open source technology.

Open IoT Challenge 4.0

You probably remember last year’s edition and its winner, Sébastien Lambour, who built an amazing solution: InTheMoodForLife. The project aims at analyzing sleep patterns to anticipate mood disorder episodes for people suffering from bipolar disorder.

Sébastien won $3,000, and we also provided him with an additional $1,000 to fund his participation in Eclipse IoT events where he presented his project. For example, Sébastien attended the Eclipse IoT Day in London, a couple months ago, and gave a brilliant talk where he shared his experience.

As a reminder, there is already a nice incentive for you to enter the challenge, even if you feel like you can’t compete for the first place (and you shouldn’t feel like that, by the way!). In fact, if you are among the 10 best submissions, our jury will award you a $150 gift certificate that will help you buy the hardware you need to build your project!

I thought it would be useful to share some of the ideas I would like to see come to life, and some of the technologies that I think would be interesting to use.

Deep learning

With Deeplearning4j moving to Eclipse, now is the perfect time to think of having deep learning play a role in your project. Whether you plan on doing image recognition, predictive maintenance, or natural language processing, I am curious to see if Deeplearning4j, or other open source projects in the machine learning / deep learning area, can help you implement your solution. There are some IoT gateways out there that are fairly capable, and even some that include a GPU, so please be crazy and push the boundaries of edge computing! 🙂

Industrial IoT

Many – too many, probably – IoT solutions are very consumer oriented (connected toothbrush anyone?), so I certainly hope to see lots of projects that are more industry-focused. Our production performance management testbed can probably give you some pointers regarding open source projects that are applicable, and even provide you some code to get started.

Blockchain and distributed ledgers

Beyond the buzzword and the hype, blockchain is still an interesting topic, which you may want to put to use in your Open IoT Challenge project.

Security is one key aspect of IoT. Blockchain might be a way for you to secure communications or data exchanges, but it is also interesting to think about using distributed ledgers as a way to enable a sharing economy.

What if you could “share” some of your IoT devices’ processing or networking power and get compensated, in a secure way, using a blockchain? And how about true “pay-per-use” scenarios, where e.g construction companies share a pool of high-value power tools for which actual usage is tracked, logged, and billed accordingly, through a blockchain?

Of particular interest, in my opinion, is IOTA, an open source distributed ledger which, by design, has no transaction fees (it has no miners, as the 2,779,530,283,277,761 (!) tokens that form the ‘tangle’ were all generated in the so-called genesis transaction).

This means that you can leverage the IOTA tangle to implement secured micro-transactions in your IoT solution, e.g to expose (and monetize) sensor data to the world. I would be particularly curious to see how IOTA performs on constrained devices, and how well it scales.

Low-power, long-life

MangOH RedOne of our sponsors this year, Sierra Wireless, will be providing the 10 best proposals with a MangOH Red IoT board. This board is perfectly suited for low-power IoT applications and would be the ideal partner in crime if running for years out of a small battery is important to you.

It is often an afterthought in IoT to think about the maintenance of the equipment. I am interested in seeing how your proposal will highlight how you plan on making your solution easy to operate, including things like over-the-air updates, energy consumption optimization, etc.


If you are not sure if your idea would make for a cool project, feel free to ping me, I will be happy to give you some feedback 🙂

I am looking forward to reviewing your proposals, and seeing all the cool projects you will be building over the next few months!

Submit by Nov. 13 for the IoT Challenge

And last but not least, big thanks to Bitreactive, CONTACT Software, Eurotech, Intel, Red Hat, and Sierra Wireless for sponsoring the Open IoT Challenge this year.

4 Takeaways from “All Things IoT” week at EclipseCon Europe

Last week, the Eclipse IoT community was pretty busy at EclipseCon Europe. We were having our largest Eclipse event of the year, and it featured lots of IoT.

After a much needed weekend break to recover from an incredibly fruitful week, I am taking some time to write down some of my personal takeaways.

Oh, and before you ask: we are working on uploading all the IoT sessions from the IoT Working Group meeting and IoT Day to YouTube. They will be available shortly, and as you can see in the blog post below, some of them already are! 🙂

IoT Day featured lots of real-world talks

I was really pleased with the turnout of the IoT Day. What was particularly interesting was hearing not only community insiders tell us about what they are doing for, and with, Eclipse IoT projects, but also getting the point of view of people who are pure consumers of the technology.

This year, for example, we had people like Müge Kural (Eteration) or Nicola La Gloria (Kynetics) telling us how projects like Eclipse Kura or Eclipse hawkBit have helped them implement use cases such as advanced dashboards for Electric Vehicles, or scalable software updates for Android-based IoT devices.

Look out for the video recordings of their talks, in the meantime, you can read more about Kynetics’ story in our recently published case study.

Eclipse IoT Testbeds Hackday

Open IoT Testbed Hackday at EclipseCon Europe
Open IoT Testbed Hackday at EclipseCon Europe

As you probably know by now, we are ramping up our Open IoT Testbeds initiative, and it was great to use EclipseCon Europe as an opportunity to spend some quality time with the different companies involved in the testbeds and discuss next steps. To encourage collaboration, we held a “hackday” on Wednesday.

There was a strong focus on the Industry 4.0 / Production Performance Management testbed, and it was great to see participants from different companies sitting at the same table, brainstorming about the roadmap for the next few months.

One of my action items from the hackday was to get a mailing list set up. So, if you want to get more involved in the testbeds, you can now subscribe to iot-testbeds@eclipse.org to participate!

It’s all about the integration

EclipseCon Europe coincides with the anniversary of the Eclipse IoT Working Group, and 6 years into the adventure it is great to see that this year, even more than others, the project teams spent a lot of time discussing how to better integrate the different projects with one another. For example, it has become pretty clear that Eclipse hono will probably become the de-facto standard for device connectivity, and there were lots of discussions on how to provide hono protocol adapters for PPMP (Eclipse Unide), OPC-UA (Eclipse Milo), or Eclipse Kura gateways.

Also, at the IoT WG meeting, we discussed how to structure the community work better and collaborate towards a more integrated Eclipse IoT stack. It has been proposed that we establish a sub-committee of the IoT WG dedicated to integration work. Expect to see more on that topic soon!

New projects got unveiled

You may have seen the news: just a week before EclipseCon, two new Eclipse IoT project proposals have been announced.

First is Eclipse Thingweb, a project that will be hosting an open-source toolkit for the W3C Web of Things ecosystem. Matthias Kovatsch from Siemens presented the project, and you can get his slides here.

The other project, Eclipse Cyclone, will put an open source implementation of the DDS (Data Distribution Service) middleware standard of IoT developers. Hans van’t Hag from ADLINK gave the presentation below, and his slides are here.

I am looking forward to seeing these projects move to Eclipse IoT over the next few months!

Hack the Eclipse IoT Open Testbeds at EclipseCon Europe

By now you are probably pretty aware that EclipseCon Europe, coming in just a few weeks in Ludwigsburg, Germany, Oct 22-25, will feature All Things IoT!

But wait, there’s more! We will use the opportunity of having a lot of people from the Eclipse IoT community in the same location to spend some time hacking the Open IoT Testbeds on Wednesday, October 24.

“What can I expect from the Testbed Hackday?”

As we will be unveiling a new testbed focusing on Production Performance Management of industrial equipment, and with the Asset Tracking Management testbed well on its way, there will be many people who have been working on these testbeds attending the hack day.

They will be available all day to discuss (and experiment live!) how to extend them, and of course ready to show you how they are built.

“Why should I attend?”

  • If your company is creating IoT products that you would like to integrate with the existing testbeds,
  • If you are involved in an Eclipse or Eclipse IoT project and would like to see how your project can get involved complement the solutions already demonstrated in the existing testbeds,
  • You want to experiment with the actual hardware that the testbeds are featuring,
  • You are interested in starting a new testbed and would like to gauge the interest of the community and brainstorm some ideas face-to-face.

Please let me know if you plan on attending! I think the Open IoT Testbeds are a great opportunity for people and companies to collaborate on real use cases, and to demonstrate by example how open source, complemented by a strong commercial ecosystem, can jump-start the development of IoT solutions.

I really think the Open IoT Testbeds are a great opportunity for people and companies to collaborate on real use cases, and to demonstrate by example how open source, complemented by a strong commercial ecosystem, can jump-start the development of IoT solutions. If you share this conviction, you really don’t want to miss the hack day… and all the other days of EclipseCon Europe!

EclipseCon Europe 2017

Unveiling the Eclipse IoT Open Testbeds

Today we are announcing the Eclipse IoT Open Testbeds, a new initiative for driving adoption of open source and open standards in the industry.

For more than five years, over 30 open source projects have been calling Eclipse IoT home. Yet, it doesn’t necessarily make it easy for people to understand how to put all the pieces together, from integration with sensors and hardware, to networking and connectivity, to cloud computing and enterprise integration.

More often than not, I am asked about where to find blueprints or reference architectures for IoT, and how one is expected to leverage open source software such as what Eclipse IoT has to offer. These are very legitimate questions as building any IoT solution requires much more than just open source software components.

I believe the Eclipse Open IoT Testbeds are a unique approach to answering these kind of questions, especially since this is the first time IoT leading companies are effectively developing the testbeds in open source.

Open Source FTW!

Creating testbeds that demonstrate how a particular set of technologies can be used is certainly not a new idea, I’ll give you that. What is unique with the approach we are taking, though, is that we are making the testbeds available in open source.

This means that you can really learn firsthand how all the pieces of an IoT solution are being put together to solve a real business case, as well as experiment with the actual code and dive into the architecture.

Over time I certainly expect people will start forking the testbeds’ code to create their own extensions and, even better, will contribute them back to the community.


Open Testbed for Asset Tracking

The first testbed we have been working on is around Asset Tracking Management.

In a nutshell, we are showing how to track valuable assets (think expensive/valuable parcels such as artwork) in real-time in order to optimize their transport, and in particular minimize the costs due to spoilage, damage or delays.

The testbed features Eclipse open source projects such as Eclipse Kura, Eclipse Kapua, Eclipse Paho or Eclipse Che, but is of course also leveraging other technologies and commercial offerings – like any solution should, right?

Head over to the Asset Tracking testbed webpage to learn how, to name a few, OpenShift, Zulu Embedded, Samsung ARTIK, and more, have been integrated to demonstrate a full end-to-end IoT solution, all the way from data collection to complex event processing, to exposing information to 3rd parties through open APIs.

What’s next?

The Asset Tracking Open Testbed is our first take at demonstrating how companies are building real IoT Solutions today.

We are already planning to create other testbeds around e.g Smart Manufacturing, and therefore are inviting anyone interested in existing or future testbeds to join us at https://iot.eclipse.org/testbeds.

Join us at Red Hat Summit and IoT World 2017!

If you are attending Red Hat Summit (May 2-4, Boston) or IoT World 2017 (May 16-18, Santa Clara), please make sure to stop by our Asset Tracking Testbed Demo, see it run live, and understand better the contribution each partner has been making to the testbed.

Using MQTT-SN over BLE with the BBC micro:bit

The micro:bit is one of the best IoT prototyping platforms I’ve come across in the past few months.

The main MCU is a Nordic nRF51822 with 16K RAM and 256K Flash. A Freescale KL26Z is used for conveniently implementing a USB interface as well as a mass storage driver so as deploying code onto the micro:bit is as simple as directly copying a .hex file over USB (if your familiar with the mbed ecosystem, this will sound familiar :-)).

The board is packed with all the typical sensors and actuators you need for prototyping an IoT solution: accelerometer, compass, push buttons, an LED matrix, … What’s really cool, is the built-in BLE support, combined with the battery connector, making it really easy to have a tetherless, low-power 1, IoT testing device.

So how does one take the micro:bit and turn it into an IoT device? Since there is no Internet connectivity, you need to rely on some kind of gateway to bridge the constrained device that is the micro:bit to the Internet. You can of course implement your own protocol to do just that, but then you have to basically reimplement the wheel. That’s the reason why I thought the micro:bit would be ideal to experiment with MQTT-SN.

You can jump directly to the video tutorial at the end of the post, and come back later for more in-depth reading.

What is MQTT-SN and why you should care

If I were to over simplify things, I would just say that MQTT-SN (which stands for “MQTT for Sensor Networks”, by the way) is an adaptation of the MQTT protocol to deal with constrained devices, both from a footprint/complexity standpoint, and to adapt to the fact constrained devices may not have TCP/IP support.

MQTT-SN is designed so as to make the packets as small as possible. An example is the fact that an MQTT-SN client registers the topic(s) it wishes to us against the  server, this way further PUBLISH or SUBSCRIBE exchanges only have to deal with a 2-byte long ID, as opposed to a possibly very long UTF-8 string.

Like I said before, you really don’t want to reimplement your own protocol, and using MQTT-SN just makes lot of sense since it bridges very naturally to good ol’ MQTT.

Setting up an MQTT-SN client on the micro:bit

The MQTT-SN supports the BLE UARTService from Nordic, that essentially mimics a classical UART by means of two BLE characteristics, for RX and TX. This is what we’ll use as our communication channel.

The Eclipse Paho project provides an MQTT-SN embedded library that turns out to be really easy to use. It allows you to serialize and deserialize MQTT-SN packets, the only remaining thing to do is for you to effectively transmit them (send or receive) over your communication channel – BLE UART in our case.

In order to show you how simple the library is to use, here’s an example of how you would issue a CONNECT:

MQTTSNPacket_connectData options = MQTTSNPacket_connectData_initializer;
options.clientID.cstring = microbit_friendly_name();
int len = MQTTSNSerialize_connect(buf, buflen, &options);
int rc = transport_sendPacketBuffer(buf, len);

/* wait for connack */
rc = MQTTSNPacket_read(buf, buflen, transport_getdata);
if (rc == MQTTSN_CONNACK)
{
    int connack_rc = -1;

    if (MQTTSNDeserialize_connack(&connack_rc, buf, buflen) != 1 || connack_rc != 0)
    {
        return -1;
    }
    else {
        // CONNECTION OK - continue
    }
} else {
    return -1;
}

Now what’s behind the transport_sendPacketBuffer and transport_getdata functions? You’ve guess correctly, this is where either send or read a buffer to/from the BLE UART.
Using the micro:bit UART service API, the code for transport_getdata is indeed very straightforward:

int transport_getdata(unsigned char* buf, int count)
{
    int rc = uart->read(buf, count, ASYNC);
    return rc;
}

You can find the complete code for publishing the micro:bit acceloremeter data over BLE on my Github. Note that for the sake of simplifying things, I’ve disabled Bluetooth pairing so as connecting to a BLE/MQTT-SN gateway just works out of the box.

MQTT-SN gateway

There are a few MQTT-SN gateways available out there, and you should feel free to use the one that floats your boat. Some (most?) MQTT-SN gateways will also behave as regular MQTT brokers so you won’t necessarily have to bridge the MQTT-SN devices to MQTT strictly speaking, but rather directly use the gateway as your MQTT broker.
For my tests, I’ve been pretty happy with RSMB, an Eclipse Paho component, that you can get from Github.

The README of the project is pretty complete and you should be able to have your RSMB broker compiled in no time. The default configuration file for RSMB should be named broker.cfg (you can specify a different configuration file on the command line, of course).
Below is an example of the configuration file so as RSMB behaves as both a good ol’ MQTT broker, but also an MQTT-SN gateway, bridged to iot.eclipse.org’s MQTT sandbox broker. Note that in my example I only care about publishing messages, so the bridge is configured in out mode, meaning that messages only flow from my MQTT-SN devices to iot.eclipse.org, and not the other way around. Your mileage may vary if you also want your MQTT-SN devices to be able to subscribe to message, in which case the bridging mode should be set to both

# will show you packets being sent and received
trace_output protocol

# MQTT listener
listener 1883 INADDR_ANY mqtt

# MQTT-S listener
listener 1884 INADDR_ANY mqtts

# QoS 2 MQTT-S bridge
connection mqtts
  protocol mqtt
  address 198.41.30.241:1883
  topic # out

Bridging the BLE device(s) to the MQTT-SN gateway

Now there is still one missing piece, right? We need some piece of software for forwarding the messages coming from the BLE link, to the MQTT-SN gateway.

I’ve adapted an existing Node.js application that does just that. For each BLE device that attaches to it, it creates a UDP socket to the MQTT-SN gateway, and transparently routes packets back and forth. When the micro:bit “publishes” an MQTT-SN packet, it is just as if it were directly talking to the MQTT-SN gateway.

The overall architecture is as follows:

Note that it would be more elegant (and also avoid some nasty bugs, actually 2) to leverage MQTT-SN’s encapsulation mechanism so as to make the bridge even more straightforward, and not have to maintain one UDP socket per BLE device. To quote the MQTT-SN specification:

The forwarder simply encapsulates the MQTT-SN frames it receives on the wireless side and forwards them unchanged to the GW; in the opposite direction, it decapsulates the frames it receives from the gateway and sends them to the clients, unchanged too.

Unfortunately RSMB does not support encapsulated packets at this point, but you can rely on this fork if you want to use encapsulation: https://github.com/MichalFoksa/rsmb.

Visualizing the data: mqtt-spy to the rescue!

Like in my previous article about Android Things, I used mqtt-spy to visualize the data coming from the sensors.

Note that publishing sensor data in JSON might not be the best idea in production: the MTU of a BLE packet is just 20 bytes. Those extra curly braces, commas, and double quotes are as many bytes you won’t be able to use for your MQTT payload. You may want to look at something like CBOR for creating small, yet typed, binary payloads.
However, JSON is of course pretty convenient since there’s a plethora of libraries out there that will allow you to easily manipulate the data…

Using mqtt-spy, it’s very easy to visualize the values we’re collecting from the accelerometer of the micro:bit, either in “raw” form, or on a chart, using mqtt-spy’s ability to parse JSON payloads.

Video tutorial and wrap-up

I’ve wanted to give MQTT-SN a try for a long time now, and I’m really happy I took the time to do so. All in all, I would summarize my findings as follow:

  • The Eclipse Paho MQTT-SN embedded client just works! Similarly to the MQTT embedded client, it is very easy to take it and port it to your embedded device, and no matter what actual transport layer you are using (Bluetooth, Zigbee, UDP, …), you essentially just have to provide an implementation of “transport_read” and “transport_write”.
  • You may want to be careful when doing things like “UART over BLE”. The main point of BLE is that it’s been designed to be really low-power, so if you tend to overly communicate or to remain paired with the gateway all the time, you will likely kill your battery in no time!
  • The NRF5x series from Nordic is very widely available on the market, so it would be really interesting to run a similar MQTT-SN stack on other devices than the micro:bit, therefore demonstrating how it truly enables interoperability. If you build something like this, I really want to hear from you!
  • Although it’s true that there are not quite as many MQTT-SN libraries and gateways available out there as there are for MQTT, the protocol is pretty straightforward and that shouldn’t be preventing you from giving it a try!

 

Notes:

  1. You should keep in mind that the micro:bit, like other similar boards, is meant to be a prototyping platform, and for example having the KL26Z core taking core of the USB controller might not be ideal battery-wise, if you only care about doing tetherless BLE communications.
  2. RSMB expects the first packet received on an incoming UDP connection to be a CONNECT packet. If the bridge forwards everything to the gateway transparently, that may not always be the case. If, instead, it takes care of encapsulating all MQTT-SN packets properly, that means you know need only one UDP socket from your BLE/UDP bridge to the gateway)