Category Archives: IoT

Quickly train your AI model with MXChip IoT DevKit & Edge Impulse

The first IoT device (albeit not so “smart”)? The Trojan Room coffee pot.

For the past few weeks, I’ve been spending some time digging into what some people call AIoT, Artificial Intelligence of Things. As often in the vast field of the Internet of Things, a lot of the technology that is powering it is not new. For example, the term Machine Learning actually dates back to 1959(!), and surely we didn’t wait for IoT to become a thing to connect devices to the Internet, right?

In the next few blog posts, I want to share part of my journey into AIoT, and in particular I will try to help you understand how you can:

  • Quickly and efficiently train an AI model that uses sensor data ;
  • Run an AI model with very limited processing power (think MCU) ;
  • Remotely operate your TinyML* solution, i.e. evolve from AI to AIoT.

* TinyML: the ability to run a neural network model at an energy cost of below 1 mW.

Pete Warden (@petewarden),
TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers

Simplifying data capture and model training

According to Wikipedia,supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs.

As an example, you may want to use input data in the form of vibration information (that you can measure using, for example, an accelerometer) to predict when a bearing is starting to wear out.

You will build a model (think: a mathematical function on steroids!) that will be able to look at say 1 second of vibration information (the input) and tell you what the vibration corresponds to (the output – for example: “bearing OK” / “bearing worn out”). For your model to be accurate, you will “teach” it how to best correlate the inputs to the outputs, by providing it with a training dataset. For this example, this would be a few minutes/hours worth of vibration data, together with the associated label (i.e., the expected outcome).

Adding some AI into your IoT project will often follow a similar pattern:

  1. Capture and label sensor data coming from your actual “thing” ;
  2. Design a neural network classifier, including the steps potentially needed to process the signal (ex. filter, extract frequency characteristics, etc.) ;
  3. Train and test a model ;
  4. Export a model to use it in your application.

All those steps might not be anything out of the ordinary for people with a background in data science, but for a vast majority—including yours truly!—this is just too big of a task. Luckily, there are quite a few great tools out there that can help you get from zero to having a pretty good model, even if you have close to zero skills in neural networks!

Enter Edge Impulse. Edge Impulse provides a pretty complete set of tools and libraries that provides a user-friendly (read: no need to be a data scientist) way to:

They have great tutorials based on an STM32 developer kit, but since I didn’t have one at hand when initially looking at their solution, I created a quick tool for capturing accelerometer and gyroscope data from my MXCHIP AZ3166 Developer Kit.

In order to build an accurate model, you will want to acquire tons of data points. As IoT devices are often pretty constrained, you often need to be a bit creative in order to capture this data, as it’s likely your device won’t let you simply store megabytes worth of data on it, so you’ll need to somehow offload some of the data collection.

Edge Impulse exposes a set of APIs to minimize the number of manual steps needed to acquire the data you need to train your model:

  • The ingestion service is used to send new device data to Edge Impulse ;
  • The remote management service provides a way to remotely trigger the acquisition of data from a device.

As indicated in the Edge Impulse documentation, “devices can either connect directly to the remote management service over a WebSocket, or can connect through a proxy”. The WebSocket-based remote management protocol is not incredibly complex, but porting it on your IoT device might be overkill when in fact it is likely that you can simply use your computer as a proxy that will, on the one hand, receive sensor data from your IoT device, and on the other hand communicate with the Edge Impulse backend.

So how does it work in practice should you want to capture and label sensor data coming from your MXChip developer kit?

Custom MXChip firmware

You can directly head over to this GitHub repo and download a ready-to-use firmware that you can directly copy to your MXChip devkit. As soon as you have this firmware installed on your MXChip, its only purpose in life will be to dump on its serial interface the raw values acquired from its accelerometer and gyroscope sensors as fast as possible (~150 Hz). If you were to look at the serial output from your MXChip, you’d see tons of traces similar to this:

…
[67,-24,1031,1820,-2800,-70]
[68,-24,1030,1820,-2730,-70]
[68,-24,1030,1820,-2730,-70]
[68,-23,1030,1820,-2730,-70]
[68,-24,1030,1820,-2800,-70]
[68,-24,1030,1820,-2800,-70]
[68,-24,1031,1820,-2800,-70]
[69,-22,1030,1820,-2730,-70]
[69,-22,1030,1820,-2800,-70]
…

There would probably be tons of better options to expose the sensor data over serial more elegantly or efficiently (ex. Firmata, binary encoding such as CBOR, etc.), but I settled on something quick 🙂.

Serial bridge to Edge Impulse

To quickly feed sensor data into Edge Impulse, I’ve developed a very simple Node.js app that reads input from the serial port on the one hand and talks to the Edge Impulse remote management API on the other. As soon as you install and start the bridge (and assuming, of course, that you have an MXChip connected to your machine), you’ll be able to remotely trigger the acquisition of sensor data right from the Edge Impulse portal. You will need to create an Edge Impulse account and project.

npm install serial-edgeimpulse-remotemanager -g

The tool should be configured using the following environment variables:

  • EI_APIKEY: EdgeImpulse API key (ex. ei_e48a5402eb9ebeca5f2806447218a8765196f31ca0df798a6aa393b7165fad5fe’) for your project ;
  • EI_HMACKEY: EdgeImpulse HMAC key (ex. ‘f9ef9527860b28630245d3ef2020bd2f’) for your project ;
  • EI_DEVICETYPE: EdgeImpulse Device Type (ex. ‘MXChip’) ;
  • EI_DEVICEID: EdgeImpulse Device ID (ex. ‘mxchip001’) ;
  • SERIAL_PORT: Serial port (ex: ‘COM3’, ‘/dev/tty.usbmodem142303’, …).

Once all the environment variables have been set (you may declare them in a .env file), you can run the tool:

serial-edgeimpulse-remotemanager

From that point, your MXChip device will be accessible in your Edge Impulse project.

You can now very easily start capturing and labeling data, build & train a model based on this data, and even test the accuracy of your model once you’ve actually trained it.

In fact, let’s check the end-to-end experience with the video tutorial below.

TensorFlow on an MCU?!

Now that we’ve trained a model that turns sensor data into meaningful insights, we’ll see in a future article how to run that very model directly on the MXChip. You didn’t think we were training that model just for fun, did you?

Don’t forget to subscribe to be notified when this follow-up article (as well as future ones!) comes out.

Is My IoT Device Secure? 7 Questions You Should Be Asking Yourself Today.

Getting security right when building IoT solutions can prove incredibly challenging.

Arguably, securing the “cloud” side of a solution is nothing new, and there are plenty of well-established practices and supporting frameworks that can help you do the right thing. However, when it comes to the “hardware” side—embedded devices that end up connected to public networks—this is still uncharted territory for most people.

In this article, we look at a few questions that you should be asking yourself as you’re building your connected product and provide some answers to help you make your next IoT device as secure as can be.

What is my device’s identity? Is it unforgeable?*

As you authenticate as an individual against a website or any form of online service, you are often relying on multi-factor authentication (MFA) to avoid impersonation and identity theft. Multi-factor authentication makes it virtually impossible for an attacker to forge your online identity, as they probably can’t get access to all the pieces of evidence needed to authenticate said website or service. For example, they may have gained knowledge of your password, they even stole your phone, but they don’t know your 6-digit pin, nor do they have your fingerprint. In short: they won’t be able to complete the sign-in process and impersonate you.

Can a similar mechanism be implemented for preventing identity theft of physical IoT devices?

Components of a Trusted Platform Module complying with the TPM version 1.2 standard (source: Wikipedia)

The answer revolves around the use of dedicated hardware such as Trusted Platform Modules (TPM). In a nutshell, such equipment holds cryptographic keys that are protected by hardware. The private key never leaves the module once it’s been stored in it, and the public key is then used to attest to the device’s identity. As the keys can’t be forged, impersonating the device is effectively equivalent to gaining physical access to it (this, of course, poses other security issues which we’ll talk about shortly!).

* I know, that’s two questions in one!

Where does my code run?

The term trusted computing base (TCB) refers to the set of hardware, firmware, and software components that are critical to a system’s security. There are several aspects of your IoT device that will play a role in ensuring the overall security of the system, from the TPM, mentioned earlier, to the kernel of your—potentially real-time—operating system.

As you design your IoT device, you must keep in mind that your trusted computing base should stay as small as possible. That way, you can minimize the surface of attack and reduce the risk of a bug (or feature!) in the TCB allowing an attacker to bypass security protections and deploying malicious payloads allowing them, for example, to steal valuable business data.

If you can, try to build on top of an operating system or real-time operating system that allows you to only ship and enable the features you really need. In any case, your application code should run outside of the TCB so that it can misbehave without affecting security.

Are my software components compartmentalized?

I find watchdog timers to be an exquisite concept—if a component of your embedded system stops responding, a full reset can be automatically triggered—but I also sometimes wonder if they should exist in the first place.

Historically, and often due to hardware constraints, embedded code has been designed as a rather monolithic blob, where a flaw or a breach in one component could potentially compromise the entire system, hence the need for watchdogs to mitigate the consequences of a software failure. Now, what happens if, beyond a harmless software bug causing a module (and your system) to hang, we talk about a breach in said module? Surely you don’t want an attacker to take control of your entire system due to a flaw in one of your software modules!

To mitigate this risk, a secure IoT device should allow for its various software components to be compartmentalized and should implement hardware-enforced boundaries between them. If you build on top of an RTOS that allows you to run your modules as independent tasks or processes, you’re probably in good shape!

What’s more, your device will ideally feature a security model that allows limiting what hardware resources each software module can access.

How wide is the net cast by my security mechanisms?

From distributed denial of service (DDoS) or ransomware attacks to using IoT devices as a way to get access to corporate networks, IoT devices can be a juicy target for hackers. You should assume that your devices will be hacked and that hackers will often be more creative and persistent than you might imagine.

Defense-in-depth originates in the military as a technique to delay an attacker by having resources (fortifications, military units, etc.) deployed not only at, but also well beyond, the frontline. That way, should an attacker manage to breach the frontline, they would still face several layers of defense as they penetrate deeper into the field.

Applied to IoT devices (and IT in general), defense-in-depth consists in implementing multiple layers of security controls to put several types of mitigation in front of each potential threat.

As an example, a combination of internal firewalls controlling access to the physical buses, network firewalls, and secure boot will greatly reduce the chances of your device being compromised. In the unlikely event of an attacker getting past the secure boot and being able to execute a malicious payload, the firewalls would prevent them from physically accessing the device’s peripherals or connecting to remote third-party servers.

Is authentication based on passwords or certificates?

Have you heard of shodan.io? It is a search engine that is essentially exposing IoT’s darkest secrets. And yes, ‘1234’ and ‘admin’ passwords are some of those secrets… Among other things, shodan.io allows you to find IoT and Internet devices online for which the default password may not have been changed. Ouch, right?

Beyond the obvious threat due to having publicly accessible devices rely on default credentials, password-based authentication can prove tricky to manage at scale.

For example, how do you recover from a stolen password, especially when it is used by thousands of devices deployed in the field? Certificate-based authentication, on the other hand, does not rely on shared secrets, and enables mutual authentication, limiting the risk of man-in-the-middle attacks.

Can my device be easily and automatically updated?

When it comes to IoT devices, the saying goes: “If you can’t update it, you can’t secure it.” Period.

“If you can’t update it, you can’t secure it.”

For a device to remain secure over time, it needs to allow for security updates to be rolled out as security threats evolve and attackers discover new attack vectors. Renewable device security includes, for example, the ability to roll out operating system updates (e.g., to fix zero-day vulnerabilities) or to deploy new certificates.

Ideally, you should also have hardware measures in place to prevent your device from being reverted to a known vulnerable state once it’s been updated.

Does my device report its failures?

There are many reasons why an IoT device may fail. It can, for example, be due to an attacker trying to take control of it by exploiting a vulnerability or trying to brute-force a password. It might also simply be due to the device hitting a corner case and switching to an undefined state that makes it vulnerable.

Whatever the reason, it is essential to be able to keep track of these failures across an entire fleet, so that they can not only be diagnosed and corrected but also be used to isolate and mitigate potential attack vectors before they are effectively exploited.

There are several solutions out there to help with error reporting. The Azure Security Center for IoT and its associated open-source agent are good examples of how to automatically collect and log incidents such as failed login attempts, a connection from an unusual IP address, etc.

Azure Security Center for IoT
Azure Security Center for IoT

Conclusion

Azure Sphere - Secure IoT Development Kit

This article is inspired by the paper “The Seven Properties of Highly Secure Devices” which is a must-read. You should check it out if you want to dive deeper into some of the best practices attached to each high-level security topic we listed in this article.

Finally, if you are like me, you probably like to get your hands dirty with actual devices and actual code to familiarize yourself with a new domain. If that is the case, I can only recommend an Azure Sphere development kit as a tool to grasp what it takes to build a secure IoT device.

This kit is based on an open-hardware reference design that essentially implements all the properties that are enumerated in this article and make an IoT device secure. It features a secure, connected microcontroller unit (MCU), a custom high-level Linux-based operating system, and a cloud-based security service that provides continuous, renewable security. There are many tutorials and code samples out there to help you get started with Azure Sphere and start familiarizing yourself with all the things you should have in mind as you build your next connected device.

Top 5 VS Code Extensions for IoT Developers

In just a few years, Visual Studio Code has conquered the hearts of a wide variety of developers. It took off very quickly in the web development communities, but it has now also become the IDE of choice for Java, Python, or C/C++ developers as well, whether they run Linux, MacOS, or Windows. In fact, in Stack Overflow’s most recent developer survey, VS Code is ranked at over 50% market share among the 90,000+ developers who responded.

Whether you’re just getting into IoT or whether you’ve been working on IoT solutions for some time already, you’ve probably realized that “full-stack developer” is a term that also often applies to IoT. You may very well be spending most of your days working on developing and testing the firmware of your connected embedded device in C. Still, once in a while, you may want to tune some Python scripts used for you build system, or use a command-line tool to check that your IoT backend services are up and running.

Rather than having to switch from one development environment or command line terminal to the other, I wouldn’t be surprised if, just like me, you’d be interested in doing most of your work without ever leaving your IDE.

In this article, we look at some essential VS Code extensions that will help you become a more productive IoT developer.

VS Code extension for Arduino

It’s been a very long time since I last opened the Arduino IDE on my computer. It is a great tool, especially for helping newcomers get started with the Arduino ecosystem, but it is lacking some key features for anyone interested in doing more than just blinking an LED or running basic programs. And now that more and more platforms are compatible with Arduino, from RISC-V developer kits such as HiFive1, to ESP32 or STM32 Nucleo family, there are even more reasons for looking for a better IDE for Arduino development.

The VS Code extension for Arduino is built on top of the official Arduino IDE—which you need to install once but will probably never open ever again—and provides you with all the features you’d expect to find in the classic IDE (e.g. browsing code samples or monitor your serial port).

The VS Code extension for Arduino in action.
The VS Code extension for Arduino in action.

What makes the extension particularly powerful in my opinion, is the fact it builds on top of the VS Code C/C++ tools to provide you with full-blown Intellisense and code navigation for your code, which proves to be very useful

I vividly remember the first time I put my hands on and soldered an Arduino-compatible board, circa 2010, at the TechShop Menlo Park. It’s been incredible to see the Arduino ecosystem grow over the years. Equally incredible is to think that until very recently, debugging a so-called sketch was reserved for the most adventurous programmers. If there was only one reason for you to try out the VS Code extension for Arduino, it has to be the fact it makes debugging Arduino programs so much easier (no more ‘Serial.println’ traces, yay!).

Behind the scenes, the extension leverages common debug interfaces such as CMSIS-DAP, JLink, and ST-Link. If your device already has an onboard debugging chip implementing one of these interfaces, you’re all set! If not, you will simply need to look at using an external connector that’s compatible with your chip.


PlatformIO IDE

Like I mentioned in the previous section, there are more and more platforms that tap into the Arduino paradigm, but there is, of course, more to embedded development than the Arduino ecosystem.

PlatformIO.org logo

PlatformIO originated as an open-source command-line tool to support IoT and embedded developers by providing a uniform mechanism for toolchain provisioning, library management, debugging, etc. It quickly evolved to integrate tightly with VS Code, and the PlatformIO IDE extension for VS Code is now one of the most popular ones on the Visual Studio Marketplace.

PlatformIO supports 30+ platforms (ex. Atmel AVR, Atmel SAM, ESP-32 and 8266, Kendryte K210, Freescale Kinetis, etc. ), 20+ frameworks (Arduino, ESP-IDF, Arm Mbed, Zephyr, …) and over 750 different boards! For each of these platforms, the extension will help you write your code (code completion, code navigation), manage your dependencies, build and debug, and interact with your device using the serial port monitor.

Another interesting feature is the ability to convert an existing Arduino project to the PlatformIO format, essentially making it much easier to share with your coworkers (and the world!), since it can then leverage PlatformIO’s advanced library management features. For example, it can automatically pull your 3rd party libraries solely based on the header files you’re including in your code.  


Azure IoT Tools

The Azure IoT Tools extension for VS Code is essentially an extension bundle that installs in one single click the Azure IoT Hub Toolkit, the IoT Edge extension, and the Device Workbench.

Azure IoT

As you look at connecting your devices to the cloud, Azure IoT Hub provides you with all you need to manage your devices, collect their telemetry and route it to consuming services, and more. Using the Azure IoT Hub extension, you can easily provision an IoT Hub instance in your Azure subscription, provision your devices, monitor the data they are sending, etc. all without having to leave your IDE!

If you are interested in using a container-based architecture for making your IoT gateways smart, chances are IoT Edge can help you! Thanks to the dedicated extension, you can easily build your custom IoT Edge modules, and deploy them to your edge devices connected to IoT Hub, either real ones or simulated ones running on your development machine.

Finally, the Device Workbench can help you get started very quickly with actual devices. It provides a set of tools to help with building your own IoT plug-and-play device, or simply to try out Azure IoT with an actual device, using one of the many examples bundled with the workbench.

What do I like the most with the Azure IoT Tools extension? Every few weeks, you get tons of awesome updates and new features, as the extension is actively developed.

By the way, if you don’t have an Azure subscription and want to get started with IoT on Azure, you can create a free trial account here!


Remote Development extension pack

IoT Development is much more than writing code for embedded devices. Frequently, you will find yourself in a situation where you want to interact with a folder that lives in a container on a remote edge gateway, or on a cloud server. You sure can use SSH and/or SCP to sync your local and remote development environments, but this can be pretty painful and error-prone.

The Remote Development extension pack allows you to open any folder in a container or on a remote machine and to then just use VS Code’s as if you were manipulating local resources.


REST Client

If you are like me, your go-to tool for testing REST APIs is probably Postman. It is indeed a great tool for creating and testing REST, SOAP, or GraphQL requests and it even allows you to save queries in the cloud and to share them with your colleagues. However, I recently found myself in a situation where I wanted to share some sample queries with people during a training session, and I didn’t want them to have to copy-paste unnecessarily from the training instructions to Postman; instead, I wanted the queries to be part of the actual training material!

The REST Client extension turns any file with an .http or .rest extension into an executable notebook, where you can very easily execute all the queries contained in it.

As you build an end-to-end IoT solution, it is more than likely that you will rely on 3rd party services along the way, and that you will interact with them using some form of REST API. For example, you may rely on a weather service as part of your predictive maintenance computations. Below is an example of how I shared with my students a few queries showing how to use the Azure Maps API to compute routes or render map tiles.

And now for the same queries (except for the subscription key which has been replaced by a real one 🙂) executed in real-time thanks to the REST Client extension:

How about you? Are there other VS Code extensions that you’ve found useful for your IoT projects? If so, I would love to hear about them in the comments.

You can also always find me on Twitter to continue the conversation.