Categories
Embedded IoT

5 Reasons Why I Dread Writing Embedded GUIs

A consequence of the massive adoption of Internet of Things technologies across all industries is an increasing need for embedded development skills. Yet, embedded development has historically been a pretty complex domain, and not something that one can add to their skillset overnight.

Luckily, over the last decade, silicon vendors have put a lot of effort into simplifying embedded development, especially for people with little to no experience in the domain. Communities such as Arduino and PlatformIO have also immensely contributed to providing easy-to-use tools and high-level libraries that can hide most of the scary details—yes, assembly, I’m looking at you!—of embedded programming while still allowing for professional applications to be written.

In my experience though, there is at least one area where things remain overly cumbersome: graphical user interface (GUI) development. Many applications require at least some kind of graphical user interface: the display might be small and monochrome, with hardly any buttons for the user to press, but it’s still a UI, eh?

I am sure many of you will relate: GUI development can be a lot of fun… until it isn’t!

In this article, I have compiled 5 reasons why I tend to not enjoy writing embedded GUI code so much anymore. And since you might not be interested in simply reading a rant, I am also sharing some tips and some of the tools I use to help keep GUI development enjoyable 🙂.

Hardware Integration & Portability

Most display devices out there come with sample code and drivers that will give you a head start in being able to at least display something.

But there is more to a GUI than just a screen, as an Interface is also made up of inputs, right? How about those push buttons, touch screen inputs, and other sensors in your system that may all participate in your interactions?

Image credit: Seeed Studio.

It might not seem like much, but properly handling simple hardware inputs such as buttons being pressed can be a lot of work when running on a constrained system, and you can quickly end up having to deal with complex timing or interrupt-management issues (see Event Management and “super-loop” section below for more). And as these often involve low-level programming, they tend to be pretty hardware-dependent and not easily portable.

A lot of embedded development is done using C, so, except for low-level bootstrapping code, embedded code can in theory be fairly portable. However, writing portable GUI code is a whole different story and unless you’re building on top of an existing framework such as LVGL or Azure RTOS GUIX, it requires a lot of effort to abstract all the hardware dependencies, even more so when trying to keep the performance optimal.

Of course, it is not always necessary (or possible) to have GUI code that’s 100% portable. However, in these times of global chip shortage, it can prove very handy to not have a hard dependency on a specific kind of micro-controller or LCD display.

Memory Management

Like I mentioned in the introduction, manipulating pixels can be a lot of fun—it really is! However, constrained systems have limited amounts of memory, and those pixels that you’re manipulating in your code can quickly add up to thousands of bytes of precious RAM and Flash storage.

Let’s take the example of a tiny 128×64 pixels monochrome display. As the screen only supports black & white, each pixel can be represented in memory using just a single bit, meaning a byte can hold up to 8 pixels—yay! But if you do the maths:

A 32x32px bitmap uses 128 bytes of memory!

That’s already 1KB of RAM, which is quite significant if your MCU only has, say, 16KB total. Interested in displaying a handful of 32×32px icons? That will be an additional 128 bytes for each of these tiny icons!

In short: your graphical user interface will likely eat up a lot of your memory, and you need to be extra clever to leave enough room for your actual application. As an example, a quick way to save on graphics memory is to double-check whether some of your raster graphics (ex. icons) can be replaced by a vector equivalent: surely it takes a lot less code and RAM to directly draw a simple 32x32px red square on the screen, instead of having it stored as a bitmap in memory.

Resource Management

It can be tricky to properly manage the various resources that make up a GUI project.

More specifically, and whether you are lucky enough to work with a graphics designer or not, your GUI mockups will likely consist of a variety of image files, icons, fonts, etc. However, in an embedded context, you typically can’t expect to be able to directly manipulate that nice transparent PNG file or TrueType font in your code! It first needs to be converted in a format that allows it to be manipulated in your embedded code.

If you are a seasoned embedded developer, I am sure that you (or someone at your company) have probably developed your very own macros and tools to help you streamline the conversion/optimization of your binary assets, but in my experience it’s always been a lot of tinkering, and one-off, quick and dirty, conversion scripts, which impact long-term maintainability. Add version control to the mix, and it becomes pretty hairy to keep your assets tidy at all times!

Event Handling & Performance

Graphical programming is by nature very much event-driven. It is therefore quite natural to expect embedded GUI code to look as follows (pseudo-code):

Button btnOK;
btnOK.onClick = btnOK_click;

btnOK_click = function() {
  // handle click on button btnOK
  // ...
}

As you can imagine, things are not always that simple…

Firstly, C, which is still the most used embedded programming language, is not exactly object-oriented. As a consequence, even if it’s perfectly possible to aim for a high-level API that looks like the above, there is a good chance you will find yourself juggling with error-prone function pointers whenever adding/updating an event handler to one of your UI elements.

Assuming that you have indeed found an elegant way to associate event handlers to the various pieces of your UI, you still need to implement some kind of event loop. Indeed, you must regularly process the events happening in your system (“button A pressed”, etc.) and dispatch them to the proper event handlers. A common pattern in embedded programming consists in doing so through a so-called super loop: the program is running an infinite loop that invokes each task the system needs to perform, ad infinitum.

int main() {
    setup();
    
    while (1) {
        read_sensors();
        refresh_ui();
        // etc.
    }
    
    /* program's execution will never reach here */
    return 0;
}

A benefit of this approach is that the execution flow remains pretty readable and straightforward, and it also avoids some potential headaches that may be induced by complex multi-threading or interrupt handling. However, any event handler running for too long (or crashing!) can compromise the performance and stability of your main application.

As embedded real-time operating systems such as FreeRTOS, or Azure RTOS ThreadX are becoming more popular, a more modern approach is to have the UI event loop run in a dedicated background task. The operating system can therefore ensure that this task, given its lower priority, will not compromise the performance of your main application.

An embedded GUI does not always need to be performant as in fast and responsive. However, it is considered a good practice to use embedded resources as efficiently as possible. Making sure that your GUI & app code are as performant as reasonably possible can potentially save you a lot of money as it means you can stick to using the smallest possible MCU for the task.

Tooling

Last but not least: tooling. To be honest, I have never been a big fan of designing graphical user interfaces using a WYSIWYG (What You See Is What You Get) approach. That being said, coding a graphical user interface that has more than just a couple of screens requires at least some tooling support, since most of the “boring” glue code can often be automatically generated.

What’s more, testing an embedded GUI can quickly become painful, as re-compiling your app and downloading it to your target can take quite some time. This can be very frustrating when you need to wait minutes to e.g test how that new fancy animation you coded looks like. 😒

Over the past few months, I have started to use Renode more often. It is a pretty complete open-source tool suite for emulating embedded hardware—including their display!—straight from your computer. In a future post, I plan on sharing more on how I started to use Renode for drastically shortening the “inner embedded development loop”, i.e. the time between making a code change and being able to test it live on your—emulated!—device.


I would be curious to hear about your experience, and your pain points when working with embedded GUIs. Let me know in the comments below!

Like I already mentioned, stay tuned for upcoming articles where I will be covering some of the tools and frameworks that I have started to use (and love) and that make my life SO much easier when it comes to GUI development!

If you enjoyed this article, don’t forget to subscribe to this blog to be notified of upcoming publications! And of course, you can also always find me on Twitter and Mastodon.

Categories
Eclipse IoT

Installing the Zulu open source Java Virtual Machine on Raspberry Pi

I’ve recently been playing with the early-access version of Zulu Embedded for ARM32. Zulu is an open-source and certified build of OpenJDK. It is a great alternative to the JRE from Oracle as it does not requires you to pay any licenses, and it is also a huge improvement over the OpenJDK build you can get off the Debian repositories, which is not really optimized for ARM (and that’s a euphemism! :smile:).

Before giving you more information on my experience running Eclipse IoT projects on Zulu in subsequent blog posts, here’s a quick tip for anyone interested in installing an alternate JVM on a Raspberry Pi (or pretty much any Linux-based environment, really), and switching between default JVMs easily.

Download the Zulu JVM

At the time of writing this blog post, Zulu for ARM32 is only available through an early access program. Once you’ve downloaded your Zulu archive, you need to unpack it somewhere on your system (in a zulu folder within our home directory, in this example). From the command-line, and while in the directory where you’ve downloaded Zulu:

mkdir ~/zulu
tar xvfpz ezre-1.8.0_60-8.9.0.6-cp1-eval-linux_aarch32.tar.gz -C ~/zulu

Add the Zulu JVM to the list of alternate VMs

The update-alternatives command-line utility allows to easily bind a given symbolic name to different commands. In our case, we want to update the symbolic link for /usr/bin/java

sudo update-alternatives --install /usr/bin/java java ~/zulu/ezre-1.8.0_60-8.9.0.7-cp3-eval-linux_aarch32/bin/java 100

Enable Zulu

sudo update-alternatives --config java

In the list of alternative Java version that’s proposed to you, select the one corresponding to Zulu by entering its ID and pressing .

And voila! Zulu is now your default Java Virtual Machine, you can check by executing the following command:

java -version
openjdk version "1.8.0_60-Zulu-Embedded"
OpenJDK Runtime Environment (build 1.8.0_60-Zulu-Embedded-EA3, profile compact1)
OpenJDK Minimal VM (build 25.60-b23, mixed mode, Evaluation)
Categories
Eclipse

Eclipse Day Toulouse 2012 – May 24, 2012 – Call for papersEclipse Day Toulouse 2012 – 24 mai 2012 – Appel à conférences

After the success of Eclipse Party 2011 last year, the Eclipse Foundation, Airbus, Obeo and Sierra Wireless invite you to Eclipse Day Toulouse, on May 24, 2012.

This year, we will have a full day event, with a particular focus on two topics for which Eclipse is an ecosystem of choice: embedded world, and Machine-to-Machine (M2M).

You should start thinking submitting your talk if you have interesting thoughts to share about:

  • Embedded & Modeling for embedded. Modeling technologies have a proven track record in terms of embedded and critical software development. Coupled with other tools and frameworks (ALM, requirements management, code analysis, …), they turn out to change the way software is being developed in the industry, and how it is being supported (usually for years, in e.g. the aircraft industry).
  • M2M & Internet of Things. According to analysts, there will literally be tens of billions of connected objects by the end of 2020! This proves to be a challenge in terms of new software engineering needs, of scalibility (Cloud computing and SaaS anyone? ;)), of management of these huge fleets of “objects” on the field… There are also many new use cases and business models around M2M and IoT that can, and will, benefit from Open Source ecosystems.

If you’d like to share an industrial use case, talk about a project you are working on, or anything related to these topics, you should submit your abstract by Apr. 15, 2012.

More information on the wiki page of the event…  and I am now looking forward to your proposals!

Photo credit: http://www.flickr.com/photos/benurs/5162576672Après le succès de l’Eclipse Party 2011, la Fondation Eclipse, Airbus, Obeo et Sierra Wireless vous invitent à participer à l’Eclipse Day Toulouse, le 24 mai 2012.

Cette année, l’évènement se déroulera sur une journée entière, et se focalisera plus particulièrement sur deux sujets pour lesquels Eclipse est une plateforme de premier choix : l’embarqué, et le Machine-to-Machine (M2M).
Aussi, nous vous invitons à soumettre dès aujourd’hui, vos propositions de sujets, en ciblant de préférence les thématiques suivantes:

  • Embarqué, et Modeling pour l’embarqué. Les approches MDA/MDD ont prouvé leur efficacité dans le domaine du développement de logiciel embarqué. Couplées à des outils de gestion des exigences, d’analyse de code, de simulation, … elles changent la donne dans le domaine du développement de logiciel industriel, et dans la manière dont ce logiciel est maintenu, souvent sur des périodes de plusieurs dizaines d’années.
  • M2M et Internet of Things. Les analystes s’accordent à dire que d’ici 2020, plusieurs dizaines de milliards d’objets seront connectés, plus ou moins indirectement, entre eux ! Cette perspective ouvre la porte à de nombreux défis en terme de développement, de scalabilité (qui a dit Cloud ? ;)), de gestion de flottes importantes “d’objets”… Ce sont aussi de nombreux business models jusqu’alors inaccessibles qui voient désormais le jour, et pour lesquels l’approche Open Source peut clairement rebattre les cartes.

Si vous souhaitez faire un retour d’expérience, présenter un projet sur lequel vous travaillez, ou quelque chose en relation avec ces sujets, nous attendons vos soumissions avant le 15 avril 2012.

Plus d’informations sur la page de wiki dédiée à l’évènement.

Crédit photo: http://www.flickr.com/photos/benurs/5162576672