Arduino graphics using web tricks

Working on the Arduino terminal emulator implementation required me to get very comfortable with protocol parsing and low-level graphics output code.

A TTY has to accept and process regular text and special character commands over serial, and show the results on screen (in this case a 1×1 inch TFT driven by the ILI9163C chipset) using low-level display chip instructions. This requires non-trivial logic, and at first I was feeling pretty reluctant about coding it. But my web developer experience, of all things, helped a lot.

People have already written screen controller driver code for my display’s chipset and of course there is Adafruit GFX to help out with pushing pixels to screen. The Adafruit library even has a drawChar command that prints out text:

// character display using Adafruit_GFX
tft->drawChar(cursor_x, cursor_y, 'A', 0xFFFF, 0x0000, 1);
// cursor bar
tft->fillRect(cursor_x, cursor_y + CHAR_HEIGHT - 1, CHAR_WIDTH, 1, 0xFFFF);

However, using a generic graphics library turned out to be not very performant, and low working memory on Arduino Uno required a more custom approach. E.g. the Uno has 2kb of memory, so storing a 32×16 character buffer with attributes would already take up half of that, and that is still quite small for a usable console experience. Hence, I wanted to use chip features like hardware scroll.

I also wanted flexibility to experiment with techniques like sub-pixel rendering, and just in general have runway to keep adding support for extended TTY commands. Finally, part of my goal was to allow porting the code to other displays and even other MCUs.

To support all of that I needed to pay attention to code architecture.

Reactive Rendering

I decided to use a reactive rendering approach: similar to how frameworks like React, Angular and Vue update webpage content really fast while keeping the application code maintainable.

This is the simplified pseudo-code for the main loop of the emulator:

struct term_state state; // current terminal state
while (1) {
if (data_available()) {
// parse TTY command
// update term_state fields
}
    render(&state); // idempotent render function
}

The render function is just called continuously, even when there is no new data. This is performant because render state is cached and change detection helps re-render just the needed bits.

For example, here is terminal state that tracks cursor position:

struct term_state {
int16_t cursor_col, cursor_row;
    // ... etc, etc
};

The renderer maintains its own “shadow state” that represents the last displayed state:

struct rendered_state {
int16_t cursor_col, cursor_row;
};

Every render invocation then compares the current terminal state with the last rendered version before doing anything slow: e.g. only if there is a difference between the two will it embark on a rectangle paint operation:

// detect change in cursor state
if (
rendered_state.cursor_col != term_state.cursor_col ||
rendered_state.cursor_row != term_state.cursor_row
) {
// clear cursor rectangle
clear_rectangle(rendered_state.cursor_col, rendered_state.cursor_row);
    // show new cursor rectangle
draw_rectangle(term_state.cursor_col, term_state.cursor_row);
    // save new rendered state
rendered_state.cursor_col = term_state.cursor_col;
rendered_state.cursor_row = term_state.cursor_row;
}

This is not too groundbreaking, of course, but it helps clean up the code a lot. The reason why this makes such a difference is because the parser code can then be fully blocking.

Since the renderer “contract” is extremely simple and the render() function is idempotent and can be called at any point in the sketch, the parser can call it at any point – mid-way through receiving a stream of bytes, etc.

This also helps keep display-independent abstract terminal state code separate from chip-specific logic. Draw commands are in one spot rather than scattered through the rest of the code. Chip-specific optimization hacks are still possible but do not influence code structure outside of the display routines.

And here are some initial results:

Hello world!

This approach has been working out really well so far: cursor animation, screen scrolling and swappable character rendering routine (to e.g. try out different size fonts, subpixel rendering and even anti-aliasing). It was interesting to see a web development technique cleanly port over to embedded real-time code.

And here is the aforementioned Arduino terminal emulator source code on GitHub.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s