loading...

Containerized builds for Rust on the ESP32

mtnmts profile image Matan Mates ・4 min read

What is the ESP32?

The ESP32 "is a series of low-cost, low-power system on a chip microcontrollers with integrated Wi-Fi and dual-mode Bluetooth", I've bought a few of these recently for personal projects, the very low power draw and low cost, paired with Wi-Fi and dual-mode Bluetooth capabilites make it extremely attractive for a wide range of projects.

ESP32 Chip

I've previously worked with the ESP32's predecessor, the ESP8266, both devices have a RISC CPU that runs an ISA called Xtensa.

This was a few years back and I was programming most of my embedded projects in C, Since then I fell in love with Rust and have been using it for a few projects, especially ones which are relatively close to hardware, such as programming an ESP32, so last week when I started looking programming the device I checked to see if I could target it with rust.

Compiling Rust for the Xtensa ISA

For this to work, LLVM would need to support an Xtensa backend, in LLVM a Frontend takes some higher level language such as C,Fortran, Ada or Rust and emits LLVM-IR AST, LLVM then has optimisers that work on LLVM-IR, and finally a Backend accepts LLVM-IR and emits architecture specific (x86_64/ARM/PowerPC/etc) instructions.

Unfortunately, LLVM upstream does not currently support an Xtensa ISA backend, so the rust compiler does not support Xtensa out of the box, fortunately there is a LLVM fork by Espressif implementing support for Xtensa, and thanks to the awesome work of Scott Mabin (MabezDev) there is a fork of the Rust compiler toolchain supporting Xtensa. With these two components already prepared all you need to do is build LLVM and the compiler and do a bit of environment setup and you're ready to go!

Clang Frontend/Backend design

There is a great blog post by Yoshinari Nomura explaining exactly what you need to do, while the process is relatively straightforward there are quite a few dependencies and a few pitfalls that made me spend a few hours getting it to work properly for me.

I work from a few different computers, and considering the time it takes to build LLVM and the compiler I decided to create a Docker image with the build environment.

Building for the ESP32 with Docker

I've built a Dockerfile that builds the environment (LLVM and the compiler) for out-of-the-box compilation.

a quick disclaimer about the image is that it is a local build I pushed to Docker Hub and it was not built by Docker Hub due to the build taking too long and timing out, you do not need to trust the image, and you can build the Dockerfile yourself (and possibly push it to Docker Hub yourself). be aware that pushed images that were not built by Docker Hub from the Dockerfile can essentially be anything.

You can now build your Rust ESP32 project with a single command if you have Docker installed, If you're looking for a baseline for your project you can use xtensa-rust-quickstart also by MabezDev, note that you do not need to use setenv inside the project as everything is already set up properly in the machine.

all you need to is docker run -v $PWD:/code mtnmts/rust-esp32

The image expects the project to be mounted at /code (your Cargo.toml should be at /code/Cargo.toml) inside the container.

The image is quite heavy, if you would like to create a slimmed down version you can fork it and remove some of the intermediate build artifacts (such as LLVM).

Containerized Build Demo
And that's it, Dockerfile is here if you're interested.

Serial Forwarding & Working with WSL2

My main desktop is running Windows, but i do most of my programming work in Linux under WSL2, I wanted to be able to program the ESP32 from within WSL2, to do this I forwarded the serial port to the WSL2 VM.

First, you will need socat for Windows, there is an unofficial build of the sources over at SourceForge if you don't want to build it yourself.

If you don't already know it, Find the serial port assigned to your device COM[1-9]

Device Manager Image, Finding the correct COM Port

Looking in "Device Manager" should be indicative

Next, run socat and set it to tunnel the COM port to TCP, do note that /dev/ttyS3 is not a mistake, even though there is no such device (or file system hierarchy) on Windows, this is the syntax that socat expects on Windows, the number (3 in my case) is the COM port number.

 socat -d TCP4-LISTEN:1337,reuseaddr,fork /dev/ttyS3

From there you can forward the port in many different ways to your WSL2/Linux instance, I use Tunnelier's S2C port forwarding for this, Putty can probably do this too.

esptool supports flashing over TCP, the syntax is --port socket://<host>:<port>

more information on remote serial ports programming for esptool can be found here.

Thanks

  • Everyone who worked on getting Rust on the ESP32 going, especially MabezDev, and to Yoshinari Nomura for his blog post explaining how to set up the environment

  • Espressif for creating the LLVM fork supporting your architecture!

Happy Hacking!

Anecdotes

  • First time I programmed the device I stupidly guessed I should flash to 0x0, that didn't work so I guessed again and flashed 0x1000 ,That wrote over the bootloader (yikes). Luckily I had the same board twice so I read the flash off the second one and re-flashed the first one, I highly recommend you backup your original flash!
  • (The correct flash offset ended up being 0x10000)

Posted on by:

mtnmts profile

Matan Mates

@mtnmts

Competitive CTF Player with team Pasten, Innovation at SentinelOne

Discussion

markdown guide
 

Hi Matan, thank you for the great article, very informative and precise.
I'm trying to develop esp-idf on WSL2 but unfortunately, as you also pointed out, serial peripherals are not supported yet. I'm struggling to find a way to flash my ESP using your strategy.
Could you please explain a little more in detail on how to set up Socat on windows and the steps to tunnel the serial port to a TCP one?

 

I tried to install socat on WSL1 (which has support to serial devices) but the only thing that works so far is the command "monitor". When I try to flash I get:

"A fatal error occurred: Failed to connect to ESP32: Invalid head of packet (0x30)"

I tried the same thing using Ser2net (my config: 1367:raw:600:/dev/ttyS3:115200 8DATABITS NONE 1STOPBIT) but I get:

"A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header"

However, monitor seems to work

 

The problem is port control, likely. This works well for me through rfc2217: matevarga.github.io/esp32/m5stack/...

 

Nice write up, thanks.

It'll be great if/when their LLVM fork makes it upstream (github.com/espressif/llvm-xtensa/i...). IIRC there was also some concern that Xtensa hadn't released up to date ISA docs.