For a few years now, on and off, I've been working on an unofficial port of the Elm language to WebAssembly. It's not production ready but I'm at the stage now where I have some good working demos, and things are taking shape.
My early attempts to get the SPA example running failed pretty badly. There were just too many compiler and core library bugs to be able to disentangle everything. I realised I needed to be patient and work on robustness. So I started by writing lots of unit tests for the low-level C code. You can run the tests in the browser.
And there was a specific part of the GC that was always throwing up hard-to-find bugs and was too complicated and hard to understand. It's the system that tracks references from the stack to the heap. I decided to just throw it out and keep trying different approaches until I found something that just felt stable and obvious and robust. I ended up rewriting it 4 times. The end result is something that's much more straightforward and a lot less scary, and I haven't had any bugs there since.
Once all that handwritten C code was solid, I needed to make sure the C generated from Elm was working properly. I found the source for the core library's unit tests and decided to port them into my project and add some of my own tests. You can run the tests in WebAssembly in your browser too. (Funnily enough, one of the biggest challenges was getting the Elm Test framework itself to run! The framework is more complex than the tests themselves. I still need to come back to the fuzzer tests!)
Then finally, with a bit more debugging, the SPA example came together. That has a lot of code in it and I figure if I can get it to run, I can get most things to run.
I haven't really focused on performance yet, but I did a quick analysis using Lighthouse from Chrome devtools. Already, without having optimised the JS/Wasm interface for each kernel module, it looks like the SPA example has similar performance to the official compiler's JS output with
uglify-js. That's a great place to be! There is a lot of encoding and decoding to optimise away to get more performance, which suggests the app code is running a lot faster in WebAssembly. And VirtualDom should get a lot faster.
So overall I think the project is in a pretty good place! But it's not ready for general use. Currently it's only set up to run on the "canned" demo apps in my repo, which all have their own build scripts with minor variations. And there's no solution for package management, so you can't have two apps with different versions of Kernel code.
The system breaks down into a few different areas:
Compiler: I chose to use C as an intermediate language. My forked Elm compiler generates C, then I use Emscripten/Clang to go from C to WebAssembly & JS. (I can also compile C to native code, which is a much better debugging experience.)
Garbage Collector: The Elm language expects its target platform to automatically manage memory for it. Browsers don't implement garbage collection for WebAssembly so I built a mark/sweep garbage collector in C. My measurements estimate it only adds 7kB of Wasm to the bundle.
Elm/JS interop: This is actually the toughest part of the project! Let's get into it a bit more below, because it isn't obvious.
"Web API" here means things like
XMLHttpRequest and so on. They are the interfaces between user code and the browser's internal functionality, often with an underlying implementation written in C++.
This is obviously a major drawback and the WebAssembly project has several proposals to work towards better host integration. One of the key issues is how to manage reference lifetimes - if a Wasm module is holding a reference to a DOM node, then it can't be garbage-collected. And if it is just a number in Wasm then it can be copied, which makes it hard to keep track of the copies. These issues are addressed in the GC proposal, which has been at "stage 1" since I first looked at it in 2018.
This turns out to be a huge deal!
Sure, WebAssembly "makes things fast"... but lots of serialising and deserialising "makes things slow"!
Obviously when the kernel libraries were designed, there was no slow "barrier" in between Elm code and Kernel code, or between two different parts of the Kernel code. I wonder if that constraint might have resulted in slightly different designs for some libraries? In practice, I want existing Elm apps to run in my system without modification, so I need all ported core libraries to at least retain the same APIs.
The two target languages are running in two isolated memory management zones. WebAssembly doesn't have access to the browser's main garbage collector.
Unfortunately there are cases where WebAssembly code may want to hold a long-lived reference to some JS value, and vice versa. We need to make sure we don't end up with stale references to values after they've been collected.
For example if we pass a value from external JS code through a port, it will appear in Elm as a
Probably the most important practical issues are usability and scalability. I'd like to make the build system general enough and usable enough for people to try out the system on their own apps. And, related to that, I'd like to come up with a more general and scalable way to deal with packages so that all apps don't have to use the same package versions! Maybe we can get some real apps running.
There's also lots of performance ideas I'd like to try out
- Set up a benchmark for some more focused performance work (perhaps this one)
- Port more kernel modules to C/Wasm. It looks like this could be one of the key performance drivers but there's a lot of code to port.
- Finish building a VirtualDom implementation in C using cache-friendly "data-oriented design" techniques and an arena allocator
- Remove the Emscripten layer and just use clang. Emscripten was handy to get going but it bloats code size a lot.
- Implement some optimisations that should make function calls faster