Rust: First Thoughts

My third child, Maximus, was born this Spring, and thanks to a generous parental leave policy I’ve had the summer off to spend with family and friends, and to poke at various tech projects and those with friends (Thanks WaPo! Recently one of these projects presented the opportunity to learn some Rust.

I come to Rust having spent most of my career writing Java or (Java/Type)script. Many moons ago in high school and university I did C and C++ but have happily forgotten most of that. I’ve also recently built some small projects in Go, which is the most interesting and common comparand for Rust (a nice comparison can be found here).

Rust happened to be extremely well suited for the task I had: speeding up some CPU bound geospatial JavaScript code. Porting this to Rust took execution time from about 5 minutes on my 4 year old Macbook Pro to 30s without any algorithmic improvements. Dropping in some parallelism knocked it down once again to about 10s. A 30x speed improvement is not to be ignored!

The Good

Rust has a modern dev tooling may seem par for the course for a modern language. Dependency management, a code formatter, and unit testing are all built into the standard tooling. Rust here has borrowed a number of important features from languages and ecosystems like JavaScript and Go. What’s remarkable is that Rust gives you this first class tooling in a language that compiles down to native code. Given that Rust’s primary competitors are C and C++, this is a huge asset. Dependency management in anything non-trivial in C or C++ is monstrous. Getting C-like performance without the arcane dependency management is an enormous boon.

Rust’s indebtedness to functional languages like Haskell and OCaml make it quite simple to write code in a functional style. Immutability is a first class concern, and we get a full set of generic functions for operating on collections (map, filter,fold a.k.a. Rust’s reduce). This means the day to day experience of writing Rust is not unlike writing functionally inflected Typescript (my high-level language of choice at the moment). I write lots map and fold transformations, and the language provides robust support for fallible and nullable operations through the Result and Option enums. For instance, say I want to iterate over a list of objects and create a HashMap. In Typescript, that would probably look like this:

interface Block {
    id: string,
    x: number,
    y: number
}

const blocks: Block[] = [ /* ... */]

const blockMap = blocks.reduce((acc, block) => {
    acc[block.id] = block
    return acc
})

In Rust, you can do the same:

struct Block {
    id: &str,
    x: f64,
    y: f64
}

let mut blocks: Vec<_> = Vec::<Block>::new();
// push some blocks...

const blockMap: HashMap<String, Block> = blocks.fold(|mut acc, block| 
    acc.push(block.id.clone(), block);
);

Rust’s generic return implementation of the collect function even allow you to do this:

// map to a tuple, then collect into a HashMap
const blockMap: HashMap<String, Block> = blocks.map(|block| 
    (block.id, block)
).collect();

Go makes it much harder to write this kind of code, particularly in an immutable fashion. The same code in Go would look like this:

blockMap := make(map[string]Block)
for _, block in blocks {
    blockMap[block.id] = block
}

This is nice and concise, but you have to create the map and then mutate it. In Rust, by contrast, data is presumed immutable unless you mark it as mut. Writing code in Go that minimizes mutation is ugly and not idiomatic (for example, see this post). And Go’s lack of generics means map, filter, and reduce or fold are not really a thing in Go. In practice this means writing a lot more for loops manually and building up slices or maps.

With Rust, thread-safe code become much easier to write, which follows from Rust’s core principle of ownership (In short, you can only have one mutable reference to any given piece of data). Simple map and reduce operations can be parallelized with great ease by dropping in Rayon. For other use cases you can wrap your code in an Arc (Atomic reference counter) and proceed a bit more manually. It’s still possible to shoot your self in the foot, but Rust protects you from many mistakes at compile time. I was able to parallelize some geospatial operations and graph traversal in a period of hours thanks to Rust’s robust tools for concurrency and parallelism.

Go also has robust concurrency primitives built around go functions and channels. Go enforces less at compile time, however. For instance, the default map in Go is not thread safe. But the Go compiler will happily let you mutate a map within in go function. You’ll just likely hit a panic at runtime (at least according to this book you need channels to update a map in parallel).

Of course most application level code is IO bound. Neither Go nor Rust make things as easy as Promise.all in JavaScript. But hopefully this will get easier as stable async and await continue to trickle out through the Rust ecosystem.

The Hard

Rust has a reputation for being hard to learn, and to an extent this is true. Learning to deal with the borrow checker is hard. Rust’s borrow checker is the part of the compiler that enforces memory safety. In short, you can’t have multiple mutable references to the same data. Lots of things you do without thinking in a language with a garbage collector become harder (passing a list to a function for instance: do I need reference to a vector? Or a reference to a vector of references? does anything need to be mutated? Ahh!). You end up with a lot of statements like this from the compiler:

The upshot of all this is great: memory and thread safety largely enforced by the compiler without a garbage collector. That Rust delivers that is, quite frankly, amazing. But it does force the programmer to think about memory and references and makes certain data modeling patterns nigh impossible (double linked lists or anything really with circular references). The compiler enforces good habits but the virtue is hard at first, especially when you’re used to being proficient quickly in a new language.

Go is useful as a comparand here: Go’s minimal syntax and set of distinctives are quite easy for an experienced developer to pick up. You can go from no Go to building a functional app in a few weeks if you’ve already learned a few languages before. There will be relatively few surprises or headaches. You won’t be a master, but you’ll be productive quickly. Rust’s learning curve is steeper.

On the other hand, we shouldn’t overstate Rust’s weirdness. If you’re mostly writing application level code, rather than libraries, writing Rust feels a lot like writing functionally inflected Typescript. Their type systems diverge, but the everyday experience is much closer than one might expect, especially when considering that Rust compiles down to low level machine code. Rust is vastly more pleasant to work in than C.

My other big initial headache with Rust was learning the Rust idioms for conversion between types. Rust has a generics system built on traits that is extremely powerful, and the idiomatic way to convert between types is to use the From and TryFrom traits. Here’s an example of converting between two types of geometry:

let geojson_str = r#"
{
    "type": "Feature",
    "properties": {
        "name": "Firestone Grill"
    },
    "geometry": {
        "type": "Point",
        "coordinates": [-120.66029,35.2812]
    }
}
"#;
let geojson = geojson_str.parse::<GeoJson>().unwrap();
let geometry: geo::Geometry<f64> = geojson
    .geometry
    .unwrap()
    .try_into()
    .unwrap();

let geos_geometry: geos::Geometry = geometry.try_into().unwrap();

// use Rust's geos bindings to do something interesting...

The hard part of this is discoverability. I’m a big fan of “autocomplete as documentation.” It’s really nice like to be able to type geometry. and see what I can convert a type into. But generic methods like this lead to a lot of trial and error until you learn how to make sense of the code docs. It’s even worse when you’re dealing with enums, since then you have to sort out whether the enum or the different enumerated items are valid to convert. The Rust community is well aware of the problem here: I think the main solution is for projects to have full code samples illustrating conversions, not simple a reference to the traits.

A minor annoyance around readability. I rather like Go’s idiom of “keep the happy path on the left.” Rust’s enumerations make this much harder to do. With idiomatic Rust you end up having to read a lot like Lisp, from the inside out. The ? operator in Rust 2018 makes this easier, but you have to produce custom error types to use in a function or method that can error in more than one way.

A final point to note is the ecosystem. Rust’s ecosystem isn’t as mature as Go’s or (of course) JavaScript’s. You’re much less likely to have mature api clients for that service you need (even AWS’s are still in beta). On the other hand, there are mature libraries for doing web apps and some areas where Rust is definitely better than Go: geospatial processing being a crucial one for me, web assembly being another. Rust’s popularity is only increasing and the broader developer community has largely settled on Rust as a replacement to C and C++. The creator of Node is writing its spiritual successor, Deno, in Rust, for instance, and recently said on a podcast that he’d never start another C project).

The need for more libraries is also an opportunity for an aspiring open source dev: there’s tons of interesting work to do and my experience has been that the Rust community is extremely friendly and welcoming. Pick a project you like and ask how you can help!

Links

• Why Rust is hard: https://vorner.github.io/difficult.html
• Geospatial Libraries in Rust: https://georust.org/