My journey in to Crystal from Ruby

I've been working with Ruby and Rails now since 2006 back in the old Rails 1.1.6 days. I've seen it go through many different iterations, and it's been awesome each step of the way. Back in 2016, I came across crystal-lang. This looked like a really neat way to start learning about how compiled languages worked, so I thought I'd give it a try!

Fast forward a few years, and now I'm actually working on an app that will go in to production using Crystal. It took a while for some frameworks and ORMs to mature enough, but now there's a few great ones to use! The one I settled on (and by "I", I mean "we the team" I work with) is Lucky.

I just want to talk about my journey coming from Ruby and Rails over to using Crystal and Lucky

The pain of Ruby

The apps that I work on at work are really old legacy apps that have been migrated from PHP to Rails, and then upgraded through Rails 2 -> 5+. So the code base has seen tons of different iterations over the years. The database schema has also seen it's share of changes from fields existing to no longer being needed. On top of this, the team that has maintained these apps have changed a lot over the years. You see the remnants of the "clever" code, or the "meh, whatever" code, and of course the "OMG THE APP IS DOWN, PATCH PATCH PATCH!" code.

The beauty of ruby is that you can write some quick updates and things will be great. There's tons of tools that are a joy to use and don't require a doctorate in Math to get setup. I really love the moments when I write some code that I have no idea if it will work, but it looks like it should, and it does!

I'll admit, I do have a Software Engineering degree, but I really don't consider myself an amazing developer. I just don't think about problems in the same way that a lot of my colleagues do. I write my tests, and make sure I have code coverage like a good dev should, but I'll often hear "Why didn't you just do it like this?" and see an example that's 2 lines compared to my 20. I think to myself "oh yeah, I forgot you could do that...". For the most part, this is fine. Our apps work, they make money, users are mostly happy.

The issues are when I start seeing errors like Undefined method xxx for NilClass. This is a very common error for us because our data CAN NOT be trusted. I've recently started littering our codebase with stuff like thing&.do_stuff. We use a lot of hashes from our APIs too, so another thing that's littered all over looks like Hash(thing.dig(:a, :b, :c))[:d] or Array(thing).each. Basically typecasting just to avoid all the nil errors. Rollbar even posted this blog post about common errors. Maybe you've seen a few of these?

Now, are these bugs show stoppers? No. Any language you write in is going to have issues, and these issues will only increase based on how you wrote your code. I have other rails apps that don't have these sames issues. But let's break down a list of what started the journey to Crystal.

1. Tons of Nil errors
2. 10+ second load times
3. Massive cache setup using Varnish
4. Varnish is really far out of date, and not easily updatable
5. Accidentally cleared app cache and tanked the whole app for 2 hours while it re-primed
6. Typos! I can't spell initialize correctly the first time ever. Even that time took me 2 tries.
7. The number of times we've pushed code that broke one of the apps.
8. Running specs taking almost a minute
9. Clever code rabbit holes to find where an error is actually living.
10. Waiting on nokogiri to build so I can make sure the app is still up so I can go home.
11. Wrong version of rake was activated.
12. Circular Dependency error. (I still don't really understand this one).

Here comes crystal

I knew by using a compiled language, we would get speed. There's several ones out there we looked at. Rust, Go, and Elixir were on the table. Our team has "technically" 4 people, but only 3 of us really write code. 2 of us were familiar with Elixir, and liked it. Our fear though was once getting an app in to production, we would have no clue what to do for scaling, errors, etc... We would need to start with something small if we tried that. The other 2 were familiar with Go, and had some decent experience with it. This would mean I would have to play catch up if we went that route. With Rust, it was foreign to all 3 of us, so probably not a great choice.

When I saw Crystal a few years back, I loved that it was so close to Ruby syntax. This would allow all of us a huge boost in the learning curve. We decided to mess with it doing a few small pet projects and we quickly enjoyed writing it. It was a great feeling to see those undefined method for NilClass errors BEFORE we even booted the app. So at this point we got speed, and type checking. Just by using Crystal, half our issues we had with Ruby were gone.

1. Nil errors caught at compile time
2. App load time is 1.5 seconds down from 10+
3. No need for cache (just yet). This saves on some money too
4. No need for Varnish (see 3)
5. No cache clearing to do (yet)
6. Typos are caught during compilation, so I never commit typos
7. We still push some code that breaks the app, but it's not full "production" just yet, and this is getting less and less as we work out our new flow.
8. Running specs taking almost a second
9. There are still some rabbit holes with macros that are kind of annoying
10. Compilation takes some time, so deployment is about the same amount of time.
11. Haven't seen any version mismatch errors yet.
12. No circular dependency errors yet.

Using Lucky

Since we decided to roll with using Lucky, one if it's goals is to catch errors before it goes to production. Lucky really focuses hard on that fact too. It's probably the most frustrating thing in the world trying to run a single spec and constantly getting no method matches with type String, overloads are...., or getting the undefined method for compile time String | Nil. After the 20th error, you just want to flip your table and go make tacos for a living.

But then you finally get it. Your spec suite of 100 tests runs in 10ms, the app compiles and boots. Then HOLY SHIT! this thing is solid. It's this moment when you feel that pride of this thing isn't going anywhere. It's like hanging a shelf where your bolts are directly in the studs. You can do pull ups on this thing, and it's not budging. With rails, it was like a couple small screws. It held up just fine, as long as it wasn't misused. Unfortunately, you can't tell your users to not misuse your site. They're gonna be jerks.

Aside from the speed, and type safety, it's been nice to be able to contribute and really make a difference with these different shards like Lucky. Working with Crystal has given me a renewed sense of accomplishment as a developer. Unfortunately, it's not all sunshine and rainbows on this side either.

1. Crystal isn't 1.0 yet, so almost every update breaks your app.
2. Broken shards means lots of forking and PRs, or tons of patience while the shard maintainer gets it updated.
3. Missing shards... So many things we use with rails that we take for granted that just don't exist, or are production ready in crystal
4. Re-learning things. It's a new language and new environment, so it takes time.
5. Compatibility (see 2). The shard you want to use might work great on some random distro of Linux, but breaks on mac because the maintainer doesn't use mac.
6. Understanding errors. I've seen this index out of bounds error where the stack trace has ?? in place of line numbers a ton. That's one of those, just get up and take a break for a bit errors.
7. Docs, or the lack there of. Actually, this is starting to get better, but again, just going to take time since things are still new.

Conclusion

Overall, I'm happy with Crystal, and I'm super stoked that our team is going this path. It's going to take some time to get things working, but in the long run, I feel like it will be a great decision. In my next post, I'd like to talk about how LuckyRecord looks compared to ActiveRecord.