Types as a Learning Tool with Crystal and Lucky

Why Crystal and Lucky?

It's been a while since I've taken even a moderately deep dive into a new language or framework. For the past few years I've been happily working on the backend using Elixir, Phoenix and Absinthe. Generally, this is a really awesome and productive environment and I quite enjoy it. However, on occasion I run into a nasty problem and after struggling with it for bit and solving the problem, I say, "Man, having a type system would have prevented that."

I've never used a backend web framework that had strong types and was internally debating what I should play with. I've always been intrigued by Rust, but I've never had that much fun dabbling in that language. It felt too low-level and syntactically busy. I'd gotten used to the terseness of Elixir and Ruby and didn't feel like going in that direction. I had read a bit on F# and it looks awesome, but wasn't sure there was a web framework that would make me happy (I have a real fear of .net).

Years ago, I had looked a little bit at Crystal, but I was afraid that taking Ruby and adding types would end up feeling clunky. Upon digging in a bit more, it seemed a lot smarter than that and looks to have some incredible performance possibilities. After discovering that there was also a well-documented and interesting looking web framework called Lucky, I was ready to give it a shot.

I'm not sure if this will be the start of a number of posts about this tech stack or just a one-off but I thought I'd share some of my initial findings as I unwrapped this new framework and language.

Caveats

First off, I'll admit that I'm learning these tools in a non-optimal way. Back when I first learned Ruby, I was a newbie at both Ruby and Rails and as I tried to learn both at once, I made a big mess and didn't learn either very well for a while. So I told myself that in the future, I'd first take a deep dive into the underlying language before I dove into a framework that uses it.

You can guess where I'm going here. I skimmed through half of a book on Crystal and want to write some code. But the thought of playing with exercises and building something with minimal functionality using only Crystal isn't very appealing right now. On the other hand, building a real but tiny web application with both a REST API and a basic UI seems super fun. So I'm breaking my rule and jumping head first into both Crystal and Lucky all at once. I know this won't go completely well but at least I acknowledge what I'm doing this time.

Getting Started

As with most frameworks these days, getting a sample application up and running is pretty simple. The Lucky framework has some great documentation and I simply followed their directions to get Crystal and Lucky installed and running.

As with most frameworks, Lucky has generators that will build you a basic project that you can modify. One added bonus with Lucky is the option to add authentication to the application out of the box. Since I knew I could easily spend days learning how to build login pages and authenticate securely, I took them up on this offer and run their wizard and answered yes at the prompt to add authentication code.

This worked perfectly and I quickly had an app with a signup form, a login form and a landing page for me as an authenticated user. And this all came without writing a single line of code.

After creating a user and checking that it was really in the database, it was time to try to write some code.

Exploring Lucky

I'm not much on jumping deep into documentation before I've explored a bit and beat my head against the wall a bit so I first started poking around in the src tree and trying to put some of the pieces together. Being familiar with a few MVC frameworks and noting that upon login, I was routed to a /me page, I decided to figure out how this URL got this page rendered.

Based on my experience with Rails and Phoenix, I first started to dig around and find the routes file that would get me from /me to a controller and eventually a view. This was my first big learning. With Lucky, there is no routes file where we store routing information for every page. Instead, Lucky ties the route and controller information in one file in the src/actions directory. That action looked like this.

# src/actions/me/show.cr
class Me::Show < BrowserAction
  get "/me" do
    html ShowPage
  end
end

This looked pretty self-explanatory. Instead of a routes file that told me that a certain path would route to a particular controller action, the Lucky action puts that all in one place. I think I like this as the dedicated routes files sometimes feel cluttered and messy to me as projects grow.

So, looking at this, it seems that the me action renders something named ShowPage. Let's next try to figure out where that lives.

A bit of poking around and a few grep commands brought me to: src/me/show_page.cr. So, it looks like what I normally think of as a controller is an action in Lucky. And what I think of as a view or template is a page in Lucky. That page looked like this.

# src/pages/me/show_page.cr
class Me::ShowPage < MainLayout
  def content
    h1 "This is your profile"
    h3 "Email:  #{@current_user.email}"
    helpful_tips
  end  

  private def helpful_tips
    h3 "Next, you may want to:"
    ul do
      li { link_to_authentication_guides }
      li "Modify this page: src/pages/me/show_page.cr"
      li "Change where you go after sign in: src/actions/home/index.cr"
    end  
  end  

  private def link_to_authentication_guides
    a "Check out the authentication guides",
      href: "https://luckyframework.org/guides/authentication"
  end  
end

This all looks reasonable. We seem to define a class that inherits from a layout page and define a function named content that renders the page. I also notice that Lucky seems to provide so nice Crystal functions to render HTML elements. This all feels a bit like Elm and makes me happy so far.

Now that I think I know my way around a Lucky project (while still reading very little documentation or studying Crystal syntax, let's try to build our first new page and see what happens.

Let's Build Something Now

Given that the project I built already had a user model, I decided that the first thing to build would be a page that listed all of the users. So, I'd be looking to get a list of email addresses for all users at the /users route.

A Basic Action and Page

I knew enough to get started here. I wanted to build an action for this new route. So I started by creating src/actions/users/index.cr. I could easily render a dummy page with.

# src/actions/users/index.cr
class Users::Index < BrowserAction
  get "/users" do
    html IndexPage
  end  
end

In order for this to work, I needed to build a page to render.

# src/pages/users/index_page.cr

class Users::IndexPage < MainLayout
  def content 
    h1 "Users"
  end
end

Passing Parameters from Action to Page

That worked just as expected. Now, I need to learn how to get data from the action into the page. A quick dig into the docs showed me that you can pass data to the page with optional parameters on the html function. So I first tried this with some hardcoded data.

# src/actions/users/index.cr

class Users::Index < BrowserAction
  get "/users" do
    html IndexPage, users: ["bill@example.com", "sophia@example.com"]
  end  
end

Now, we need to modify the page to accept and use these passed parameters. Based on a quick glance at the docs, I tried this.

# src/pages/users/index_page.cr

class Users::IndexPage < MainLayout
  needs users : Array(String)

  def content 
    h1 "Users"

    ul do
      users.each do |user|
        li user
      end  
    end  
  end
end

This works perfectly and shows that we can pass parameters. Now, let's try to put it all together and fetch real database records and render them.

Rendering with Real Database Data

Now we just have one final step. We want to fetch real users from the database, pass them from the action to the page and render the email for each of the passed users.

To get here, I first read up a bit on the ORM-type capabilities of Lucky. They have functionality that looks a lot like ActiveRecord in Rails.

Each model we build inherits from a parent that adds basic methods to perform basic queries.

# src/queries/user_query.cr

class UserQuery < User::BaseQuery
end

This little bit of OOP magic allows us to make a call like this to return all users (almost).

users = UserQuery.all

Let's modify our action to try this.

# src/actions/users/index.cr

class Users::Index < BrowserAction
  get "/users" do
    users = UserQuery.all

    html IndexPage, users: users
  end  
end

Then we want to modify our page to expect a new type and properly dereference to get the email field.

# src/pages/users/index_page.cr

class Users::IndexPage < MainLayout
  needs users : Array(User)

  def content
    h1 "Users"
    ul do
      users.each do |user|
        li user.email
      end  
    end  
  end  
end

This should all work, right? Instead we get a compile error.

Error: no overload matches 'Users::IndexPage.new', context: HTTP::Server::Context, users: UserQuery, current_user: User

After a bit of study, I realized that was a type mismatch. The page was expecting an Array(User) but I was passing in a UserQuery. It turns out this is because queries in Lucky are lazy and are not executed into they are used. So until we attempt to use that query (commonly with an each or map), no query is executed.

Given this, let's do something weird and try to force the query to happen in the action.

# src/actions/users/index.cr

class Users::Index < BrowserAction
  get "/users" do
    user_query = UserQuery.all
    users = user_query.map { |u| u }

    html IndexPage, users: users
  end  
end

Note that we use a map that looks like it does nothing to trigger the database query and this works.

But it doesn't quite feel right. Lucky uses lazy queries for a reason. Imagine a case where the page we are rendering may or may not use that data. We might want to only show the list of users to logged in users. We would end up performing a query that we'd throw away in that case.

So, why not pass that query into the page and let the each already on that page trigger the actual database access? Here's what I ended up with.

# src/actions/users/index.cr

class Users::Index < BrowserAction
  get "/users" do
    user_query = UserQuery.all

    html IndexPage, users: user_query
  end  
end

Note that above pass the query instead of the list of fetched users.

# src/pages/users/index_page.cr

class Users::IndexPage < MainLayout
  needs users : UserQuery

  def content
    h1 "Users"
    ul do
      users.each do |user|
        li user.email
      end  
    end  
  end  
end

And in the page, we change the expected type to be UserQuery instead of Array(User). We now have a working page with the performance bonus that we don't do the database query until the code on the page actually needs it.

Conclusions

When I dove into this project, I was hoping to learn a little about Crystal and Lucky. I expected that the type system might be more of a hindrance as I was initially learning with this small example. But I actually found the type system forced me to learn a bit more about how Lucky worked. So, to summarize the main things I learned with this exercise:

1. The controller part of the MVC framework is done with a Lucky action.

2. Router functionality is combined with the action instead of being in a standalone file.

3. The view functionality of the MVC framework is done with a Lucky page

4. Lucky provides an Elm-like functionality to render HTML elements programmatically.

5. Parameters can be passed from action to page and these are strongly typed.

6. Database queries are done with an ActiveRecord-like query language and are lazy. The database queries do not happen until they are actually needed.

7. Type errors are found at compile time so we don't have to wait until we run. our code to know it won't work.