Blazing with Phoenix: Project Structure

In this article, we'll be using Phoenix, a web framework written in the Elixir programming language. If you're not familiar with these tools, that's fine too! We're going to talk about some general concepts of software design applied to web development which I'm sure you'll enjoy thinking about.

Introduction

Most new Phoenix developers, especially ones coming from other web frameworks like Ruby on Rails, take some time to wrap their head around how to properly structure business logic (the M in MVC) in a functional way, so I'm here to propose a layering convention that will keep your project organized in a sane manner while also abiding to some core principles of Domain Driven Design that are so elegantly encouraged by Phoenix itself.

It's important to note that the conventions laid out here are focused on optimizing larger codebases, so if you have a small project, following the patterns set by the Phoenix generators is completely fine and will make you more productive. It's always best to start with simple abstractions and refactor as the project evolves.

In order to balance all of this conceptual talk, let's get our hands dirty with some code. We'll be building a fun little RPG based on a side-project of mine called MOBA, which you can check out to see the patterns described here applied in a more elaborate way.

Starting with mix phx.new rpg, we get the following structure:

├── _build
├── assets
├── config
├── deps
├── lib
│   └── rpg
│   └── rpg.ex
│   └── rpg_web
│   └── rpg_web.ex
├── priv
└── test

From the get-go, Phoenix generates our project with a major layer separation: the rpg folder which is where our business logic will live, and the rpg_web folder with all of the intricasies necessary to actually expose our application to the Web via controllers, views, templates, routing, etc.

First iteration: a single top-level domain

Keeping this central idea of explicit layering in mind, let's code the first iteration of the game which will be to simply list and create its main resource, the Hero:

Rpg.create_hero(attrs)
Rpg.list_heroes()
Rpg.list_enabled_heroes_by_level(level)

That is how our public API will be called by the Web layer. Now, let's dive in the rpg folder to see how we can structure our business logic with a simple convention focused on developer productivity. For every database table, I suggest having 3 supporting modules: a Schema, a Query and a Service. Applying that to our Hero resource, we would have:

├── lib
│   └── rpg
│       └── hero.ex (schema)
│       └── hero_query.ex (query)
│       └── heroes.ex (service)
│   └── rpg.ex
│   └── rpg_web
│   └── rpg_web.ex

Let's start by coding our Schema:

# lib/rpg/hero.ex
defmodule Rpg.Hero do
  schema "heroes" do
    field :level, :integer
    field :is_enabled, :boolean
    field :gold, :integer
  end

  def changeset(hero, attrs) do
    hero
    |> cast(attrs, [:level, :is_enabled, :gold])
  end
end

Schemas have a simple and strict ruleset: they should only have the actual schema definition and the changeset functions, which map external values to that definition. It's tempting to start throwing business logic here because this feels like a Model, but resist the urge, we'll get to the business logic in a second.

Moving on to the Query module:

# lib/rpg/hero_query.ex
defmodule Rpg.HeroQuery do
  import Ecto.Query

  def filter_by_level(query, level) do
    from hero in query, where: hero.level == ˆlevel
  end

  def filter_by_enabled(query) do
    from hero in query, where: hero.is_enabled == true
  end

  def order_by_level(query) do
    from hero in query, order_by: [desc: hero.level]
  end
end

Here, we should aim to have only functions that return and receive composable Ecto query structs. This allows us to very clearly describe our queries by chaining them together in our Service module, which we will define now:

# lib/rpg/heroes.ex
defmodule Rpg.Heroes do
  alias Rpg.{Repo, Hero, HeroQuery}

  def create(attrs) do
    Hero.changeset(%Hero{}, attrs)
    |> Repo.insert()
  end

  def list do
    Hero
    |> HeroQuery.order_by_level()
    |> Repo.all()
  end

  def list_enabled_with_level(level) do
    Hero
    |> HeroQuery.filter_by_enabled()
    |> HeroQuery.filter_by_level(level)
    |> Repo.all()
  end
end

The Service module is the one responsible for the bulk of our business logic. All Repo operations should be placed here as well as any other code that exclusively manipulates its resource, Hero.

Finally, to glue all of these 3 modules, we have the top-level domain:

# lib/rpg.ex

defmodule Rpg do
  alias Rpg.Heroes

  def create_hero(attrs \\ %{}) do 
    Heroes.create(attrs)
  end

  def list_all_heroes do 
    Heroes.list()
  end

  def list_enabled_heroes_by_level(level \\ 1) do
    Heroes.list_enabled_by_level(level)
  end
end

Functions in the top-level domain should either defer to child services directly like we're doing here, or orchestrate work between multiple child services, which we'll do in a bit.

If you were to add another database table for, say, Items that a Hero can buy, you would add functions to this same top-level domain, like so:

# lib/rpg.ex
defmodule Rpg do
  alias Rpg.{Heroes. Items}

  def create_hero(attrs \\ %{}) do # ...
  def list_all_heroes do #...
  def list_enabled_heroes_by_level(level \\ 1) do #...

  def buy_item(item, hero) do
    Items.buy(item, hero)
  end

  def sell_item(item, hero) do
    Items.sell(item, hero)
  end
end

Note that even though we are passing in a hero struct as an argument to both item functions, conceptually we are not manipulating just heroes anymore, so having an exclusive Service to handle all Item related functions keeps the code organized and most importantly, avoids bloating the Heroes service as a sort of God module.

Moving on, let's jump to the Web layer to see how we will use all of this, we'll wire up a basic route to a HeroController which will call our newly created Rpg functions.

# lib/rpg_web/router.ex
resources "/heroes", RpgWeb.HeroController, only: [:index, :create]

# lib/rpg_web/controllers/hero_controller.ex
defmodule RpgWeb.HeroController do
  use RpgWeb, :controller

  def index(conn, %{"level" => level}) do
    heroes = Rpg.list_enabled_heroes_by_level(level)
    render(conn, "index.html", heroes: heroes)
  end

  def index(conn, _params) do
    heroes = Rpg.list_all_heroes()
    render(conn, "index.html", heroes: heroes)
  end

  def create(conn, %{"hero" => hero_params}) do
    case Rpg.create_hero(hero_params) do
      {:ok, hero} -> 
        conn |> redirect(to: hero_path(conn, :index))
      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "index.html", changeset: changeset)
    end
  end
end

Through our layering, no implementation details were leaked to the controller so that it doesn't know how we fetch or create our heroes, heck, it doesn't even know we have a database. All of this was exposed through a public API which we can test in isolation and access through means other than the Web layer, like iex.

Ok, so for a first iteration, using a single top-level domain (Rpg) was fine. But what about when we start adding more functionality that doesn't really have to do with gameplay, like user registration, an admin panel or maybe even a payment system? Throwing everything under the Rpg domain would undo all of our layering efforts, so we need go one level deeper.

Second iteration: multiple top-level domains

In most real-world applications, you will have multiple top-level domains to represent different contexts. To showcase this, let's add user registration and chat functionalities to our game that will live under the Accounts domain and move all of our existing functionality to the Gameplay domain:

├── lib
│   └── rpg
│       └── gameplay.ex
│       └── accounts.ex
│       └── gameplay
│           └── hero.ex
│           └── hero_query.ex
│           └── heroes.ex
│           └── item.ex
│           └── items.ex
│       └── accounts
│           └── user.ex
│           └── users.ex
│           └── message.ex
│           └── messages.ex
│   └── rpg.ex
│   └── rpg_web
│   └── rpg_web.ex

Translating this to code, we would have the following:

# lib/rpg/accounts.ex
defmodule Rpg.Accounts do
  alias Rpg.Accounts.{Users, Messages}

  def create_user(attrs) do
    Users.create(attrs)
  end
  def set_current_hero(user, hero) do 
    Users.set_current_hero(user, hero)
  end
  def create_message(attrs, user) do 
    Messages.create(attrs, user)
  end
end

# lib/rpg/gameplay.ex 
defmodule Rpg.Gameplay do
  alias Rpg.Gameplay.{Heroes. Items}
  alias Rpg.Accounts

  def create_hero(attrs) do 
    attrs
    |> Heroes.create()
    |> Items.equip_initial()    
  end

  def list_heroes, do: Heroes.list()
  def list_enabled_heroes_by_level(level), do: #...
  def buy_item(hero, item), do: Items.buy(hero, item)
  def sell_item(hero, item), do: Items.sell(hero, item)
end

# lib/rpg.ex
defmodule Rpg do
  alias Rpg.{Gameplay, Accounts}

  def create_current_hero(attrs, user)
    hero = Gameplay.create_hero(attrs)
    user = Accounts.set_current_hero(user, hero)
    {hero, user}
  end
end

There are a few things to unpack here so let's go by topics:

Users and Messages

We have a whole new context called Accounts which we can use to deal with requirements that are unrelated to gameplay, like user registration and chat.
Once a User registers, he will be able to send Messages, create Heroes or set an existing Hero as his current one.

Orchestrating multiple services inside Gameplay

We've added an extra feature to our Hero creation process where an initial item is equipped to help the player out a little bit. Notice how equip_initial is not exposed publicly like buy_item and sell_item because it shouldn't be used anywhere outside of the hero creation process. We have also isolated each part of the process to their own service: Heroes.create worries only about returning us a new Hero while Items.equip_initial knows only how to equip an existing hero with a specific item, and this is all beautifully laid out on a descriptive chain -- you immediately know what's going on just by looking at it.

Orchestrating multiple domains inside Rpg

The same technique is then used for our app-level domain, Rpg. After a user creates a Hero, that newly created Hero should be assigned as the user's current hero, but managing Users is part of another domain, so we ask the parent domain, in this case Rpg, to orchestrate work between its children for us. Gameplay deals with returning a hero, Accounts deals with updating the user. Layers.

So what's the public API? App-level context or top-level domains?

It's up to you if you would like to make the app-level Rpg context your only public API. The downside of this is that you would have a lot of functions that just defer to either Gameplay or Accounts and on a larger app that can get very repetitive, so I personally prefer to provide public access to the top-level domains (Gameplay and Accounts), accessing the app-level context (Rpg) only for functions that need to cross to other top-level domains (like previously demonstrated) or for other app-wide uses like instrumentation, constants or helpers.

Conclusion

Layering a project like this can definitely be a little intimidating as it requires discipline from all collaborators to not break encapsulation by taking shortcuts, but the benefits of having high cohesion and low coupling between layers make your app much more maintainable, testable and robust. New functionality is also easier to add because a strong, future-proof convention has been put in place.

If you'd like to see these patterns applied in a real application, I invite you again to check out MOBA, a project I recently open-sourced that is looking for collaborators, so if building games sounds like fun to you, get involved :)

EDIT: An excellent discussion about project structuring is happening over at this topic on ElixirForum, I highly recommend reading it if you enjoyed the article.