Test driving a Phoenix endpoint, part I

This story was originally posted by me on Medium on 2016-10-18, but moved here since I'm closing my Medium account.

I have worked part time professionally with Elixir for around a half year. Starting next month I will work with Elixir full time. And I can’t wait!

The project I’m working on is split into four applications, one of which is a REST api using Phoenix as the main framework.

I am very thrilled about Elixir, ExUnit, Ecto and a lot of other stuff in the Elixir ecosystem. But I have been a little careful not liking Phoenix too much.

Maybe because I burned myself when I jumped into Rails around 5 years ago. Now I think Rails and Active* is everything Ruby should not be. I hope Phoenix will develop differently and it certainly looks good right now.

But some of the example code attached to the Phoenix documentation is a bit alarming. I know. It is only example code and people shouldn’t copy-paste it into their own production code. But it could be exemplary examples.

As an example, take a look at this update action from the Phoenix documentation:

def update(conn, %{"id" => id, "user" => user_params}) do
  user = Repo.get!(User, id)
  changeset = User.changeset(user, user_params)

  case Repo.update(changeset) do
    {:ok, user} ->
      conn
      |> put_flash(:info, "User updated successfully.")
      |> redirect(to: user_path(conn, :show, user))
    {:error, changeset} ->
      render(conn, "edit.html", user: user, changeset: changeset)
  end
end

In my opinion, controllers should be doing web stuff and delegate all non-web stuff to other places in the codebase. Because the non-web stuff tend to grow in complexity and keeping it inside the controller is a mix of concerns.

This example also shows the default behaviour of defining a changeset in the schema module. In the User module in this case.

Changesets should be defined in a specific context. Validations, accepted attributes and so on might change depending on which endpoint, cron-job or other code I’m in.

Why? Because reusability, testability and maintainability.

Test driving

This is the first part of a mini-series of blog posts. I estimate it will consist of 2–4 blog posts. This post is about testing the controller web logic and separating it from the non-web logic.

Later posts will address other issues in testing such as writing a mock for an external service.

Edit: Second post about mocks can be found here.

I will take you through the journey of test-driving a single Phoenix endpoint. The endpoint lives in POST /users and is responsible for inviting a new user to our system.

When we’re done, we have build an endpoint that will create an inactive user and send an email to the user with instructions on how to activate.

When test-driving the code, it usually end up being reusable and maintainable, simply because we write the consumer of the code first.

Let’s get started!

We will create a Phoenix application from scratch. We start by using the phoenix.new mix task to create our application called MyApp. Since we are building a REST api, we opt-out of using HTML and Brunch stuff:

$ mix phoenix.new --no-html --no-brunch my_app

First test

We start by building the create-user-part. Later we will build the send-activation-email-part.

We can go one of two ways. Inside-out or outside-in. Start with the REST endpoint or start with the inner most layer of the application, which might be a database table or something else. Since we don’t know which layers come after the controller yet, we choose outside in: Starting in the controller and work our way in to the inner layers.

The first thing we do before anything else, is writing a test. We will use the first test as a kind of integration test, testing all the layers of the application and the integration between them. Later we will add unit tests to test edge case behaviours of the controller and other pieces of code.

Here is our first test:

defmodule MyApp.UserController.InviteTest do
  use MyApp.ConnCase

  test "inviting a user responds with the new user" do
    conn =
      build_conn
      |> post("/users", email: "alice@example.com")

    body = conn |> response(201) |> Poison.decode!

    assert body["id"] > 0
    assert body["email"] == "alice@example.com"
  end
end

We will run this test. But not yet.

First we should make some assumptions about what is going to fail. If we are wrong in our assumption, we have either written our test wrong or we have learned something new.

Looking at the test, we first post some payload to /users which should be ok. Then we assert that the response has status 201, which will not be true, because the endpoint does not exist yet. Let’s give it a try:

$ mix test
...

  1) test inviting a user responds with the new user (MyApp.UserController.InviteTest)
     test/controllers/user_controller/invite_test.exs:4
     ** (RuntimeError) expected response with status 201, got: 404, with body:
     {"errors":{"detail":"Page not found"}}
     stacktrace:
       (phoenix) lib/phoenix/test/conn_test.ex:362: Phoenix.ConnTest.response/2
       test/controllers/user_controller/invite_test.exs:9: (test)



Finished in 0.08 seconds
4 tests, 1 failure

Randomized with seed 448141

Surely enough. Our tests reveals that we get a 404 when posting our payload. The test result almost tells us what to do next, which is one of the things I like about test driven development. The next thing we should do is to make sure our application understands a POST /users which means we should add something to the router.

We edit the web/router.ex file:

Adding the route

defmodule MyApp.Router do
  use MyApp.Web, :router

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", MyApp do
    pipe_through :api

    resources "/users", UserController, only: [:create]
  end
end

We added the users resource and run the tests again:

$ mix test
Compiling 3 files (.ex)
...

  1) test inviting a user responds with the new user (MyApp.UserController.InviteTest)
     test/controllers/user_controller/invite_test.exs:4
     ** (UndefinedFunctionError) function MyApp.UserController.init/1 is undefined (module MyApp.UserController is not available)
     stacktrace:
       MyApp.UserController.init(:create)
       (my_app) web/router.ex:1: anonymous fn/1 in MyApp.Router.match_route/4
       (my_app) lib/phoenix/router.ex:261: MyApp.Router.dispatch/2
       (my_app) web/router.ex:1: MyApp.Router.do_call/2
       (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.phoenix_pipeline/1
       (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.call/2
       (phoenix) lib/phoenix/test/conn_test.ex:224: Phoenix.ConnTest.dispatch/5
       test/controllers/user_controller/invite_test.exs:7: (test)



Finished in 0.06 seconds
4 tests, 1 failure

Randomized with seed 999258

The test tells us that we need the UserController, which was expected, since we reference it from the router, but did not yet create it.

Adding the controller

defmodule MyApp.UserController do
  use MyApp.Web, :controller

  def create(conn, _params) do
    conn
    |> resp(201, "{}")
  end
end

We just return status 201 and an empty JSON object.

Again, we make assumptions about where the test will fail and run the test again. First, let’s review the test:

body = conn |> response(201) |> Poison.decode!

    assert body["id"] > 0
    assert body["email"] == "alice@example.com"

Take a moment to figure out where it will blow up next before we run the test. Done? Now for the test output:

$ mix test
Compiling 1 file (.ex)
Generated my_app app
...

  1) test inviting a user responds with the new user (MyApp.UserController.InviteTest)
     test/controllers/user_controller/invite_test.exs:4
     Assertion with == failed
     code: body["email"] == "alice@example.com"
     lhs:  nil
     rhs:  "alice@example.com"
     stacktrace:
       test/controllers/user_controller/invite_test.exs:12: (test)



Finished in 0.06 seconds
4 tests, 1 failure

Randomized with seed 773707

Ok, this was suprising! At least to me. My guess was that it would fail on body["id"] > 0 since we don’t return any id. But it fails on the email assertion on the next line. What happened?

In Elixir everything is comparable to everything else. Not every comparison makes sense, but be sure to remember that we can compare any two data types.

Looking in the Elixir documentation, we see this order for comparing non-same types:

number < atom < reference < function < port < pid < tuple < map < list < bitstring

Since nil is an atom and 0 is a number, our assertion passes. Which means our test is wrong. Let’s fix it:

 body = conn |> response(201) |> Poison.decode!

    assert body["id"] |> is_integer
    assert body["id"] > 0
    assert body["email"] == "alice@example.com"

Running the test again now shows that our expectations are correct (the expectation that the assertion is wrong) and we should return an id and an email to satisfy the test.

But what should we return? We have no id and no user row in the database. We don’t even have a User schema. The next step is to actually create a user. The logic of user-creation is something we should delegate to another module, since we want to be able to reuse and unit test it.

We update the controller:

defmodule MyApp.UserController do
  use MyApp.Web, :controller

  def create(conn, params) do
    {:ok, user} = MyApp.Users.Invite.call(params)

    conn
    |> resp(201, "{}")
  end
end

This, of course, makes the test fail because MyApp.Users.Invite does not exist yet. We will leave the controller test for now, while we create the invitation logic. After that we will return here and make it green.

Creating the invitation logic

Our efforts so far has lead us to the clarification that we need a Users.Invite module which will contain the logic related to inviting a user.

We start on a blank page since we’re building a new thing now. So just as before we will start by writing a failing test and take it from there. When this test is green, we will return to the controller test.

defmodule MyApp.Users.InviteTest do
  use MyApp.ModelCase

  alias MyApp.Users.Invite
  alias MyApp.User

  test "it creates a user" do
    {:ok, user} = Invite.call(%{email: "alice@example.com"})

    assert user.id > 0
    assert user.email == "alice@example.com"

    assert (from u in User, select: count(u.id)) |> Repo.one == 1
  end
end

Running the test tells us that we should create MyApp.Users.Invite:

$ mix test test/contexts


  1) test it creates a user (MyApp.Users.InviteTest)
     test/contexts/users/invite_test.exs:6
     ** (UndefinedFunctionError) function MyApp.Users.Invite.call/1 is undefined (module MyApp.Users.Invite is not available)
     stacktrace:
       MyApp.Users.Invite.call(%{email: "alice@example.com"})
       test/contexts/users/invite_test.exs:7: (test)



Finished in 0.04 seconds
1 test, 1 failure

Randomized with seed 973477

So we will create it:

defmodule MyApp.Users.Invite do

  alias MyApp.User
  alias MyApp.Repo

  @create_params [:email]

  def call(%{email: email}) do
    email
    |> build_changeset
    |> create_user
  end

  defp build_changeset(email) do
    params = %{email: email}

    %User{}
    |> Ecto.Changeset.cast(params, @create_params)
    |> Ecto.Changeset.unique_constraint(:email)
  end

  defp create_user(changeset) do
    changeset
    |> Repo.insert
  end
end

This code is pretty simple for now. It will 1) build a changeset using only the email and 2) save the changeset.

Where does this fail when we run the test again? I have an assumption. Let’s see:

$ mix test test/contexts
Compiling 1 file (.ex)

== Compilation error on file lib/my_app/contexts/users/invite.ex ==
** (CompileError) lib/my_app/contexts/users/invite.ex:17: MyApp.User.__struct__/1 is undefined, cannot expand struct MyApp.User
    (stdlib) lists.erl:1354: :lists.mapfoldl/3
    (stdlib) lists.erl:1354: :lists.mapfoldl/3

Right. It fails because we don’t have a user schema yet. So we create the user schema and migration.

Creating the schema and database table

First we simply create the User schema:

defmodule MyApp.User do
  use MyApp.Web, :model

  schema "users" do
    field :email, :string

    timestamps
  end
end

Notice the lack of a changeset function. When using the Phoenix generators, a changeset function is created for us in the schema modules. We don’t need nor want this.

We defined the changeset in the Users.Invite module. This module contains all the logic we need to create and invite a user. Hence the changeset related to creating a user this way is also placed there. In other places we might need another changeset definition for creating a user.

Defining the changeset where we use it allows us to maintain the changeset definition with confidence and without fear of breaking a random part of our application.

We run the test again:

$ mix test test/contexts     


  1) test it creates a user (MyApp.Users.InviteTest)
     test/contexts/users/invite_test.exs:7
     ** (Postgrex.Error) ERROR (undefined_table): relation "users" does not exist
     stacktrace:
       (ecto) lib/ecto/adapters/sql.ex:463: Ecto.Adapters.SQL.struct/6
       (ecto) lib/ecto/repo/schema.ex:397: Ecto.Repo.Schema.apply/4
       (ecto) lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
       test/contexts/users/invite_test.exs:8: (test)



Finished in 0.04 seconds
1 test, 1 failure

Randomized with seed 711320

Just as we thought! We need a database table to fill the user into. We create a migration for this.

$ mix ecto.gen.migration add_users_table

defmodule MyApp.Repo.Migrations.AddUsersTable do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :string

      timestamps
    end

    create index(:users, [:email])
  end
end

$ mix ecto.migrate

We migrate only for the dev environment, since Phoenix by default is set up to migrate the test environment before every test run with these lines in the mix.exs file:

defp aliases do
  [...more aliases here],
  "test": ["ecto.create --quiet", "ecto.migrate", "test"]]
end

It’s time to run the test for Users.Invite again.

$ mix test test/contexts                           
.

Finished in 0.05 seconds
1 test, 0 failures

Randomized with seed 163288

Yay! First green test. We now have a reusable component in our application and it is working. This is a big deal.

We can now revisit our failing controller test and make that green.

Going back the the controller test

Since we started with the test for Users.Invite we only ran that test every time we tested. We now go back to run all tests again, since we only expect one test to fail.

Let’s run all tests and see how the controller test fails:

$ mix test              
...

  1) test inviting a user responds with the new user (MyApp.UserController.InviteTest)
     test/controllers/user_controller/invite_test.exs:4
     ** (FunctionClauseError) no function clause matching in MyApp.Users.Invite.call/1
     stacktrace:
       (my_app) lib/my_app/contexts/users/invite.ex:8: MyApp.Users.Invite.call(%{"email" => "alice@example.com"})
       (my_app) web/controllers/user_controller.ex:5: MyApp.UserController.create/2
       (my_app) web/controllers/user_controller.ex:1: MyApp.UserController.action/2
       (my_app) web/controllers/user_controller.ex:1: MyApp.UserController.phoenix_controller_pipeline/2
       (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.instrument/4
       (my_app) lib/phoenix/router.ex:261: MyApp.Router.dispatch/2
       (my_app) web/router.ex:1: MyApp.Router.do_call/2
       (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.phoenix_pipeline/1
       (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.call/2
       (phoenix) lib/phoenix/test/conn_test.ex:224: Phoenix.ConnTest.dispatch/5
       test/controllers/user_controller/invite_test.exs:7: (test)

.

Finished in 0.1 seconds
5 tests, 1 failure

Randomized with seed 633478

Ok? This was a surprise again. A function clause not matching in Users.Invite. After reviewing the code we see that from the controller we pass in %{"email" => "someemail"} but Users.Invite expects the email key to be an atom.

The actual mistake was that we passed the params directly to the underlaying logic without first extracting the things we need. We will fix this now.

We change

{:ok, user} = MyApp.Users.Invite.call(params)

to:

attrs = %{email: params["email"]}
{:ok, user} = MyApp.Users.Invite.call(attrs)

This will fix the problems. But note that normally we should parse the input params with an Ecto.Schema or similar.

When we only have one parameter this will be fine. If we’re dealing with just a few more parameters this method of manually parsing them will lead to bugs at some point. Parsing the params should be a job of it’s own, but for the sake of this blog post let’s keep it simple as above.

We run the test again:

$ mix test
Compiling 1 file (.ex)
warning: variable user is unused
  web/controllers/user_controller.ex:6

....

  1) test inviting a user responds with the new user (MyApp.UserController.InviteTest)
     test/controllers/user_controller/invite_test.exs:4
     Expected truthy, got false
     code: body["id"] |> is_integer
     stacktrace:
       test/controllers/user_controller/invite_test.exs:11: (test)



Finished in 0.08 seconds
5 tests, 1 failure

Randomized with seed 847631

Yes! It now yells about the missing id again, which was what we hoped would fail. This is because we still return an empty JSON object from the controller even though we create a user. We even get a warning that our user is unused.

Serializing an internal data type to a json object is also a job of it’s own, and many great libraries exists for this single purpose. Again, for the sake of the essence in this blog post, we simply just create a map and serialize it with Poison.

This is the updated controller:

defmodule MyApp.UserController do
  use MyApp.Web, :controller

  def create(conn, params) do
    attrs = %{email: params["email"]}
    {:ok, user} = MyApp.Users.Invite.call(attrs)

    {:ok, body} = %{
      id: user.id,
      email: user.email
    }
    |> Poison.encode

    conn
    |> resp(201, body)
  end
end

We run the tests again:

$ mix test
Compiling 1 file (.ex)
.....

Finished in 0.08 seconds
5 tests, 0 failures

Randomized with seed 174980

Finally! For the first time since we started the blog post, all tests are green. We have now made a REST endpoint that can create a user. Good job!

Edge cases

We have, however, only tested the “normal case” of the application. What should happen if we send an obviously incorrect email address like "Alice and Bob"? What if we try to create a user with an email that exists on another user? What if the email is blank? Not submitted at all?

A good REST API knows how to respond to all kinds of wierd input. A meaningful HTTP status code and a detailed error message can be a life-saver for an api user getting a response back from an invalid request.

We will of course test drive edge cases just like we did with the “normal case”.

We decide that we only want to handle one edge case for the sake of the blog post. In a real production site we should handle all the weird edge case scenarios we can think of.

An invalid email should give 400: {"message": "email is invalid”}.

Check for missing email

Up until now we have gone through the testing very thoroughly. This process is usually pretty quick in practice, but very verbose in writing.

We will now go up one level and dicuss which tests should be written and where certain parts of the logic belong. For this section, I will not post the entire test output for each test run.

When checking for invalid email, the actual check should be in the Users.Invite logic, since we also want to check for invalid emails if we reuse the module somewhere else. But the HTTP status and error message belongs in the controller, since that is both web specific and specific to that controller.

We start by adding the test to the controller:

 test "invalid email gives a 400 response" do
    conn =
      build_conn
      |> post("/users", email: "Alice and Bob")

    body = conn |> response(400) |> Poison.decode!

    assert body["message"] == "email is invalid"
  end

This test fails because the code just creates the user with invalid email.

We realize that the actual check should happen in Users.Invite so we write a similar test for that.

  test "it handles invalid email" do
    {:error, :invalid_email} = Invite.call(%{email: "Alice and Bob"})
  end

Notice how much smaller this test is compared to the controller test. This is a sign that this is the right place to do edge case testing. If we want to test a lot of invalid and valid email addresses, we should add them to this test and just leave the single test in the controller.

Running the test tells us that the logic in Users.Invite accepts "Alice and Bob" as a valid email. We fix this by ensuring that the email has at least one @. This is intentionally simple, since we can’t possibly write a good validator for whatever RFC document describes what a valid email looks like.

As an example "Alice and Bob"@example.com is a valid email address. We just check for the @ because leaving it out is usually a sign that someone misread a form somewhere and entered a name instead of an email address. Our real email validation happens when we send out the activation email. If the user can get the email, the email is valid.

We fix the logic:

  def call(%{email: email}) do
    with :ok <- validate_email(email) do
      email
      |> build_changeset
      |> create_user
    end
  end

  #...

  defp validate_email(email) do
    case String.match?(email, ~r/@/) do
      true -> :ok
      false -> {:error, :invalid_email}
    end
  end

We wrap the user creation in a with :ok <- validate_email(email). This tells Elixir to go on if validate_email returns :ok, but return the result of that function call if it returns another value than :ok. This makes it easy to step out early froma series of function calls.

The new test now passes, but the controller test is still red. Let’s fix that. This is the entire controller after the fix:

defmodule MyApp.UserController do
  use MyApp.Web, :controller

  def create(conn, params) do
    attrs = %{email: params["email"]}

    case MyApp.Users.Invite.call(attrs) do
      {:ok, user} ->
        render_user(conn, user)
      {:error, :invalid_email} ->
        resp(conn, 400, %{message: "email is invalid"} |> Poison.encode!)
    end
  end

  defp render_user(conn, user) do
    {:ok, body} = %{
      id: user.id,
      email: user.email
    }
    |> Poison.encode

    conn
    |> resp(201, body)
  end
end

Notice the case statement. This is a place to add all the kinds of responses we can think of. When we later review the code, it’s easy to see what this controller can return.

Summing up

We have build a highly reusable and testable module for inviting a user. It is so reusable that it already has two consumers: The controller and the unit test.

The controller we have build is only concerned with web-specific stuff like parsing params, returning the right HTTP status code and serializing the response body to JSON. It’s easy to read and easy to maintain.

This blog post is long and it might seem like a long way to get to something very simple.

But in practice, when you sit down and write code with your headphones on, listening to the best of Hans Zimmer, writing this code might take around 5 minutes. Maybe only 3 minutes on a good day.

I don’t test because I’m afraid that the code base will break later on. I test because it yields a better code base. One that is easy to maintain and is highly reusable.

That I can run my tests again and again to ensure that nothing is broken is just a nice bi-product.

Stay tuned for Part II in which we’ll explore how to build a Mailer mock to use in tests.

Edit: Part II is now ready here.