You can check the latest updated version of this article at lobotuerto's notes - Building a JSON API with Phoenix 1.3 and Elixir.
It also includes a content table for easy navigation over there. ;)
Do you have a Rails background?
Are you tired of looking at outdated or incomplete tutorials on how to build a JSON API using Elixir and Phoenix?
Then, read on my friend!
Introduction
I have found there are mainly two types of tutorials one should write:
- Focused, scoped tutorials.
- Full step-by-step tutorials.
Scoped, focused tutorials, should be used to explain techniques, like this one: Fluid SVGs with Vue.js.
But, fully-integrated tutorials should be used to teach about new tech stacks.
Going from zero to fully working prototype without skipping steps.
With best practices baked-in, presenting the best libreries available for a given task.
I really like tutorials that take this holistic approach.
So, this won’t just be about how to generate a new Phoenix API only app. That’s easy enough, you just need to pass the --no-brunch --no-html
to mix phx.new
.
This tutorial is about creating a small, but fully operational JSON API for web applications.
To complement your API, I recommend Vue.js on the frontend:
Quickstart guide for a new Vue.js project.
What we’ll do:
- Create a new API-only Phoenix application —skip HTML and JS stuff.
- Create a User schema module (model) and hash its password —because storing plain text passwords in the database is just wrong.
- Create a Users endpoint —so you can get a list of, create or delete users!
- CORS configuration —so you can use that frontend of yours that runs on another port / domain.
- Create a Sign in endpoint —using session based authentication. I can do a JWT one later if enough people are interested.
Let me be clear about something…
I’m just starting with Elixir / Phoenix, if there are any omissions or bad practices, bear with me, notify me and I’ll fix them ASAP.
This is the tutorial I wish I had available when I was trying to learn about how to implement a JSON API with Elixir / Phoenix, but I digress.
Prerequisites
Install Elixir
We will start by installing Erlang and Elixir using the asdf version manager —using version managers is a best practice in development environments.
Install Hex
Let’s take on Hex the package manager now:
mix local.hex
You can now print some info about your Elixir stack with:
mix hex.info
You’ll see something along the lines of:
Hex: 0.17.3
Elixir: 1.6.4
OTP: 20.3.2
Built with: Elixir 1.5.3 and OTP 18.3
Install Phoenix
Now let’s install the framework:
mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez
Install PostgreSQL
PostgreSQL is the default database for new Phoenix apps, and with good reason: It’s a solid, realiable, and well engineered relational DB.
About REST clients
You might need to get a REST client so you can try out your API endpoints.
The two most popular ones seem to be Postman and Advanced Rest Client I tested both of them and I can say liked neither —at least on their Chrome app incarnations— as one didn’t display cookie info, and the other didn’t send declared variables on POST requests. ¬¬
In any case if you want to try them out:
If you are using a web frontend library like Axios, then your browser’s developer tools should be enough:
If you go with Axios dont’ forget to pass the configuration option withCredentials: true
, this will allow the client to send cookies along when doing CORS requests.
Or you can just use good ol’ curl
it works really well!
I’ll show you some examples on how to test out your endpoints from the CLI.
Create a new API-only Phoenix application
Generate the app files
In your terminal:
mix phx.new my-app --app my_app --module MyApp --no-brunch --no-html
From the command above:
- You’ll see
my-app
as the name for the directory created for this application. - You’ll see
my_app
used in files and directories insidemy-app/lib
e.g.my_app.ex
. - You’ll find
MyApp
used everywhere since it’s the main module for your app.
For example in my-app/lib/my_app.ex
:
defmodule MyApp do
@moduledoc """
MyApp keeps the contexts that define your domain
and business logic.
Contexts are also responsible for managing your data, regardless
if it comes from the database, an external API or others.
"""
end
Create the development database
If you created a new DB user when installing PostgreSQL , add its credentials toconfig/dev.exs
and config/test.exs
. Then execute:
cd my-app
mix ecto.create
NOTE:
You can drop the database for the dev environment with:
mix ecto.drop
If you’d like to drop the database for the test environment, you’d need to:
MIX_ENV=test mix ecto.drop
Start the development server
From your terminal:
mix phx.server
Visit http://localhost:4000
and bask in the glory of a beautifully formatted error page. :)
Don’t worry though, we’ll be adding a JSON endpoint soon enough.
Errors in JSON for 404s and 500s
If you don’t like to see HTML pages when there is an error and instead want to receive JSONs, set debug_errors
to false
in your config/dev.ex
, and restart your server:
config :my_app, MyAppWeb.Endpoint,
# ...
debug_errors: false,
# ...
Now, visiting http://localhost:4000
yields:
{ "errors": { "detail": "Not Found" } }
User schema (model)
We'll be generating a new User schema (model) inside an Auth context.
Contexts in Phoenix are cool, they serve as API boundaries that let you
organize your application code in a better way.
Generate the Auth context and User schema module
mix phx.gen.context Auth User users email:string:unique password:string is_active:boolean
From above:
- Auth is the context’s module name.
- User is the schema’s module name.
- users is the DB table’s name.
- After that comes some field definitions.
The migration generated from the command above looks like this:
defmodule MyApp.Repo.Migrations.CreateUsers do
use Ecto.Migration
def change do
create table(:users) do
add :email, :string#, null: false
add :password, :string
add :is_active, :boolean, default: false, null: false
timestamps()
end
create unique_index(:users, [:email])
end
end
Adjust it to your liking —i.e. not allowing null
for emails— then run the new migration:
mix ecto.migrate
If you want to read some info about this generator, execute:
mix help phx.gen.context
Hash a user’s password on saving
Add a new dependency to mix.exs
:
defp deps do
[
# ...
{:bcrypt_elixir, "~> 1.0"}
]
end
This is Bcrypt, we will use it to hash the user’s password before saving it; so we don’t store it as plain text inside the database.
Fetch the new app dependencies with:
mix deps.get
Change lib/my_app/auth/user.ex
to look like this:
defmodule MyApp.Auth.User do
# ...
def changeset(user, attrs) do
user
|> cast(attrs, [:email, :password, :is_active])
|> validate_required([:email, :password, :is_active])
|> unique_constraint(:email)
|> hash_user_password()
end
defp hash_user_password(%Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset) do
change(changeset, password: Bcrypt.hash_pwd_salt(password))
end
defp hash_user_password(changeset) do
changeset
end
end
Notice the call and definitions of hash_user_password/1
.
What this does is run the changeset through that function, and if the changeset happens to have a password
key, it’ll use Bcrypt to hash it.
Running Bcrypt.hash_pwd_salt("hola")
would result in something like"$2b$12$sI3PE3UsOE0BPrUv7TwUt.i4BQ32kxgK.REDv.IHC8HlEVAkqmHky"
.
That strange looking string is what ends up being saved in the database instead of the plain text version.
Fix the tests
Run the tests for your project with:
mix test
Right now they will fail with:
1) test users create_user/1 with valid data creates a user (MyApp.AuthTest)
test/my_app/auth/auth_test.exs:32
Assertion with == failed
code: assert user.password() == "some password"
left: "$2b$12$PUK73EqrvBTuOi2RiVrkOexqiVS.wIwbOtyR0EtzQLpbX6gaka8T2"
right: "some password"
stacktrace:
test/my_app/auth/auth_test.exs:36: (test)
2) test users update_user/2 with valid data updates the user (MyApp.AuthTest)
test/my_app/auth/auth_test.exs:43
Assertion with == failed
code: assert user.password() == "some updated password"
left: "$2b$12$cccPJfQD3seaBc8pHX8cJO/549lojlAjNNi/qo9QY0K7a7Zm5CaNG"
right: "some updated password"
stacktrace:
test/my_app/auth/auth_test.exs:49: (test)
That’s because of the change we just made, the one that hashes the password.
But this is easily fixed by changing those assertions to use Bcrypt.verify_pass/2
.
Open the file test/my_app/auth/auth_test.exs
and change these lines:
# ...
test "create_user/1 with valid data creates a user" do
# ...
assert user.password == "some password"
end
# ...
test "update_user/2 with valid data updates the user" do
#...
assert user.password == "some updated password"
end
# ...
To:
# ...
test "create_user/1 with valid data creates a user" do
# ...
assert Bcrypt.verify_pass("some password", user.password)
end
# ...
test "update_user/2 with valid data updates the user" do
# ...
assert Bcrypt.verify_pass("some updated password", user.password)
end
# ...
Now mix test
should yield no errors.
Users endpoint
Generate a new JSON endpoint
Let’s generate the users JSON endpoint, since we already have the Auth
context and User
schema available, we will pass the --no-schema
and --no-context
options.
mix phx.gen.json Auth User users email:string password:string is_active:boolean --no-schema --no-context
Fix the tests
Now, if you try and run your tests you’ll see this error:
== Compilation error in file lib/my_app_web/controllers/user_controller.ex ==
** (CompileError) lib/my_app_web/controllers/user_controller.ex:18: undefined function user_path/3
It’s complaining about a missing user_path/3
function.
You need to add a resources
line to lib/my_app_web/router.ex
.
Declaring a resource in the router will make some helpers available for its controller —i.e. user_path/3
.
defmodule MyAppWeb.Router do
# ...
scope "/api", MyAppWeb do
pipe_through :api
resources "/users", UserController, except: [:new, :edit]
end
end
Nevertheless, tests will still complain.
To fix them we need to make two changes:
- Comment out the
password
line inlib/my_app_web/views/user_view.ex
:
defmodule MyAppWeb.UserView do
# ...
def render("user.json", %{user: user}) do
%{id: user.id,
email: user.email,
# password: user.password,
is_active: user.is_active}
end
end
We don’t need —nor should we— be sending the hashed password in our responses.
- Since we won’t be receiving the hashed password in the response, comment out these
password
lines intest/my_app_web/controllers/user_controller_test.exs
:
# ...
assert json_response(conn, 200)["data"] == %{
"id" => id,
"email" => "some email",
"is_active" => true
# "password" => "some password"
}
# ...
assert json_response(conn, 200)["data"] == %{
"id" => id,
"email" => "some updated email",
"is_active" => false
# "password" => "some updated password"
}
# ...
Tests should be fine now.
Create a couple of users
Using IEx
You can run your app inside IEx (Interactive Elixir) —this is akin to rails console
— with:
iex -S mix phx.server
Then create a new user with:
MyApp.Auth.create_user(%{email: "asd@asd.com", password: "qwerty"})
Using curl
If you have curl
available in your terminal, you can create a new user through your endpoint using something like:
curl -H "Content-Type: application/json" -X POST \
-d '{"user":{"email":"some@email.com","password":"some password"}}' \
http://localhost:4000/api/users
CORS configuration
You’ll need to configure this if you plan on having your API and frontend on different domains.
If you don’t know what CORS is, have a look at this: Cross-Origin Resource Sharing (CORS).
That said, here we have two options:
I’ll be using CorsPlug in this tutorial, but if you need more features for configuring your CORS requests —or want a more strict libray, try Corsica out.
Add this dependency to mix.exs
:
defp deps do
[
# ...
{:cors_plug, "~> 1.5"}
]
end
Fetch new dependencies with:
mix deps.get
Add plug CORSPlug
to lib/my_app_web/endpoint.ex
:
defmodule MyAppWeb.Endpoint do
# ...
plug CORSPlug, origin: "http://localhost:8080"
plug MyAppWeb.Router
# ...
end
You can pass a list to origin
, as well as a regular expression, or justplug CORSPlug
to accept all origins —since origin: "*"
is the default.
In my case, the rule above will accept CORS requests from a Vue.js frontend —Vue.js development servers go up on port 8080 by default.
Simple authentication
Verify a user’s password
Let’s add some functions to the lib/my_app/auth/auth.ex
file to verify a user’s password:
defmodule MyApp.Auth do
# ...
def authenticate_user(email, password) do
query = from u in User, where: u.email == ^email
query |> Repo.one() |> verify_password(password)
end
defp verify_password(nil, _) do
{:error, "Wrong email or password"}
end
defp verify_password(user, password) do
case Bcrypt.verify_pass(password, user.password) do
true -> {:ok, user}
false -> {:error, "Wrong email or password"}
end
end
end
sign_in endpoint
Then add a new sign_in
endpoint to lib/my_app_web/router.ex
:
defmodule MyAppWeb.Router do
# ...
scope "/api", MyAppWeb do
# ...
resources "/users", UserController, except: [:new, :edit]
post "/users/sign_in", UserController, :sign_in
end
end
sign_in controller function
Finally add the sign_in
function to lib/my_app_web/controllers/user_controller.ex
:
defmodule MyAppWeb.UserController do
# ...
def sign_in(conn, %{"email" => email, "password" => password}) do
case MyApp.Auth.authenticate_user(email, password) do
{:ok, user} ->
conn
|> put_status(:ok)
|> render(MyAppWeb.UserView, "sign_in.json", user: user)
{:error, message} ->
conn
|> put_status(:unauthorized)
|> render(MyAppWeb.ErrorView, "401.json", message: message)
end
end
end
Define sing_in.json and 401.json views
In lib/my_app_web/user_view.ex
add this:
defmodule MyAppWeb.UserView do
# ...
def render("sign_in.json", %{user: user}) do
%{data:
%{user:
%{id: user.id, email: user.email}}}
end
end
In lib/my_app_web/error_view.ex
add this:
defmodule MyAppWeb.ErrorView do
# ...
def render("401.json", %{message: message}) do
%{errors: %{detail: message}}
end
# ...
end
You can try the sign_in
endpoint now.
Try out your new endpoint with curl
Let’s send some POST requests to http://localhost:4000/api/users/sign_in
.
Good credentials
curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in -i
You’ll receive a 200 with:
{
"data": {
"user": { "id": 1, "email": "asd@asd.com" }
}
}
Bad credentials
curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"not the right password"}' \
http://localhost:4000/api/users/sign_in -i
You’ll get a 401 and:
{ "errors": { "detail": "Wrong email or password" } }
Sessions
Add plug :fetch_session
to your :api
pipeline in lib/my_app_web/router.ex
:
defmodule MyAppWeb.Router do
# ...
pipeline :api do
plug :accepts, ["json"]
plug :fetch_session
end
# ...
end
Save authentication status
Now let’s modify out sign_in
function in lib/my_app_web/controllers/user_controller.ex
:
defmodule MyAppWeb.UserController do
# ...
def sign_in(conn, %{"email" => email, "password" => password}) do
case MyApp.Auth.authenticate_user(email, password) do
{:ok, user} ->
conn
|> put_session(:current_user_id, user.id)
|> put_status(:ok)
|> render(MyAppWeb.UserView, "sign_in.json", user: user)
{:error, message} ->
conn
|> delete_session(:current_user_id)
|> put_status(:unauthorized)
|> render(MyAppWeb.ErrorView, "401.json", message: message)
end
end
end
Protect a resource with authentication
Modify your lib/my_app_web/router.ex
to look like this:
defmodule MyAppWeb.Router do
use MyAppWeb, :router
pipeline :api do
plug :accepts, ["json"]
plug :fetch_session
end
pipeline :api_auth do
plug :ensure_authenticated
end
scope "/api", MyAppWeb do
pipe_through :api
post "/users/sign_in", UserController, :sign_in
end
scope "/api", MyAppWeb do
pipe_through [:api, :api_auth]
resources "/users", UserController, except: [:new, :edit]
end
# Plug function
defp ensure_authenticated(conn, _opts) do
current_user_id = get_session(conn, :current_user_id)
if current_user_id do
conn
else
conn
|> put_status(:unauthorized)
|> render(MyAppWeb.ErrorView, "401.json", message: "Unauthenticated user")
|> halt()
end
end
end
As you can see we added a new pipeline called :api_auth
that’ll run requests through a new :ensure_authenticated
plug function.
We also created a new scope "/api"
block that pipes its requests through :api
then through :api_auth
and moved resources "/users"
inside.
Isn’t it amazing the way you can define this stuff in Phoenix?!
Composability FTW!
Endpoint testing with curl
Try to request a protected resource, like /api/users
with:
curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i
You’ll get:
{ "errors": { "detail": "Unauthenticated user" } }
Let’s login with:
curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in \
-c cookies.txt -b cookies.txt -i
You’ll get:
{
"data": {
"user": { "id": 1, "email": "asd@asd.com" }
}
}
Now, try requesting that protected resource again:
curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i
You’ll see:
{
"data": [
{ "is_active": false, "id": 1, "email": "asd@asd.com" },
{ "is_active": false, "id": 2, "email": "some@email.com" }
]
}
Success!
Bonus section
Customize your 404s and 500s JSON responses
In lib/my_app_web/views/error_view.ex
:
defmodule MyAppWeb.ErrorView do
# ...
def render("404.json", _assigns) do
%{errors: %{detail: "Endpoint not found!"}}
end
def render("500.json", _assigns) do
%{errors: %{detail: "Internal server error :("}}
end
# ...
end
Links
- User Authentication from Scratch in Elixir and Phoenix
- Tip for Phoenix 1.3 Fallback Controller error
- curl: how to send cookies via command line?
- Show HTTP response header using curl
This was a long one, that’s it for now folks!
Top comments (2)
Victor,
This is an excellent tutorial; especially for a total beginner like myself. I learned a great deal and will be using this as a reference in the future.
I have a few notes:
--no-brunch no longer exists
I see one other post about it on "Today I Learned" but no mention in the docs. I suspect, in my ignorance, this was replaced with --no-webpack.
Typo under "Save authentication status"
Now let’s modify our sign_in function in
Test paths are a little different
test/my_app/auth/auth_test.exs is in test/my_app/auth_test.exs
This tutorial is way cool. Thank you.
Sadly Phoenix has changed a lot and, as of today, your tutorial is not very useful anymore.