There several good articles about how to setup a Phoenix cluster with PubSub messaging through the nodes but I found them to be incomplete or slightly out of date so in this article I plan to give step by step instructions on how to create a clustered Phoenix 1.7 application in 2023.
The goal of this article will be to run multiple instances of our Phoenix which seamlessly can do message passing between nodes.
The target production environment will be Fly.io for this article, but I'll also show how to setup your project for local development as well so you can make sure you are subscribing and broadcasting correctly.
Simplest Demo Project
We're going to create a stripped down Phoenix app that does only two things:
- Exposes a live view that displays messages received
- Has an API end-point to receive external messages via HTTP
Create project
Note - we will build this project step by step but feel free to check out the completed project here
Setup Phoenix app
Phoenix new with most things removed:
mix phx.new talkie --no-ecto --no-mailer
Update the config/dev.exs file to allow passing in a PORT since we'll need to run two instances:
# Before
config :talkie, TalkieWeb.Endpoint,
http: [ip: {127, 0, 0, 1}, port: 4000],
...
# After
port = String.to_integer(System.get_env("PORT") || "5000")
config :talkie, TalkieWeb.Endpoint,
http: [ip: {127, 0, 0, 1}, port: port],
...
Create the ping API endpoint:
# Add this controller
defmodule TalkieWeb.APIController do
use TalkieWeb, :controller
def ping(conn, _params) do
# Not doing anything but responding so far
json(conn, %{pong: true})
end
end
Add the ping route to your router.ex inside the existing /api scope:
scope "/api", TalkieWeb do
pipe_through :api
get "/ping", APIController, :ping
end
Test it:
curl http://127.0.0.1:5000/api/ping
# {"pong": true}
Create the Liveview viewer
Add a LiveView module like this
defmodule TalkieWeb.ViewerLive.Index do
use TalkieWeb, :live_view
@impl true
def render(assigns) do
~H"""
<h1>Messages</h1>
<%= for msg <- @messages do %>
<span><%= msg %></span>
<% end %>
"""
end
@impl true
def mount(_params, _session, socket) do
{:ok, assign(socket, :messages, [])}
end
@impl true
def handle_info({:message, msg}, socket) do
{:noreply, assign(socket,
:messages, [msg | socket.assigns.messages])}
end
@impl true
def handle_info(
%Phoenix.Socket.Broadcast{
topic: "messages",
event: "ping",
payload: {:message, msg}
}, socket) do
handle_info({:message, msg}, socket)
end
end
And add this line to your router.ex:
# in the browser scope
live "/viewer", ViewerLive.Index
Test it by pointing your browser at (http://127.0.0.1:5000/viewer)
Listen for messages on One instance
Now that we have those components. Let's wire them together with a single instance.
We're going to have our Viewer listen for messages, and our API broadcast whenever the /ping api is hit.
You "listen" by subscribing to a topic. Update the liveview mount to look like this now:
def mount(_params, _session, socket) do
TalkieWeb.Endpoint.subscribe("messages")
{:ok, assign(socket, :messages, ["test message..."])}
end
And you broadcast by adding the broadcast to the ping function this the API Controller:
def ping(conn, _params) do
Phoenix.PubSub.broadcast!(Talkie.PubSub, "messages",
%Phoenix.Socket.Broadcast{
topic: "messages",
event: "ping",
payload: {:message, "ping"}
}
)
json(conn, %{pong: true})
end
Test it
Start the server:
PORT=5000 mix phx.server
Open a browser to: (http://127.0.0.1:5000/viewer)
Hit the ping api again:
curl http://127.0.0.1:5000/api/ping
This should cause the Viewer to display a "ping" message instantly.
Clustering for local dev
Now let's see clustering NOT work first. Open up two instances like this:
PORT=5000 elixir --name a@127.0.0.1 -S mix phx.server
PORT=5001 elixir --name b@127.0.0.1 -S mix phx.server
Bring up a Viewer pointed to both servers:
(http://127.0.0.1:5000/viewer)
(http://127.0.0.1:5001/viewer)
And if you run the curl on the ping API you'll see that only one viewer will update.
Add libcluster to your dependencies in mix.exs and rerun mix deps.get
{:libcluster, "~> 3.3"},
Add the libcluster config to your config/dev.exs:
config :libcluster,
topologies: [
example: [
strategy: Cluster.Strategy.Epmd,
config: [hosts: [:"a@127.0.0.1", :"b@127.0.0.1"]],
connect: {:net_kernel, :connect_node, []},
disconnect: {:erlang, :disconnect_node, []},
list_nodes: {:erlang, :nodes, [:connected]},
]
]
Update your application.ex
You will define a topologies variable in your start() function and add a Cluster.Supervisor to the list of children.
It will look like:
def start(_type, _args) do
topologies = Application.get_env(:libcluster, :topologies) || []
children = [
...
{Cluster.Supervisor, [topologies, [name: Talkie.ClusterSupervisor]]}
]
...
Now if you restart both of the instances and test again you SHOULD see both Viewers update when you ping either one of the instances.
Pretty cool right!
Clustering with Fly.io
First, deploy the app as-is to Fly and test that you see it not working as expected again.
What you should see (assuming you default to 2 instances) is a 50/50 chance that you'll receive the ping message if you pull up a /viewer page and then hit the ping API. A rough test is to bring up 2-4 separate windows and to the /viewer path and test a ping to see how many of the windows update.
Add libcluster configuration to prod in the config/runtime.exs file
app_name = System.get_env("FLY_APP_NAME") || "talkie"
config :libcluster,
debug: true,
topologies: [
fly6pn: [
strategy: Cluster.Strategy.DNSPoll,
config: [
polling_interval: 5_000,
query: "#{app_name}.internal",
node_basename: app_name
]
]
]
Change how the RELEASE_NODE value is set in the rel/env.sh.eex
Note if you don't have this file, then you just haven't completed fly deploy yet. Do that first.
#Before
export RELEASE_NODE="${FLY_APP_NAME}-${FLY_IMAGE_REF##*-}@${FLY_PRIVATE_IP}"
#After
ip=$(grep fly-local-6pn /etc/hosts | cut -f 1)
export RELEASE_NODE="${FLY_APP_NAME}@${ip}"
See more from Fly.io guide
See the full project
Check out the Github Repofor the project used to test this article.
Other Resources
There are some good examples here:
Well written article from Alvise Susmel that helped me a lot
Top comments (0)