DEV Community

Cover image for Handling state between multiple processes with elixir
Cherry Ramatis
Cherry Ramatis

Posted on • Updated on

Handling state between multiple processes with elixir

Elixir works really well for concurrent code because of it's functional nature and ability to run in multiple processes, but how we handle state when our code is running all over the place? Well, there is some techniques and in this article we'll learn more about it together shall we?

Table of contents

What is a process? How to use it with send and receive

image

Processes are the answer from Elixir to concurrent programming; they're basically a continuous-running node that can send and receive messages. In fact, every function in Elixir runs inside a process. Although this sounds really expensive, it's super lightweight compared to threads in other languages, which empowers us developers to build incredibly scalable software with hundreds of processes running at the same time. Another great advantage of using this specifically with the Elixir language is that this language is built on top of immutability and other functional programming concepts, so we can trust that these functions are running completely isolated and without changing or maintaining global state.

The basic way of seeing a process in action is by using the spawn function, with that we can execute a function in a process and get the pid of it.

iex(3)> pid = spawn(fn -> IO.puts("teste") end)
teste
#PID<0.111.0>
iex(4)> pid
#PID<0.111.0>
iex(5)> Process.alive?(pid)
false
iex(6)>
Enter fullscreen mode Exit fullscreen mode

As you can see from the return of Process.alive?(pid) this process is already dead once it runs correctly, but we can easily add a sleep function to check this mechanism:

iex(2)> pid = spawn(fn -> :timer.sleep(10000); IO.puts("teste") end)
#PID<0.111.0>
iex(3)> Process.alive?(pid)
true
teste
iex(4)> Process.alive?(pid)
false
iex(5)>
Enter fullscreen mode Exit fullscreen mode

Since we're sleeping for 10 seconds, the process is alive until it runs out after the sleep function and dies. Cool right? It's important to know that our main program did not hang, it simply put the function in a process and forgot about it. This allows us to create really modular and performant code that runs on multiple nodes.

Besides spawning functions in a process, we can transition information between processes using the functions send and the receive block, as shown below:

iex(1)> defmodule Listener do
...(1)> def call do
...(1)> receive do
...(1)> {:hello, msg} -> IO.puts("Received: #{msg}")
...(1)> end
...(1)> end
...(1)> end
{:module, Listener,
 <<70, 79, 82, 49, 0, 0, 6, 116, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 240,
   0, 0, 0, 25, 15, 69, 108, 105, 120, 105, 114, 46, 76, 105, 115, 116, 101,
   110, 101, 114, 8, 95, 95, 105, 110, 102, 111, ...>>, {:call, 0}}
iex(2)> pid = spawn(&Listener.call/0)
#PID<0.115.0>
iex(3)> send(pid, {:hello, "Hello World"})
Received: Hello World
{:hello, "Hello World"}
iex(4)>
Enter fullscreen mode Exit fullscreen mode

Observe that we define a function that acts as a general listener using the receive block. This works as a switch case where we can pattern match and do a quick action, in this case, we're simply printing to STDOUT. Once we spawn this listener, it's possible to use the returned pid to send information using the send/2 function that expects a PID and a value as arguments.

That way, it's possible to keep state in an immutable and separate environment such as elixir.

Incrementing our experience with tasks

The Task module offers an abstraction on top of the spawn function while adding support for asynchronous behavior, i.e., creating a function in a separate process and observing its behavior with wait functions. As you delve into Elixir, you'll discover that the Task module allows you to start a new process that executes a function and returns a task structure. With this structure in hand, you can easily get the value from this function using the Task.await(task) clause, as shown below:

iex(1)> task = Task.async(fn ->
...(1)>   IO.puts("Task is running")
...(1)>   42
...(1)> end)
Task is running
%Task{
  mfa: {:erlang, :apply, 2},
  owner: #PID<0.109.0>,
  pid: #PID<0.110.0>,
  ref: #Reference<0.0.13955.659691257.723058689.43945>
}
iex(2)> IO.puts "a code"
a code
:ok
iex(3)> answer_to_everything = Task.await(task)
42
iex(4)> answer_to_everything
42
iex(5)>
Enter fullscreen mode Exit fullscreen mode

First we saw the Task is running message printed out, and then we got the task struct. Further, we could execute any code in between, and when we're ready, it's just a matter of using the Task.await function to retrieve the function return.

The Task module also provides a common interface for the regular spawn function called start, we can even reuse the code shown at the beginning with the new module abstraction:

iex(1)> defmodule Listener do
...(1)> def call do
...(1)> receive do
...(1)> {:print, msg} -> IO.puts("Received message: #{msg}")
...(1)> end
...(1)> end
...(1)> end
{:module, Listener,
 <<70, 79, 82, 49, 0, 0, 6, 244, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 245,
   0, 0, 0, 26, 15, 69, 108, 105, 120, 105, 114, 46, 76, 105, 115, 116, 101,
   110, 101, 114, 8, 95, 95, 105, 110, 102, 111, ...>>, {:call, 0}}
iex(2)> {:ok, pid} = Task.start(&Listener.call/0)
{:ok, #PID<0.115.0>}
iex(3)> send(pid, {:print, "Eat more fruits"})
Received message: Eat more fruits
{:print, "Eat more fruits"}
Enter fullscreen mode Exit fullscreen mode

It's useful to use the Task module because we can get a higher level of abstraction. You must have noticed that the interface for Task.start and Task.async is the same, right? Yeah, we can swap those and get the power of using Task.await and Task.yield on top of it, that's the power of abstracting lower-level concepts!

Designing state with the agent wrapper

image

The Agent module provides another layer of abstraction focused on controlling state between multiple instances of a process, it acts like a data structure for long-running interactions.

We can first start an agent instance with an initial value passed from a function return, as shown below:

iex(1)> {:ok, agent} = Agent.start_link(fn -> [] end)
{:ok, #PID<0.110.0>}
iex(2)> agent
#PID<0.110.0>
iex(3)>
Enter fullscreen mode Exit fullscreen mode

As you can see, we get a PID just like the other abstractions, the difference here can be observed in the usage of other methods.

For example, we can update the original array by appending a value to it:

iex(3)> Agent.update(agent, fn list -> ["elixir" | list] end)
:ok
iex(4)>
Enter fullscreen mode Exit fullscreen mode

That's the whole difference in this abstraction provided by the Agent module, we can continuously update a state by appending immutable functions as callbacks and reusing the same PID.

We can also return a particular value from the data structure by using the following function:

iex(4)> Agent.get(agent, fn list -> list end)
["elixir"]
iex(5)>
Enter fullscreen mode Exit fullscreen mode

See? It's as simple as returning the whole list from the callback function, you can imagine that it's possible to use any method from Elixir to filter down this list if wanted and keep iterating over the data structure.

Conclusion

This is a simple introduction to this concept that is new for me, and I hope it's useful for anyone reading it! And in the next articles, we'll dive deeper into other topics in elixir, such as Gen Servers, Supervisors, etc. May the force be with you! 🍒

Top comments (26)

Collapse
 
brunonovais profile image
Bruno Rezende Novais • Edited

It's accurated to say that Elixir have lightweight threads then? Idk how exactly the elixir compiler deals with physical threads (1:1 or 1:n, for virtual threads example).
But seeing this point that one function means one process in the concurrency scenario makes me think about lightweight and heavy threads

Collapse
 
cherryramatis profile image
Cherry Ramatis

I think we can say that elixir have lightweight threads indeed, the BEAM Virtual machine manage these multiple processes like an linux OS basically, if it's throwing any errors it kills the process and immediately restarts (following a supervisor tree, I'll write about on the next article maybe :D )

Collapse
 
zoedsoupe profile image
Zoey de Souza Pessanha

Actually the Erlang BEAM virtual machine starts a Schedulers (an especial type of process that supervises and manage tons of others processes, including processes supervisors) for each core in your CPU. So in my case I have 8 cores, so when I start a BEAM (erlang/elixir/gleam) application it will start 8 Schedulers. Its good to notice that all BEAM schedulers are preemptive, that means that each Scheduler can pause/continue each process execution on the “Run Queue”.

So i don;t think the term “lightweight threads” is a good fit for the BEAM… The Erlang virtual machine acts exactly like an operational system, where there are processes, applications (like ExUnit or hex for example), and so on. These Schedulers would then pick some processes from the “Run Queue” and let them been executed, paused or continued.

Each Scheduler represents a real thread!

An example image can be found here: kagi.com/images?q=eralng+beam+sche...

Thread Thread
 
zoedsoupe profile image
Zoey de Souza Pessanha

I think that would be more appropriate to say that the BEAM uses real threads, and those Schedulers contexts and processes management can be seen as “lightweight threads”, but they are semantically different, althouth share some similarities and can be easier to understand comparing both (:

Collapse
 
tandrieu profile image
Thibaut Andrieu

I saw a talk about Elixir at a conferences. The guy implemented Conway's Game of Life on 1000x1000 grid with one process per cells. So 1 million processes running in parallel. Quite impressive indeed !

Collapse
 
zoedsoupe profile image
Zoey de Souza Pessanha

Awesome article 🍒! I really liked the approach you followed writing about processes and message passing!

Collapse
 
cherryramatis profile image
Cherry Ramatis

It means a lot coming from you! Thanks

Collapse
 
canhassi profile image
Canhassi

Nice article!!

Collapse
 
cherryramatis profile image
Cherry Ramatis

thanks a lot <3

Collapse
 
elixir_utfpr profile image
Elixir UTFPR (por Adolfo Neto)

Great post! I created a livebook for the code in this post gist.github.com/adolfont/a928aff18...

Collapse
 
cherryramatis profile image
Cherry Ramatis

Wow!!! Thanks a lot this is so useful

Collapse
 
renanvidal profile image
Renan Vidal Rodrigues

Perfect content as always, congratulations Cherry!!

Collapse
 
cherryramatis profile image
Cherry Ramatis

Thanks a lot <3 hope it's useful for you

Collapse
 
g33knoob profile image
G33kNoob

I just learned elixir by instal blockcscout great article

Collapse
 
cherryramatis profile image
Cherry Ramatis

Happy that was useful for you! soon I'll be making more about this whole processes series

Collapse
 
ilonavarro profile image
Ilo Navarro

Nice article, thanks a lot!

Collapse
 
cherryramatis profile image
Cherry Ramatis

it's a pleasure to bring cool content <3

Collapse
 
artenlf profile image
Luís Felipe Arten

Bravo, Cherry 🍒!!! 👏🏻👏🏻👏🏻👏🏻
Always posting great articles, full of details and technical content! Thank you for sharing your knowledge with the community!

Collapse
 
cherryramatis profile image
Cherry Ramatis

It's always a pleasure to share cool stuff !

Collapse
 
phenriquesousa profile image
Pedro Henrique

The queen

Collapse
 
cherryramatis profile image
Cherry Ramatis

omg thanks cousin! <3 <3

Collapse
 
yayaflc profile image
Yasmin Felício

Elixir 😱 Nice article, prima! 💜

Collapse
 
cherryramatis profile image
Cherry Ramatis

thanks sweeheart <3 <3

Collapse
 
fransborges profile image
Fran Borges

I've saved it, so I can read it slowly and learn as much as I can from the knowledge I know there is in this article

Collapse
 
cherryramatis profile image
Cherry Ramatis

If you need anything just reach out <3 <3

Collapse
 
vegadvalentin profile image
Valentin Vegad

Thank you