DEV Community

Cover image for Techniques for Idempotency in seeds.exs
Byron Salty
Byron Salty

Posted on

Techniques for Idempotency in seeds.exs

Idempotency is an annoying word that makes people think you are an asshole when you use it, at least if they aren't exactly sure what it means.

But it's such a useful concept that I use it anyway and will just suffer the consequences.

For the uninitiated, it simply means that I can run the same code over and over and over but the result will always be the same.

Consider these two versions of a function trying to add tax to a bill:

defmodule Bill do
  defstruct total: 0.0, subtotal: 0.0
end

defmodule Tax do
  def calc_tax(bill) do
    %Bill{bill | total: bill.total * 1.07}
  end

  def calc_tax_w_subtotal(bill) do
    tax = bill.subtotal * 0.07
    %Bill{bill | total: bill.subtotal + tax}
  end
end
Enter fullscreen mode Exit fullscreen mode

Both would work fine, if they are only called one time. In both cases, the current total is increased by 7%. The second version is more complex because we have the concept of both a subtotal and a total.

But the first version is not idempotent.

A race condition, bad calling code, or any number of other reasons could result in something like the following:

bill1 = %Bill{total: 10.0}
bill1 = Tax.calc_tax(bill1)
bill1 = Tax.calc_tax(bill1)
bill1.total # 11.449
Enter fullscreen mode Exit fullscreen mode

vs:

bill2 = %Bill{subtotal: 10.0}
bill2 = Tax.calc_tax_w_subtotal(bill2)
bill2 = Tax.calc_tax_w_subtotal(bill2)
bill2.total # 10.7
Enter fullscreen mode Exit fullscreen mode

In the case of the first, the tax compounds because we're only working with a single total and we can't tell if that total already includes tax. In the second case, we can calculate tax as much as we want but it doesn't effect the result after the first time.


Where it matters - Seeds

I see the need for idempotency in a lot of places but a very likely place you'll run into and need to care about it is with your seeds code.

If you run your seeds multiple times you probably don't want your default users or system categories or whatever created more than once.

I tried out a new form today to achieve idempotency in my seeds.exs and I found it cleaner than a simple if based approach so I thought I'd share. I'd love to hear if others have found better forms.

Approach one - ecto.reset

I suppose should start with the brute force method. If you always run your seeds as part of mix ecto.reset then you are achieving idempotency by always starting with a blank slate.

Similarly, you could create another approach in which you try to delete specific entities if they exist, before creating them in the seeds. (We can call that 1B)

Approach two - try

We could rely on database unique constraints to prevent multiple similar entries. We just try to create every time and ignore any errors:

try do
  Repo.insert!(%Category{
    name: "Books",
  })
rescue
  _ ->
    IO.puts("Already seeded")
end
Enter fullscreen mode Exit fullscreen mode

Approach three - if

I've done this one a fair amount. It's not too bad, and rather readable, especially if all you want to do is create the entry (but not use it later in the seeds).

For example, this doesn't look horrible:

if Repo.get_by(Category, name: "Books") == nil do
  Repo.insert!(%Category{
    name: "Books",
  })
end
Enter fullscreen mode Exit fullscreen mode

But it looks worse if you want to use that entity later:

book_category = 
  if Repo.get_by(Category, name: "Books") == nil do
    Repo.insert!(%Category{
      name: "Books",
    })
  else
    Repo.get_by(Category, name: "Books")
  end
Enter fullscreen mode Exit fullscreen mode

(Yes, you could keep the result of the first get_by but the form doesn't really get much cleaner.)

lookup = Repo.get_by(Category, name: "Books")
book_category = 
  if lookup == nil do
    Repo.insert!(%Category{
      name: "Books",
    })
  else
    lookup
  end
Enter fullscreen mode Exit fullscreen mode

Approach four - case

This is my current favorite way of achieving idempotency and capturing the value for later use:

book_category =
  case Repo.get_by(Category, name: "Books") do
    nil -> Repo.insert!(%Category{
      name: "Books",
    })
    category -> category
  end
Enter fullscreen mode Exit fullscreen mode

Is there something even better that you use?

Please share!


If you found this article helpful, show your support with a Like, Comment, or Follow.

Read more of Byron’s articles about Leadership and AI.

Development articles here.

Follow on MediumDev.toTwitter, or LinkedIn.

Top comments (2)

Collapse
 
andreasknoepfle profile image
Andreas Knöpfle

We usually do upserts and hardcode the primary key, since there is always a unique constraint on it:

book_category =   Repo.insert!(
    %Category{name: "Books", id: "f671a68d-3042-4269-8067-89a78a11be48"},
    on_conflict: :nothing
  )
Enter fullscreen mode Exit fullscreen mode

This has some advantages:

  • You can write development tools that make some assumptions on ids (mock servers, etc)
  • You can bookmark development resources that were seeded, since they will have the same paths even if their primary key is in the routes

The only downside is that one has to manage the primary keys a bit.

Collapse
 
byronsalty profile image
Byron Salty

This is awesome. Exactly the type of feedback that I was hoping to get.

The other piece that it seems you are doing differently than me is non-sequential ids. Probably a good idea in any case.