You need a Circle of Trust for faster and safer development

Do you want faster development, less code, fewer tests, higher quality code, fewer production failures, and a better customer experience all at nearly no additional development cost? Consider building a Circle of Trust.

One of the fantastic principles enabled by Elixir is the freedom to “let it crash”. We all recognize that it’s impossible to write perfect, failure-free code. And even if we could somehow, hardware failures and even radiation can still corrupt your state, creating situations you simply cannot recover from. So … let the process and its bad state die, and start over from a clean slate. Simple. Beautiful.

Sadly though, most failures we encounter are of our own making. Either our algorithms introduced the bug, or more likely, we didn’t account for certain state. That uncertainty of state invariably leads to overly defensive and hard-to-follow code, and frequent production failures that are difficult to debug. Which is only made worse when using Elixir’s super-efficient processes to improve performance because the logged stack traces can be nearly useless trying to locate the error source. “Awesome … an anonymous in-lined Erlang function inside a GenServer callback. Good luck finding that one!”.

While test suites run during build/deploy to prove that anticipated problems are handled somewhere in our code, a Circle of Trust is a runtime concept like a Customs officer; always checking that the provided goods are in fact what we were told to expect. That way, if we’re given bad state from an outside source it’ll never reach our algorithms, and a full report can be created to make it easier to find the offender. And, like the Customs officer, you have only one place to go to define what is permitted, how sources should be packaged, and what is available in the shipment.

A Circle of Trust has almost no real development cost. It’s easy to start, provides immediate rewards, and dividends increase the more you invest. Stay with me, I’ll clarify some of the substantial benefits below once I’ve explained what it is.

So … what IS a Circle of Trust?

A Circle of Trust adapts the OO engineering principles of Encapsulation, and Separation of Concerns into Functional Programming using modules to validate data that originated outside your control and provide accessors for that data. This allows internal algorithms a level of trust for source data and a measure of isolation from structural changes source data may experience later. Once the module is created, move all defensive “what about this” code into that module’s validation routine.

That’s it.

Granted, some of the benefits of a Circle of Trust can be realized by more traditional functional or interface test suites. But unless your system has super high performance requirements, the run time validation in a Circle of Trust can provide far more bang for your buck. And, if some validations do create bottlenecks, the costly portions can be moved to your interface test suite.

Author’s side note: I would be surprised if you found a formal software related definition for “Circle of Trust”. It is a term our Atlas team at aQuantive started using to refer to the concept of runtime validation and encapsulation of source data. Even though we had substantial unit and interface test suites, adopting this principle provided all the benefits described here.

Let’s have an example applied to incoming data from a simple “Contact Us” form a customer might use to inquire about something. To shorten things up a bit I’ve left off most @spec and @doc attributes.

Here I’ll use Ecto to validate that required fields were present and mutate data types where needed, provide declarative valid?, if data is invalid errors clearly identifies what was wrong, and finally it provides accessors to those fields either as a collection from fields or individually such as email.

defmodule Email.Schema.ContactUs do
  use Ecto.Schema
  import Ecto.Changeset
  alias Email.Schema.ContactUs

  embedded_schema do
    field(:email, :string)
    field(:subject, :string)
    field(:message, :string)
    field(:request_date, :date)
  end

  @type t :: %__MODULE__{
          email: String.t(),
          subject: String.t(),
          message: String.t(),
          request_date: Date.t()
        }

  @valid_subjects [
    "General Question",
    "Order Status",
    "Compliment",
    "Complaint",
    "Other"
  ]

  def cast(form_data) do
    %ContactUs{}
    |> cast(form_data, ~w(email subject message request_date)a)
    |> validate_required(~w(email subject message)a)
    |> validate_format(:email, ~r/\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\z/i)
    |> validate_inclusion(:subject, @valid_subjects)
  end

  def valid?(%Ecto.Changeset{valid?: valid}), do: valid
  def errors(%Ecto.Changeset{errors: errors}), do: errors

  def fields(%Ecto.Changeset{changes: changes}), do: struct!(__MODULE__, changes)

  def email(%Ecto.Changeset{changes: %{email: email}}), do: email
  def subject(%Ecto.Changeset{changes: %{subject: subject}}), do: subject
  def message(%Ecto.Changeset{changes: %{message: message}}), do: message
  def request_date(%Ecto.Changeset{changes: %{request_date: request_date}}), do: request_date
end

Its unit test file clarifies expectations from validation and accessors.

defmodule Email.Schema.ContactUsTest do
  use ExUnit.Case

  alias Email.Schema.ContactUs

  describe "cast/1 when all data is valid" do
    setup [:valid_data, :execute]

    test "valid? returns true", %{data: _data, results: results} do
      assert ContactUs.valid?(results)
    end

    test "email matches source data", %{data: data, results: results} do
      assert ContactUs.email(results) == data.email
    end

    test "subject matches source data", %{data: data, results: results} do
      assert ContactUs.subject(results) == data.subject
    end

    test "message matches source data", %{data: data, results: results} do
      assert ContactUs.message(results) == data.message
    end

    test "request_date is converted to Date type", %{data: _data, results: results} do
      assert ContactUs.request_date(results) === ~D[2019-10-28]
    end

    test "fields returns expected struct", %{data: data, results: results} do
      fields_is_as_expected(data, results)
    end
  end

  describe "cast/1 when missing required fields" do
    setup [:missing_data, :execute]

    test "valid? returns false", %{data: _data, results: results} do
      assert ContactUs.valid?(results) == false
    end

    test "reports missing required fields", %{data: _data, results: results} do
      assert ContactUs.errors(results) == [
               email: {"can't be blank", [validation: :required]},
               subject: {"can't be blank", [validation: :required]},
               message: {"can't be blank", [validation: :required]}
             ]
    end

    test "fields returns expected struct", %{data: data, results: results} do
      fields_is_as_expected(data, results, nil)
    end
  end

  describe "cast/1 when invalid subject" do
    setup [:valid_data, :invalid_subject, :execute]

    test "valid? returns false", %{data: _data, results: results} do
      assert ContactUs.valid?(results) == false
    end

    test "reports missing required fields", %{data: _data, results: results} do
      assert ContactUs.errors(results) == [
               subject:
                 {"is invalid",
                  [
                    validation: :inclusion,
                    enum: ["General Question", "Order Status", "Compliment", "Complaint", "Other"]
                  ]}
             ]
    end

    test "fields returns expected struct", %{data: data, results: results} do
      fields_is_as_expected(data, results)
    end
  end

  describe "cast/1 when invalid request_date" do
    setup [:valid_data, :invalid_request_date, :execute]

    test "valid? returns false", %{data: _data, results: results} do
      assert ContactUs.valid?(results) == false
    end

    test "reports missing required fields", %{data: _data, results: results} do
      assert ContactUs.errors(results) == [
               request_date: {"is invalid", [type: :date, validation: :cast]}
             ]
    end

    test "fields returns expected struct", %{data: data, results: results} do
      fields_is_as_expected(data, results, nil)
    end
  end

  defp fields_is_as_expected(data, results, expected_date \\ ~D[2019-10-28]) do
    expected = %Email.Schema.ContactUs{
      email: data[:email],
      message: data[:message],
      request_date: expected_date,
      subject: data[:subject]
    }

    assert expected == ContactUs.fields(results)
  end

  defp valid_data(_context) do
    data = %{
      email: "foo@bar.com",
      subject: "Compliment",
      message: "message",
      request_date: "2019-10-28"
    }

    {:ok, data: data}
  end

  defp missing_data(_context) do
    {:ok, data: %{}}
  end

  defp invalid_subject(%{data: data}) do
    {:ok, data: put_in(data, [:subject], "this subject is not in approved list")}
  end

  defp invalid_request_date(%{data: data}) do
    {:ok, data: put_in(data, [:request_date], "10-28-2019")}
  end

  defp execute(%{data: data}), do: {:ok, results: ContactUs.cast(data)}
end

And now an example of how we might use it.

Given the form data from the customer’s request, use the ContactUs.cast function to fully validate the data. If data from the form is not valid? we can use errors to clearly inform the user what was wrong (or perhaps log if source data is from an external API). If state is OK, we can use accessors (like ContactUs.email(contact_data)) to obtain form data without worrying about where it is or about possible future minor structural changes.

defmodule Email do
  @moduledoc """
  Sends emails from our `Contact Us` form.
  """
  alias Email.{Mailer, View}
  alias Email.Schema.ContactUs
  import Bamboo.Email

  @doc """
  Generates and sends an email from a contact_us form submission.
  """
  def send_contact_us_email(contact_us_attrs) do
    contact_us_attrs
    |> ContactUs.cast()
    |> case do
      %{valid?: true} = contact_data ->
        contact_data
        |> contact_us_email()
        |> Mailer.deliver_now()

        {:ok, contact_data}

      %{errors: errors} ->
        {:error, errors}
    end
  end

  @doc false
  def contact_us_email(%ContactUs{} = contact_data) do
    new_email()
    |> to("customerservice@my_company.com")
    |> from("contact-form@email.my_company.com")
    |> put_header("Reply-To", ContactUs.email(contact_data))
    |> subject(ContactUs.subject(contact_data))
    |> render("contact_us_email.html")
  end

  defp render(_data, _template), do: "left as an exercise for the reader"
end

Finally, let’s explore some ways a Circle of Trust can lend a hand.

1. Failures happen. Support Agile’s Fail Fast principle
2. Ensures contracts. We’re all working together. That’s both good and bad. It can be difficult keeping a growing number of developers in synch. You’ll identify problems with production pushes/rollouts sooner.
3. Enables better error messaging for known-bad conditions. “Hey, we aren’t supposed to be getting a nil image URL and we got one”. We can log something sane rather than a huge pattern match failure somewhere inside of an anonymous callback with no idea where the call originated from.
4. Having context and source data provides the opportunity for better customer feedback on failure rather than “oops, something went wrong”.
5. Makes testing easier. Because you know that certain data states can never happen, you don’t have test for those cases.
6. Makes testing easier #2. Data-modules can help create more resilient tests. By abstracting data state to these modules, the tests can be shielded from change when underlying data formats change.
7. Reduces production code. Because you can filter out invalid data-states, you don’t have to write code to handle them.
8. Coding is faster and safer. The declarative nature of these data modules makes writing both production and test code much faster and safer.
9. Refactoring is easier. It should be obvious that it’s simpler to update and test an accessor than to find all client code directly accessing source data. “Their API just changed id to user_id" … yeah, that’ll be a fun grep and Code Review.”
10. Shared business logic. These modules provide a convenient place for common code. For instance, rather than having date format logic sprinkled everywhere in your code, do it here.
11. When combined with Application Layering, you’ll have an appropriate place to provide “living” documentation about your architecture, and its expectations.
12. YAGNI and KISS. We should avoid over-engineered and unneeded “what if” investments in our code. And sometimes the best way to reduce code complexity is to do a better job of domain modeling. However, it is quite rare that you’ll fully understand your data domain right away. A Circle of Trust makes it far easier to make these additions later when things are clearer.

There are many paths to good code. I hope you can see that adding a Circle of Trust allows for faster development, less code, fewer tests, higher quality code, fewer failures, and more helpful error messages. All at the negligible cost of writing a module that encapsulates the data parsing and validation you’re already doing.