Payload and parameters validation in Rails

Overview

Here we go again! A lot of text and code in this post 😄 We are going to continue where we stopped last time with focus on input parameters validation. In my opinion, this is the weakest and almost missing part of the standard Rails setup which can be easily improved, polished and extended. We are going to show how with a few tricks and tips one can solve validations once and for all.

In order to read and understand this article you don't necessary need to go through the previous one, but if you have not read first part you may notice some strange details which are not standard for Rails and may look awkward. If you find yourself a bit confused try to go read first part and come back. It explains a lot of basics we use here.

Consider subscribing to my Dev.to blog to to stay in touch! 😉 And feel free to ask questions in the comments section below.

You can also find me here:

• Telegram: https://t.me/kgcodes
• Twitter: https://twitter.com/kimrgrey

See you!

What do we have?

My professional life developed in such a way that most of the time I was working on different kinds of mobile applications mostly in transportation area: taxi, delivery, bike and car sharing, public transport. When you are building a common API based mobile app a lot of your every day work can be boiled down to the following sequence:

1. Take input from mobile
2. Validate this input
3. Store something in database
4. Make it possible to fetch records you created

Of course, you have some business logic and algorithms around that which creates actual value and essence of your application. This logic is what makes it worth money in the end of the day. It is often complicated and requires a lot of thinking to cook and deliver. That is why it is so important to offload most of the routine to your framework. And Rails really shines in such an environment.

Convention over configuration principle saves you from living in the world of making stack traces out of XML files, as we were joking back in the Java days 😄

Framework takes care of a database connection for you. It provides you with very convenient way of creating, validating and accessing your records via ActiveRecord.

You can render and send an e-mail in a few lines of code thanks to ActionMailer. Which I actually rarely use for my projects, but that is a topic for another day 😉

And, of course, asynchronous jobs are taken care of by ActiveJobs.

And so on and so far.

In many aspects Rails decides and does a lot of heavy lifting so you can think about important things. It saves a lot of arguing time as well. I remember good old days of battles about the best way to layout your data objects or structure your configuration files. Such a waste of time!

It is really good to live in a world where boilerplate is solved and be efficient, isn't it?

What do we want?

With time though I realized that there are a few caveats which spoil the party. One of the things almost neglected by Rails is input parameters validation.

Let's say we are developing delivery application for a dark store or dark kitchen. Shortly speaking, we have an app which allows user to pick products, put them into a cart, estimate a price, order instant or scheduled delivery and pay for it.

If you want to create such an app it is necessary to take care of many questions. How do we represent and calculate the price? Should we provide some promo codes? How do we allow service fee or handle surges? How user can sign up and login into the system? How do we manage and keep track of stocks? How picking and packing processes are organized? What should we do if something ordered is actually missing? All of that creates value. All of that is what matters the most. All of that pays your bills.

In addition to hard questions we also should provide our procurement team with ability to manage assortment. Our admins should be able to create products, categorize them, hide and show them in store, fill in descriptions, specify prices, tags and so on. Doesn't matter how big and cool your application is anyway you have to solve simple things too. It all grows up from the back office.

Let's start from creating very simplistic table and model for our products:

# db/migrate/20240309154032_create_products.rb

class CreateProducts < ActiveRecord::Migration[7.1]
  def change
    create_table :products do |t|
      t.string :name, null: false
      t.string :description, null: true
      t.timestamps
    end
  end
end

# app/models/product.rb

class Product < ApplicationRecord
  validates :name, presence: true
end

Now we should make it possible to create products through API requests. Again, obviously oversimplified code which does the job may look like this:

# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  def create
    product = Product.create!(create_params)

    render json: product.as_json(only: [:name, :description])
  end

  private def create_params
    params.permit(:name, :description)
  end
end

Should we try to use it?

curl --location '127.0.0.1:3000/products' \
--header 'Content-Type: application/json' \
--data '{
    "name": "Lindt Excellence Dark 70% Cacao Bar",
    "description": "Extra fine dark chocolate Lindt EXCELLENCE Dark 70% Bar 100g. A finely balanced taste sensation, finished with a hint of fine vanilla."
}'

Response is looking good:

{
    "name": "Lindt Excellence Dark 70% Cacao Bar",
    "description": "Extra fine dark chocolate Lindt EXCELLENCE Dark 70% Bar 100g. A finely balanced taste sensation, finished with a hint of fine vanilla."
}

We should also have at least ability to see the list with pagination, update existing products and delete them. Let's keep it very simple as well:

# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  DEFAULT_LIMIT = 25
  DEFAULT_OFFSET = 0

  def index
    products = Product.order(id: :desc)
      .limit(params[:limit] || DEFAULT_LIMIT)
      .offset(params[:offset] || DEFAULT_OFFSET)

    render json: {
      products: products.as_json(only: [:id, :name, :description])
    }
  end

  def create
    product = Product.create!(create_params)

    render json: product.as_json(only: [:id, :name, :description])
  end

  def show
    product = Product.find(params[:product_id])

    render json: product.as_json(only: [:id, :name, :description])
  end

  def update
    product = Product.find(params[:product_id])
    product.update!(update_params)

    render json: product.as_json(only: [:id, :name, :description])
  end

  def destroy
    product = Product.find(params[:product_id])
    product.destroy!

    render json: {}
  end

  private def create_params
    params.permit(:name, :description)
  end

  private def update_params
    params.permit(:name, :description)
  end
end

We have entire set of standard CRUD operations for our products. But even from this little example it is easy to see capabilities which Rails lacks out of the box.

First, it mixes all actions together. Which is not such big of a problem, but we have to do some naming exercise to differentiate create_params from update_params, for example. Plus constants DEFAULT_LIMIT and DEFAULT_OFFSET which are only relevant for index are present for every action. Core models, such as product, tend to have a lot of methods: update this, update that, toggle, activate, hide, show, etc. More actions you have in one controller harder it is going to find relevant stuff. And more tempting it will be to re-use some parts and bits. Which is not always a great idea.

Second, we have URL parameters, such as product_id for some methods. For others we have payload parameters, such as name or description. And sometimes we have both. Also name is required while description is completely optional.

Unless you can read and understand Ruby in general and Rails models particularly there is no way to know all of that. Code is going to grow and we will have more parameters and rules. Mobile and back office teams would like to know what endpoints do we have, what they should send and in which form. We can't ask them to use source code to find this out, because it is not obvious from the first glance at all. So, in order to provide some insights on our API we gonna need to spend time on documenting it.

Can we improve it a bit and save some time on documenting internal API-s without going too far?

Let's move things around, shall we?

What if for the beginning we isolate every action into separated class. That is not classic Rails approach, but taking into account our module friendly project structure from the previous article it should be possible to do it quite easily and the result should look quite nicely.

We call such classes "handlers". Their role is to take some set of parameters, payload, handle the request and respond with some data. It is important to remember, that our goal here is to improve developer's experience, coding speed and readability without going too far away from vanilla Rails common sense, so that our code won't surprise new people too much.

Here we define a base class for our handlers to share common logic:

# app/application_handler.rb

class ApplicationHandler
  attr_reader :controller

  def initialize(controller)
    @controller = controller
  end

  protected def render(**args)
    controller.render(**args)
  end

  protected def params
    controller.params
  end

  protected def headers
    controller.headers
  end
end

After that we can move our first action out from controller to a handler:

# app/products/handlers/create.rb

module Products
  module Handlers
    class Create < ApplicationHandler
      def handle
        product = Product.create!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end

      private def payload
        params.permit(:name, :description)
      end
    end
  end
end

As you can see, this file is very narrow focused. It covers one specific flow - product creation. It shares zero space with other flows. All methods, names and parameters are scoped. In the same moment, it contains all information required about this specific endpoint. In order to work with product creation flow you need this file and this file only.

I think it is quite obvious how other handlers will look like, so I'll share one more example here just to make a point clear:

# app/products/handlers/index.rb

module Products
  module Handlers
    class Index < ApplicationHandler
      DEFAULT_LIMIT = 25
      DEFAULT_OFFSET = 0

      def handle
        products = Product.order(id: :desc)
          .limit(params[:limit] || DEFAULT_LIMIT)
          .offset(params[:offset] || DEFAULT_OFFSET)

        render json: {
          products: products.as_json(only: [:id, :name, :description])
        }
      end
    end
  end
end

Once we moved all of our actions to handlers we should change the controller which, frankly speaking, doesn't make a lot of work now:

# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  def index
    handler = Products::Handlers::Index.new(self)
    handler.handle
  end

  def create
    handler = Products::Handlers::Create.new(self)
    handler.handle
  end

  def show
    handler = Products::Handlers::Show.new(self)
    handler.handle
  end

  def update
    handler = Products::Handlers::Update.new(self)
    handler.handle
  end

  def destroy
    handler = Products::Handlers::Destroy.new(self)
    handler.handle
  end
end

We could make one extra step here and introduce some DSL or convention, so that calling of handlers is done for us "automagically". But based on my experience it is much better idea to keep a little boilerplate, because it makes whole thing understandable for any new developer joining the team. What we are looking at are just plain simple Ruby classes and methods. Nothing too fancy, too scary or too controversial.

Good job, but this move achieves only one thing. It gives us some namespace for programming and isolates one handler from another. But there is very little improvement in other areas.

Can we make things better? And what about validations?

Finally, validations!

As it was mentioned before, Rails does very good job in solving almost every common web development problem. But somehow it is not true for the most frequent one - input validation.

The best tool we have in Rails for validating parameters is params.require(:product).permit(:name, :description). Which is at very least limited, if not to say "primitive". Without going out of the box it is not easy to make any next step.

Luckily, there is a large pool of community wisdom around and outside of Rails which may help us a lot here. Instead of inventing our own wheel for now we will use one invented before us by others. Pretty much sure you have seen this magic used outside of Hogwarts before: https://dry-rb.org/gems/dry-validation.

As we agreed, each handler represents one endpoint with some set of parameters and it's payload. But while logic is right there, in the handle method, other stuff stays implicit. We can't immediately tell from code what is expected as an input. The idea is to make our contract with client explicit and very much official by using dry-schema right in our handlers.

First, let's add dry-validation into our Gemfile:

gem "dry-validation", "~> 1.10"

Now I'll add a few more methods into AppplicationHandler:

# app/application_handler.rb

def self.payload(&block)
  if block_given?
    klass = Class.new(ApplicationContract) do
      json(&block)
    end

    @payload_contract = klass.new
  end

  @payload_contract
end

def self.params(&block)
  if block_given?
    klass = Class.new(ApplicationContract) do
      params(&block)
    end

    @params_contract = klass.new
  end

  @params_contract
end

Method payload helps us to define a contract for the payload we expect to come in request. It uses helper method json which takes care of type coercion for us. And method params does the same for URL parameters. It uses another helper conveniently called params which coerces data coming from URL query. You can read more about it here and here.

Now we can go to one of the handlers and define what parameters and payload it expects:

# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end

      private def payload
        params.permit(:name, :description)
      end
    end
  end
end

That helps with documentation issues, but sadly contracts are not applied and enforced, so it is easy to break what you agreed on. Let's add some more code into ApplicationHandler to actually use our definitions:

# app/application_handler.rb

attr_reader :controller
attr_reader :params
attr_reader :payload
attr_reader :errors

def initialize(controller)
  @errors = []

  if self.class.params.present?
    result = self.class.params.call(controller.params.to_h)

    if result.failure?
      @errors << result.errors(full: true).to_h
    end

    @params = result.to_h
  end

  if self.class.payload.present?
    result = self.class.payload.call(controller.params.to_h)

    if result.failure?
      @errors << result.errors(full: true).to_h
    end

    @payload = result.to_h
  end

  @controller = controller
end

protected def render(**args)
  controller.render(**args)
end

protected def headers
  controller.headers
end

In the constructor we check if our class has contracts for params and payload defined. Both contracts if present are applied and the result is stored in an appropriate instance variable available as an attribute. In case of validation errors we store relevant messages in another instance variable which we can use later on. In addition we had to remove method params from ApplicationHandler. It is used to call controller.params but now it was replaced by the attribute.

There is one more problems to solve. By default Rails doesn't allow you to call to_h on params because it expects you to explicitly permit specific keys. But in our case contracts are taking care of validations and they are way-way more powerful than build-in tools. So, let's just get rid of that check:

# app/application_controller.rb

class ApplicationController < ActionController::API
  before_action { params.permit! }
end

Handler can now be changed to utilize all of what has been done:

# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        if errors.present?
          render json: { errors: errors }, status: 422
          return
        end

        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end
    end
  end
end

Let's hit our endpoint with some corrupted payload just to test it and see what happens:

curl --location --request PATCH '127.0.0.1:3000/products/7' \
--header 'Content-Type: application/json' \
--data '{
    "name": 100
}'

Here is what response with status 422 Unprocessable Entity looks like:

{
    "errors": [
        {
            "name": [
                "name must be a string"
            ]
        }
    ]
}

That's great! Contracts we defined in handler are not just there, they are actually being used and enforced. Checking errors in every handler is a bit noisy though, so let's move this part out to ApplicationHandler as well:

# app/application_handler.rb

def handle!
  if errors.present?
    render json: { errors: errors }, status: 422
    return
  end

  handle
end

Now instead of calling directly handle method from specific handler controller will call handle! defined in ApplicationHandler. First it checks validation errors and if everything is fine it call down to the handler logic itself.

# app/controllers/products_controller.rb

def update
  handler = Products::Handlers::Update.new(self)
  handler.handle!
end

The result is pretty much the same, but we don't need to remember about checking validation errors every time now. We just have to define a contract, everything else will work by itself:

# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end
    end
  end
end

Here is another handler after adapting it to the latest changes:

# app/products/handlers/index.rb

module Products
  module Handlers
    class Index < ApplicationHandler
      DEFAULT_LIMIT = 25
      DEFAULT_OFFSET = 0

      params do
        optional(:limit).filled(:integer, gt?: 0)
        optional(:offset).filled(:integer, gteq?: 0)
      end

      def handle
        products = Product.order(id: :desc)
          .limit(limit)
          .offset(offset)

        render json: {
          products: products.as_json(only: [:id, :name, :description])
        }
      end

      private def limit
        params.fetch(:limit, DEFAULT_LIMIT)
      end

      private def offset
        params.fetch(:offset, DEFAULT_OFFSET)
      end
    end
  end
end

Conclusion

With just one gem added and a few lines of code written on top of it we managed to solve one of the most annoying problems in Rails applications. Important part here is that we have not moved too far away from default Rails path, so everything looks and feels familiar and can be easily traced through. No magic tricks, just simple code which one can read and understand in ~ 15 minutes.

Next topic we are going to talk about is authentication and authorization. In the next article I'm going to show how we can authenticate users and admins and check their permissions in handlers using quite simple but very powerful approach. Please, subscribe to stay tuned!