Building an image processing app with GraphQL and async serverless

TL;DR

• Image processing is best done asynchronously so that the state is not lost when the app is refreshed
• Serverless functions can be used to run the image processing logic
• Realtime GraphQL helps you be in sync with the processing state
• Hasura GraphQL Engine gives instant realtime GraphQL APIs over Postgres
• Source code: client, image processing logic

Introduction

In this post we'll build a basic image processing app that lets you upload an image and add a sepia tone to it asynchronously. We will not go into the code subtleties as this is more of a philosophical rant about how async image processing apps can be built using realtime GraphQL and serverless.

We'll use:

• Hasura GraphQL Engine as a free realtime GraphQL server over Postgres.

Hasura gives you realtime GraphQL APIs over any Postgres database

• An event sourcing system to trigger external webhooks on mutations

Architecture

This app follows the 3factor.app architecture pattern. 3factor app is an architecture pattern for resilient and scalabale fullstack apps. It involves:

• Realtime GraphQL
• Reliable Eventing
• Async serverless

In case of our image processing app, the flow boils down to:

1. The client uploads the image to cloud and inserts the image URL into the database with a GraphQL mutation. The client then watches the changes in the database for that particular image with GraphQL subscriptions.
2. When the image URL has been inserted in the database, Hasura's event system calls an external webhook with the image details
3. The external webhook converts the image to sepia tone, uploads the converted image to cloud and updates the database with the converted image URL
4. The client receives the update when the converted image has been uploaded to the database (GraphQL subscriptions)

Backend

First step is to get a realtime GraphQL server running in the form of Hasura GraphQL Engine. Click here to deploy it to Heroku's free tier with free Postgres (no credit card required).

Data Model

We need only one table for this app.

images (
  id serial not null primary key,
  image_uri text not null,
  converted_image_uri text
)

When you create this table called images, Hasura provides the following root fields in its GraphQL schema:

• images: To query or subscribe to the list of images (comes with clauses such as where, order_by, limit)
• insert_images: To insert one or more images into the images table
• update_images: To update one or more images in the images table
• delete_images: To delete one or more images in the images table

Image processing logic

We need our image processing logic that takes our uploaded image and adds a sepia tone to it. This could be written in any language or framework and deployed on any platform. For example, in NodeJS, you can write this logic using jimp and serverlessify it using Zeit. The code would look something like:

const jimp = require ('jimp');

function convertImage(image) {
    return jimp.read(image.image_uri).then(function(i) {
      return i.sepia().writeAsync(`/tmp/${image.id}.png`)
    })
}

The above function simply takes an object of the following form, converts it to sepia and stores the converted image at /tmp/.png.

{
    "id": 233,
    "image_uri": "https://mycloudbucket.com/image"
}

Once the image has been converted, you also want to update the converted image URI to the database.

const fetch = require('node-fetch');

function updateConvertedImage(image) {
  convertImage(image).then(function() {
    uploadToCloud(`/tmp/${image.id}.png`).then(function(uploadResp) {
      fetch(
        'https://image-processing-app.herokuapp.com/v1alpha1/graphql',
        {
          method: 'POST',
          body: JSON.stringify({
            query: `
              mutation ($id: Int, $converted: String) {
                update_images (
                  _set: { converted_image_uri: $converted }
                  where: { id: { _eq: $id } }
                ) {
                  returning {
                    converted_image_uri
                    id
                  }
                }
              }
            `,
            variables: {
              converted: uploadResp.secure_url,
              id: image.id
            }
          })
        }
      ).then(function(response) { return response.json() })
      .then(function(responseObj) { console.log(responseObj) })
    })
  })
}

Event sourcing

Hasura lets you define event triggers that listen on mutations and invoke an external webhook with the mutation data. We will create one such event trigger that listens on insert_images mutation and calls the webhook that performs the logic discussed above.

With this, our backend is ready.

Frontend

Most of the logic happens on the backend, so the front-end stays fairly clean. The front-end has two screens:

• Upload screen: Uploads image to cloud, inserts the image URL in the database and redirects to the Convert screen with URL param id=<image_id>
• Convert screen: Waits for the image to get processed and shows the converted image

Upload screen

This screen does the following:

• Takes image from user
• Uploads image to cloud
• Inserts this image in the database with GraphQL mutation
• Redirects to the new screen with URL parameter id= where image_id is the unique id of the inserted image

The GraphQL mutation for inserting the image to the database looks like:

mutation ($uri: String) {
    insert_images (
    objects: [{
        image_uri: $uri
        }]
    ) {
    returning {
            id
        image_uri
    }
    }
}

The above mutation inserts the image URI to database and returns the id of the inserted row. Next, you redirect to the Convert screen where you wait for the processed image.

Convert screen

In the convert screen, you look at the id of the image in URL parameters and make a live query to the database with GraphQL subscriptions. The subscription looks like:

subscription ($id: Int) {
  images (
    where: {
      id: {
        _eq: $id
      }
    }
  ) {
    id
    image_uri
    converted_image_uri
  }
}

While rendering the UI, you would check if the received converted_image_uri is null. If it is, you show a loading indicator, or you show the converted image. If done in React with Apollo's Subscription components, it would look something like:

import React from 'react';
import { Subscription } from 'react-apollo';
import gql from 'graphql-tag';

const ConvertScreen = () => {
  const urlParams = new URLSearchParams(window.location.search);
  const id = urlParams.get('id');

  return (
    <Subscription
      subscription={
        gql`
          subscription ($id: Int) {
            images (
              where: {
                id: {
                  _eq: $id
                }
              }
            ) {
              id
              image_uri
              converted_image_uri
            }
          }
        `
      }
      variables={{id}}
    >
      {
        ({data, error, loading}) => {
          if (error) { console.error(error); return <ErrorScreen /> }
          if (loading) return <LoadingScreen />;
          if (data.images.length === 0) {
            return "Invalid image ID";
          }
          if (!data.images[0].converted_image_uri) {
            return <LoadingScreen />
          }
          return (
            <ImageScreen
              original={data.images[0].image_uri}
              converted={data.images[0].converted_image_uri}
            />
          );
        }
      }

    </Subscription>
  )
}

As you see, in the above component:

1. If there is an error in subscription, we render some error UI.
2. If the subscription is in loading state, we show a loading UI.
3. If the subscription response is empty i.e. there is no row in the database where id is equal to the given id, we say that the id in the URL parameters is invalid.
4. If we get a response but the converted_image_uri is null, we assume that the processing is still in progress
5. If we have a valid converted_image_uri, we render it.

Finishing up

We discussed a pattern to build async image processing applications. You could use this architecture to build mostly all kinds of async applications. Check out 3factor.app and hasura.io to know more about building resilient and scalable fullstack applications.

If you have any questions, stack them up in the comments and they'll be answered ASAP.

References

• Hasura docs
• Event Triggers docs
• GraphQL
• Apollo client