Serverless background tasks with OpenJS Architect - Part 3

Queues example

@queues are very similar to @events because they also allow for asynchronous message processing. @queues will provision an AWS SQS queue and register a Lambda function to handle messages that are sent to the queue. There are notable differences between @queues and @events. While @events push messages to all of its subscribers, @queues will poll for messages. Queues work on the first message in the queue before moving onto the next. Queues will also keep retrying until it is delivered for up to 4 days.

Let's make an example message queue by starting with a fresh Architect project

npm init @architect ./arc-queues-app
cd arc-queues-app
npm install @architect/functions

Open up the app.arc file and modify the manifest to include our @queues function as follows:

# app.arc
@app
arc-queues-app

@http
get /

@queues
account-signup

After modifying the app.arc file you can run arc init from the project root to scaffold the function folders with Architect.

The queue function uses SQS as an event source. You can use this pattern to move data between Lambda functions and using the queue as a temporary data store. To write a queue function, make a new file in /src/queues

// src/queues/account-signup/index.js
exports.handler = async function queue (event) {
  event.Records.forEach(record => {
    const { body } = record;
    console.log(body);
  });
  return {};
}

Now we can modify our get-index function to publish a message as follows:

// src/http/get-index/index.js
let arc = require('@architect/functions')

exports.handler = async function http(req) {
  let name = 'account-signup'
  let payload = {body: req.body, text: 'yolo'}
  await arc.queues.publish({name, payload})
  return {statusCode: 201}
}

Run npm start from the root of the project to start Sandbox. Navigate to http://localhost:3333 and take a look in the terminal for our output.

You should see the @queue event object being logged in the console.

@queue {
  "name": "account-signup",
  "payload": {
    "text": "yolo"
  }
}

This event has name and payload keys which are reflected in the records when the queue is polled by the Lambda. You should also see the output from the account-signup function handler:

{"text":"yolo"}

Let's follow how Architect implements @queues by default with CloudFormation. Run arc deploy --dry-run and open up sam.json. You should see that AccountSignupQueueLambda has an Events source specified by type of "SQS":

"Events": {
          "AccountSignupQueueEvent": {
            "Type": "SQS",
            "Properties": {
              "Queue": {
                "Fn::GetAtt": [
                  "AccountSignupQueue",
                  "Arn"
                ]
              }
            }
          }
        }

Architect also creates an SQS Queue with CloudFormation. Notice below that the queue is set to FIFO, instead of standard. This means this Lambda will try to ensure that it handles all messages in order as they come in batches.

"AccountSignupQueue": {
      "Type": "AWS::SQS::Queue",
      "Properties": {
        "VisibilityTimeout": 5,
        "FifoQueue": true,
        "ContentBasedDeduplication": true
      }
    },

The @architect/functions library has publish() and subscribe() methods that wrap the JSON payload with a compatible Lambda function signature.