Executing long running tasks with AppSync

AWS recently announced a small (but notable) feature for the AppSync service where a data source associated with a Lambda can invoke the function asynchronously. Before this, AppSync was to only process all requests synchronously.

How does it help

The current update opens the door to handling certain situations which weren’t possible earlier (or at least they were not as straight-forward to implement).

Imagine when you have specific AppSync mutations that take more than 30 seconds to process owing to technical constraints. Since you always need to return a synchronous response, the way to deal with this would typically involve offloading the actual processing of the request to another function either by passing the payload via an SQS (provided payload is under 256KB) or doing a direct asynchronous invocation of the other lambda function and then returning a generic response by the Lambda function associated with the resolver. By offloading the actual processing to another function, the 30 second timeout limitation has been handled.

Assuming the caller needs the final result of that long-running task, it would also need to make a subscription call so that it can receive the response once it has been completed. Triggering a mutation call (without an actual data source) is needed to deliver the subscription response.

Specifying “Event” invocation method type now allows calling the function asynchronously. So now the additional step of handing off the actual processing through an SQS/Lambda can be eliminated.

The setup

With in the JS resolver request function, all we need to do is specify the invocationType attribute. You would already be familiar with this when a function is invoked using the AWS SDK/CLI.

The response function of the resolver can return a static response to indicate to the caller that AppSync has received the request for processing.

export function request(ctx) {
    return {
      operation: "Invoke",
      invocationType: "Event",
      payload: ctx,
    };
}

export function response(ctx) {
  return "Received request";
}

Things to keep in mind

Asynchronous lambda invocations cannot have payload sizes more than 256KB. So if your existing synchronous AppSync request has a payload size beyond this value, switching to async mode is not going to be possible.

Likewise, for the response, an AppSync request (query/mutation) can return a payload with a size limit of 5MB.

The flow indicated above assumes the caller needs the result after the long-running task has finished processing. Eventbridge pipes is a great tool leveraging AWS infrastructure for triggering the mutation call whose response is the result for a Subscription request.

With the use of Subscriptions we are able to deliver the final payload, but the payload size cannot exceed 240 KB. This isn’t something new but just to keep in mind that even though AWS enabled asynchronous request processing with AppSync, the final act of delivering a large payload is still a catch.