Strategies to test AWS AppSync VTL templates

AWS AppSync is a serverless GraphQL and Pub/Sub API service that many developers love. Turing the complex job of creating and managing a GraphQL API with real-time capabilities into a few lines of IaC (or clicks in the console) is a great win for AWS serverless developers.

The AWS AppSync enables you to integrate with other AWS services via AWS Lambda functions or VTL mapping templates. Historically it was pretty hard to test the logic written in the VTL templates. As the service matured, AWS added more and more ways to do so, enabling us, developers, to ensure the service we are developing is working correctly.

In this blog post, I will showcase all the strategies for testing AWS AppSync VTL templates I've seen in the wild. This, by no means, is a comprehensive list – these are the ways I'm familiar with. If you know a different way, please feel free to reach out!

You can find the code for all the strategies listed in the article in this GitHub repository.

Testing VTL templates using the AWS SDK

This is a technique I've discovered recently, and it quickly became my favorite. The AppSync SDK exposes the evaluate-mapping-template CLI call for parsing AppSync VTL templates using the AppSync SDK.

When I read about this in this AWS blog post I was thrilled as I was not very pleased with how I've been testing the templates so far (another strategy that I will touch on in this article).

The test boils down to evaluating the VTL template via the AWS SDK, then asserting on the result. You have only one dependency to maintain – the AWS SDK for AppSync, which is a huge win. The following is a snippet that evaluates a VTL template via the AppSync SDK responsible for fetching a user from AWS DynamoDB.

You can find the full source code for this testing strategy here.

import {
  AppSyncClient,
  EvaluateMappingTemplateCommand
} from "@aws-sdk/client-appsync";

const client = new AppSyncClient({});

test("template test", async () => {
  const template = `{
    "version": "2018-05-29",
    "operation": "GetItem",
    "key": {
        "id": $util.dynamodb.toDynamoDBJson($context.arguments.id),
    }
}`;
  const context = JSON.stringify({ arguments: { id: "USER_ID" } });

  const templateEvaluationResult = await client.send(
    new EvaluateMappingTemplateCommand({
      template,
      context
    })
  );

  expect(templateEvaluationResult.evaluationResult).toMatchInlineSnapshot(`
    "{
        \\"version\\": \\"2018-05-29\\",
        \\"operation\\": \\"GetItem\\",
        \\"key\\": {
            \\"id\\": {\\"S\\":\\"USER_ID\\"},
        }
    }"
  `);
});

Since we are using the official AWS SDK, we can be pretty confident that if we were to deploy an API and use this template, this is how the AWS AppSync service would render it internally.

One drawback of this way of testing VTL templates is that the environment in which the test is running has to have access to AWS credentials. That never was a problem for me, but it all depends on your situation, so be mindful of that.

Testing VTL templates using VTL parsers

This is the technique I've used before I learned about the AppSync SDK's ability to render templates. Instead of relying on the SDK, we use libraries the AWS Amplify uses internally to render the template.

The test setup is a bit more involved, but the result is pretty much the same as in the previous example. The following is the snippet rendering the VTL template using Amplify VTL parsers.

You can find the full source code for this testing strategy here.

import velocityUtil from "amplify-appsync-simulator/lib/velocity/util";
import velocityTemplate from "amplify-velocity-template";
import velocityMapper from "amplify-appsync-simulator/lib/velocity/value-mapper/mapper";

test("template test", async () => {
  const template = `{
    "version": "2018-05-29",
    "operation": "GetItem",
    "key": {
        "id": $util.dynamodb.toDynamoDBJson($context.arguments.id),
    }
}`;

  const errors: any[] = [];
  const now = new Date();
  const graphQLResolveInfo = {} as any;
  const appSyncGraphQLExecutionContext = {} as any;

  const vtlUtil = velocityUtil.create(
    errors,
    now,
    graphQLResolveInfo,
    appSyncGraphQLExecutionContext
  );

  const vtlAST = velocityTemplate.parse(template);
  const vtlCompiler = new velocityTemplate.Compile(vtlAST, {
    valueMapper: velocityMapper.map,
    escape: false
  });

  const context = {
    arguments: { id: "USER_ID" },
    args: { id: "USER_ID" }
  };

  const renderedTemplate = vtlCompiler.render({
    util: vtlUtil,
    utils: vtlUtil,
    ctx: context,
    context
  });

  expect(renderedTemplate).toMatchInlineSnapshot(`
    "{
        \\"version\\": \\"2018-05-29\\",
        \\"operation\\": \\"GetItem\\",
        \\"key\\": {
            \\"id\\": {\\"S\\":\\"USER_ID\\"},
        }
    }"
  `);
});

As I mentioned earlier, I'm not a massive fan of this technique. My main gripe is that I have to pull additional dependencies, which I have no confidence are well maintained, to my project.

Another issue I have is that the dependencies might be out of date and yield a template that might be different than the one produced internally inside the AWS AppSync service.

As for positives – since we are not making any calls to AWS, this technique does not require you to have AWS credentials in place. It might be a huge plus for some, but not so much for others.

Testing VTL templates using AWS AppSync simulator

Next up, going up in the complexity spectrum, this technique sits in a weird spot between local emulation and exercising deployed AWS infrastructure.

Instead of rendering the template and asserting its shape, we employ the amplify-appsync-simulator package to process the template and make the request to an actual (or local instance of) AWS service. This strategy stands on the shoulders of the AWS Amplify mocking and testing capabilities.

You can find the full source code for this testing strategy here.

import { AppSyncUnitResolver } from "amplify-appsync-simulator/lib/resolvers";
import {
  AmplifyAppSyncSimulator,
  AmplifyAppSyncSimulatorAuthenticationType,
  RESOLVER_KIND
} from "amplify-appsync-simulator";

test("template test", async () => {
  const simulator = new AmplifyAppSyncSimulator();
  simulator.init({
    dataSources: [
      {
        type: "AMAZON_DYNAMODB",
        name: "dynamodb",
        config: { tableName, endpoint: tableEndpoint }
      }
    ],
    appSync: {
      name: "testAppSyncAPI",
      additionalAuthenticationProviders: [],
      defaultAuthenticationType: {
        authenticationType: AmplifyAppSyncSimulatorAuthenticationType.API_KEY
      }
    },
    schema: {
      content: `
        schema {
          query: Query
        }

        type Query {
          getUser(id: ID!): User!
        }

        type User {
          id: ID!
          name: String!
        }
      `
    }
  });

  const requestTemplate = `{
    "version": "2018-05-29",
    "operation": "GetItem",
    "key": {
        "id": $util.dynamodb.toDynamoDBJson($context.arguments.id)
    }
}`;

  const responseTemplate = `$util.toJson($context.result)`;

  const resolver = new AppSyncUnitResolver(
    {
      requestMappingTemplate: requestTemplate,
      responseMappingTemplate: responseTemplate,
      kind: RESOLVER_KIND.UNIT,
      fieldName: "mockFieldName",
      typeName: "mockTypeName",
      dataSourceName: "dynamodb"
    },
    simulator
  );

  const result = await resolver.resolve(
    "SOURCE",
    { id: "USER_ID" },
    {
      appsyncErrors: []
    },
    {
      fieldNodes: []
    }
  );

  expect(result).toMatchInlineSnapshot(`
    {
      "id": "USER_ID",
      "name": "CREATED_AT_INFRASTRUCTURE_DEPLOY_TIME",
    }
  `);
});

To be perfectly honest, I'm not a massive fan of this testing technique. My biggest problem is the amount of effort required to make the test work. In addition to pulling additional dependencies into my project, I have to either set up a local mock of a given AWS service or deploy a given service before the test run.

Testing VTL templates by making GraphQL requests

And the last testing strategy I've used are the end-to-end tests. Instead of rendering the VTL template or using simulation, one could fire a GraphQL request to the API endpoint and assert the result. These tests usually run for much longer than the ones I've previously talked about but the confidence you gain that your system is working by utilizing end-to-end tests is massive.

The test body is tiny. We shift the complexity away from the test setup to the infrastructure configuration. To use this testing technique effectively, you must already have all the cloud resources related to the AWS AppSync deployed.

You can find the full source code for this testing strategy here.

import { gql, request } from "graphql-request";

test("template test", async () => {
  const query = gql`
    query {
      getUser(id: "USER_ID") {
        id
        name
      }
    }
  `;
  const response = await request(
    endpointUrl,
    query,
    {},
    {
      "x-api-key": endpointAPIkey
    }
  );
  expect(response).toMatchInlineSnapshot(`
    {
      "getUser": {
        "id": "USER_ID",
        "name": "CREATED_AT_INFRASTRUCTURE_DEPLOY_TIME",
      },
    }
  `);
});

This testing strategy is my go-to when I want to exercise the whole system I'm working on.

Closing words

I've used these four AWS AppSync VTL testing strategies with varying degrees of success. Hopefully, after reading this article, you found a technique that fits your needs.

I'm always open to feedback and learning new things. If you are aware of a different way of, directly or indirectly, testing VTL templates – please let me know!

Consider following me on Twitter – @wm.matuszewski if you wish to see more serverless content on your timeline.

Thank you for your precious time.