Instrument Newrelic for Lambda written in Typescript using Terraform

I have picked up a task to setup some AWS Lambda alerts in Newrelic, and quickly realised that to be able to get some data about Invocations and Errors it would require setting up a Newrelic Agent in Lambda, i.e. not only relying on the Newrelic integration with AWS.

Looking into the installation setup, I found that there are two ways to do it, the first way, which is now a legacy option, was to install newrelic npm package, then import it in the lambda and use a function called setLambdaHandler() to wrap the whole lambda code, see example below.

const newrelic = require('newrelic');

// Other module loads go under the require statement above

module.exports.handler = newrelic.setLambdaHandler((event, context, callback) => {
 // This is your handler function code
  console.log('Lambda executed');
  callback();
});

The above option caused me issues with testing, as the lambda is wrapped by Newrelic function and this requires it to instrument when running the unit test.

The testing part was a painful experience as I had to mock Newrelic in Jest and it proved to be a complete waste of time. Mocking wasn't the issue, but Newrelic package hasn't provided interfaces that could easily be mocked, and to mock the whole Newrelic was not an option as it was wrapping the whole lambda code.

Having said that, if we were to write custom events, then we still need to import newrelic, but we won't need to wrap the whole function, we just need to inject a code in the place where we want to send a custom event data, like the following example:

import newrelic from 'newrelic';
//use the below function call wherever you want to send custom event
 newrelic.recordCustomEvent('DaznCustomEvent', {
  name: `Event name`,
  appName: '<Name of your App>',
  ...
     });

How to instrument Newrelic lambda agent?

Luckily, there is a better option to do it, which is using Newrelic Lambda Layer, this gives us the same thing without having to inject Newrelic module to the code or wrapping up the lambda function

To setup the lambda layer using terraform, check this example

resource "aws_lambda_function" "example-lambda" {
  function_name    = local.lambda_name
  filename         = "LAMBDA-CODE.zip"
  role             = <ROLE-ARN>

  layers = [arn:aws:lambda:us-west-2:451483290750:layer:NewRelicNodeJS16X:40]

  handler     = "newrelic-lambda-wrapper.handler"
  runtime     = "nodejs16.x"

   environment {
    variables = {
    NEW_RELIC_LAMBDA_HANDLER              = "index.handler"
    NEW_RELIC_ACCOUNT_ID                  = <Your_Newrelic_Account>
    NEW_RELIC_LICENSE_KEY                 = <Your_Newrelic_License_Key>
    NEW_RELIC_TRUSTED_ACCOUNT_KEY         = <Your_Newrelic_Trusted_Account_Key>
    NEW_RELIC_LAMBDA_EXTENSION_ENABLED    = true
    NEW_RELIC_DISTRIBUTED_TRACING_ENABLED = true
    NEW_RELIC_NO_CONFIG_FILE              = true
    AWS_NODEJS_CONNECTION_REUSE_ENABLED   = 1
    NEW_RELIC_APP_NAME                    = local.lambda_name
    }
  }
}

Find the ARN of the lambda layers that matches your system here in this page: https://layers.newrelic-external.com/

It is important to provide the right api and licence keys to the lambda through the environment variables shown in the above example

Once you done the above, you can now create your alerts in Newrelic using terraform.

Check the terraform documentation which should be easy to follow comprehensive about each resource

Creating an alert policy

https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/alert_policy

Creating an alert condition, there are more than one type, but for lambda we need the newrelic_nrql_alert_condition alert condition

https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/nrql_alert_condition

Hope the above is useful for anyone wants to setup Newrelic in lambda