🤓 Introduction
Datadog is a cloud monitoring service provider which helps to track and analyse your cloud services from a health and operations perspective and providing support for debugging issues.
Datadog provides different integrations with multiple cloud providers like AWS, GCP, and Azure. In this blog we will be seeing Datadog integration with AWS using one of the AWS Cloudformation service with the help of Serverless framework.
📄 Pre Requisites
You will need AWS and Datadog account and knowledge of AWS services like IAMRole, Cloudformation and Serverless framework.
Note: Here we are using Serverless framework with TypeScript and
Serverless Composeto deploy multiple different stack.
🛠 Datadog Cloudformation Setup
In order to create Datadog Resources using Cloudformation, Datadog needs Cloudformation Extension Plugins to be activated, so our first step is to start with adding an activation stack to our Serverless application. These extension plugins needs IAM roles to be attached in order to execute the Cloudformation events. You can find more details regarding this in Datadog Amazon Cloudformation Documentation
Note: Here you can find a basic setup guide for Datadog-Cloudformation Integration
We'll create two service stack in the Serverless application, one for the
Extensions Activationwhich contains Cloudformation extension activation (as we'll only deploy it once because we can not deploy it multiple times to activate the same resources) and one for theInfra Resources(To create different Datadog monitors and dashboards).
🧩 Setting Up the Extension Activation Stack
Let's create a IAM Role for the Role Delegation and Extensions Executions Role.
🧮 Setting Up the IAM Role
We are providing access to Datadog AWS account (464622532012) using Role Delegation to our IAMRole with necessary permissions.
DatadogExecutionRole: {
Type: "AWS::IAM::Role",
Properties: {
Path: "/",
RoleName: `DatadogAWSIntegrationRole`,
Description: "IAM Role to integrate Datadog using Cloudformation",
MaxSessionDuration: 8400,
AssumeRolePolicyDocument: {
Version: "2012-10-17",
Statement: [
{
Effect: "Allow",
Principal: {
AWS: "arn:aws:iam::464622532012:root",
},
Action: "sts:AssumeRole",
Condition: {
StringEquals: {
"sts:ExternalId": ["${ssm:/datadog/integration/externalId}"],
},
},
},
{
Effect: "Allow",
Principal: {
Service: [
"hooks.cloudformation.amazonaws.com",
"resources.cloudformation.amazonaws.com",
],
},
Action: "sts:AssumeRole",
},
],
},
Policies: [
{
PolicyName: `DatadogIntegrationRolePolicy`,
PolicyDocument: {
Version: "2012-10-17",
Statement: [
{
Sid: "VisualEditor0",
Effect: "Allow",
Action: [
"health:DescribeAffectedEntities",
"budgets:ViewBudget",
"logs:DeleteSubscriptionFilter",
"states:ListStateMachines",
"states:DescribeStateMachine",
"tag:GetTagValues",
"logs:DescribeSubscriptionFilters",
"cloudtrail:GetTrailStatus",
"codedeploy:List*",
"es:ListTags",
"cloudwatch:Describe*",
"elasticmapreduce:Describe*",
"dynamodb:Describe*",
"rds:Describe*",
"route53:List*",
"elasticloadbalancing:Describe*",
"logs:FilterLogEvents",
"cloudfront:GetDistributionConfig",
"apigateway:GET",
"ses:Get*",
"cloudtrail:LookupEvents",
"fsx:ListTagsForResource",
"support:DescribeTrustedAdvisor*",
"events:CreateEventBus",
"lambda:List*",
"dynamodb:List*",
"s3:GetBucketNotification",
"cloudtrail:DescribeTrails",
"s3:PutBucketNotification",
"organizations:Describe*",
"organizations:List*",
"logs:PutSubscriptionFilter",
"fsx:DescribeFileSystems",
"kinesis:Describe*",
"codedeploy:BatchGet*",
"autoscaling:Describe*",
"s3:GetBucketTagging",
"tag:GetResources",
"logs:DescribeLogStreams",
"rds:List*",
"s3:GetBucketLogging",
"tag:GetTagKeys",
"elasticmapreduce:List*",
"backup:List*",
"sns:Publish",
"elasticache:List*",
"redshift:DescribeLoggingStatus",
"cloudwatch:Get*",
"support:RefreshTrustedAdvisorCheck",
"elasticfilesystem:DescribeFileSystems",
"es:DescribeElasticsearchDomains",
"xray:GetTraceSummaries",
"ecs:List*",
"health:DescribeEvents",
"sqs:ListQueues",
"logs:DescribeLogGroups",
"kinesis:List*",
"ecs:Describe*",
"health:DescribeEventDetails",
"cloudwatch:List*",
"logs:TestMetricFilter",
"elasticfilesystem:DescribeAccessPoints",
"elasticache:Describe*",
"sns:List*",
"xray:BatchGetTraces",
"ec2:Describe*",
"directconnect:Describe*",
"es:ListDomainNames",
"elasticfilesystem:DescribeTags",
"s3:ListAllMyBuckets",
"cloudfront:ListDistributions",
"redshift:DescribeClusters",
"s3:GetBucketLocation",
"lambda:GetPolicy",
"secretsmanager:CreateSecret",
"secretsmanager:DeleteSecret",
"secretsmanager:DescribeSecret",
],
Resource: "*",
},
],
},
},
],
},
}
Note: This role will work as extension execution role as well, if you need to have separate roles than you can create separate roles as well.
Now we'll create four AWS::CloudFormation::TypeActivation resources to attach to Activation Stack which will include.
-
Datadog::Dashboards::Dashboard- to create and manage Datadog Dashboards -
Datadog::Monitors::Monitor- to create and manage Datadog Monitors -
Datadog::Monitors::Downtime- to create and manage Monitors Downtime -
Datadog::Integrations::AWS- to create integration between AWS and Datadog
Please refer to the Cloudformation Extension - Activation Guide on how to activate the Cloudformation extensions
If we deploy this Activation Stack we will be able to create the IAMRole along with the Cloudformation Extension Activation. (Note: We'll need to deploy this stack only once)
To use the activated extensions, you will need to set the Type Configurations for each extensions plugins using below code block. Also helpful if you are creating different stages and account for deployment with same AWS account.
aws cloudformation set-type-configuration --type-name "Datadog::Monitors::Monitor" --type RESOURCE --configuration-alias ConfigurationName --configuration '{"DatadogCredentials": {"ApiKey": "DATADOG_API_KEY_PROD", "ApplicationKey": "DATADOG_APPLICATION_KEY_PROD", "ApiURL": "https://api.datadoghq.eu" }}'
Now we move on to creating the Datadog Resources such as Integration, Monitors and Dashboards.
🏗 Creating Infra Stack
We'll create a different Serverless service stack to deploy the datadog resources.
⚙️ Datadog AWS Integration
Here you will need to provide your AWS Account ID so that Datadog account can do the integration between the two AWS accounts.
RoleName would be the same as we have created in the Activation Stack before.
Note:-
ExternalIDSecretNameshould be unique in your account Secret Manager and will be used to store the external ID by the extension.
DatadogAWSIntegration: {
Type: "Datadog::Integrations::AWS",
Properties: {
AccountID: "AWS_ACCOUNT_ID", // your AWS account ID
ExternalIDSecretName: "DatadogIntegrationExternalID",
RoleName: "DatadogAWSIntegrationRole",
},
},
📊 Datadog Dashboards
Dashboards are helpful to monitor the services and different widgets for any services. Find more about Dashboards in Datadog Dashboards Documentation
Dashboard: {
Type: "Datadog::Dashboards::Dashboard",
Properties: {
DashboardDefinition: "", // stringified object with dashboard configurations and definitions.
},
},
⏱ Datadog Monitor
Monitors are to create and capture events for any particular services. You can find more about monitors in Datadog Monitors Documentation
Monitor: {
Type: "Datadog::Monitors::Monitor",
Properties: {
Type: "event-v2 alert", // type of the monitor
Query: "defines the query of the monitor to capture specific triggered events",
Name: "monitor-name",
Message: "message body which will be sent to the communication channel",
Tags: [], // array of tags assigned to the monitor
Options: {
NotifyAudit: false,
Locked: false,
EnableLogsSample: false,
Thresholds: {
Critical: 1,
CriticalRecovery: 0.5,
Warning: 0.75,
},
NewHostDelay: 300,
NotifyNoData: false,
IncludeTags: true,
EscalationMessage: "",
},
Priority: 1, // priority level - critical
}
Hope this has been a worthwhile reading! ✨
Furthermore you can explore many more solutions and services that Datadog provides like Logs Forwarding, Cloud Security, Tracing, etc.
Top comments (0)