Use AWS Systems Manager Run Command to install CloudWatch Agent in EC2 instances

#aws #monitoring #cloud

Amazon CloudWatch is a service which monitors our AWS resources and the applications running in real time. With CloudWatch, we have system-wide visibility into resource utilization, application performance, and operational health.

With CloudWatch, we can create alarms that watch metrics and send notifications or automatically make changes to the resources when a threshold is breached. However, the memory and disk metrics are not logged by default. Thus, in order for the memory and disk to be monitored and logged, we have to install the CloudWatch agent.

CloudWatch Agent helps to collect metrics and logs from EC2 instances and on-premises servers and push them to CloudWatch.

In this blog, I will be explaining the steps to be followed to create a CloudWatch alarm for EC2 instances by deploying the CloudWatch Configuration File. Below is the process to install cloudwatch agent and configure custom metrics by leveraging AWS Systems Manager.

Prerequisites:

AWS SSM Agent Installed on EC2
AmazonCloudWatchAgent installed on EC2
IAM Role(I named it “SSMagent”) with the following permissions - CloudWatchAgentServerPolicy and AmazonSSMManagedInstanceCore

Attach IAM Role

To configure each Windows EC2 instance through Systems Manager, you need to attach an IAM role with the above permissions to each EC2 instance.

Select the EC2 instance and click on Actions>Instance Settings>Attach/Replace IAM role. Then select the IAM role “SSMagent” which we have created.

Installing CloudWatch Agent on your Instances
Navigate to the ‘Systems Manager’ service under the account and region you would like to configure

In the navigation pane, choose Run Command and then click on Run a Command

In the Command document list, choose AWS-ConfigureAWSPackage

In the Name field, type AmazonCloudWatchAgent

In the Targets area, choose the instance or multiple instances on which to install the CloudWatch agent. We could see the instances in the target list as instances that have SSM agent installed by default and we attached the role(SSMagent) with necessary permissions.

Leave Version set to latest to install the latest version of the agent.

We can uncheck the output options for S3 if you don’t want to store in an S3 bucket.
Click on Run.
In the next screen, you should see a confirmation message that the command was sent successfully and the status should be Success after a few seconds.

Creating a Parameter Store with CloudWatchAgent configuration Script

Next, you will need to create a parameter store for the CWA configuration so it can be deployed across multiple EC2 instances.

Navigate to the ‘Systems Manager’ service under the account and region you’d like to configure
In the navigation pane, choose Parameter Store and then click Create Parameter

Under the Name field, you can enter any name, but you must use the same name in the next step, for example, I used CWA_config
Create and paste the JSON agent configuration created in the below into the Value field (leave all options at default).

The CW Agent will be configured to define which metric(s) are being sent to a CW Namespace, and other data required. The following JSON represents the minimum requirements.
Also, you can configure optional parameters of the CW Namespace (CWAgent is the default when not defined), and region (default region where data is located in the region where the instance runs).

{
    "metrics": {
        "namespace": "Custom_EC2_CWA",
        "metrics_collected": {
            "cpu": {
                "resources": [
                    "*"
                ],
                "measurement": [
                    {
                        "name": "usage_active",
                        "rename": "CPU_USAGE",
                        "unit": "Percent"
                    }
                ]
            },
            "disk": {
                "measurement": [
                    {
                        "name": "used_percent"
                    }
                ],
                "resources": [
                    "*"
                ]
            },
            "mem": {
                "resources": [
                    "*"
                ],
                "measurement": [
                    {
                        "name": "used_percent",
                        "rename": "MEM_USED_PERCENT",
                        "unit": "Percent"
                    }
                ]
            }
        }
    }
}

Now click Create Parameter — you should see a message saying it has been created and the parameter listed.

Deploying The CloudWatch Configuration File

In the ‘Systems Manager’ navigation pane, choose Run Command and then click on Run a Command
In the Command document list, choose AmazonCloudWatch-ManageAgent
Under Action select configure
Under Mode leave it as ec2
Change the Optional Configuration Source to ssm
Under Optional Configuration Location enter the same name of the parameter you created in the Parameter Store (previous section). In this the parameter is named CWA_config
Optional Restart should be set to Yes (This will restart the CloudWatch agent, not the instance)
In the Targets area, choose the instance or multiple instances on which you want to deploy CloudWatch Configuration on.
Now click on Run, We can see command ID status is success
To confirm that the instance is sending the Memory Metrics, head to the CloudWatch page, click on Metrics and locate the Custom_EC2_CWA (only be created if the process worked successfully). Click on InstanceId — there should be Metrics per instance for all the instances, and thus confirm that the data is collected.