Amazon Web Services(AWS) reports some good metrics on the console by default, like CPU, but some key metrics like memory usage or disk space are missing; these are important to monitor to ensure instance up-time and health.
In this post we'll look at how we can use CloudWatch to monitor these extended metrics, allowing you to build reports, dashboards, and alerts.
Before we begin, note that standard Amazon CloudWatch usage charges will be applicable for these scripts. For more information, see the Amazon CloudWatch pricing page.
✋ For simplicity, let's assume that we are using EC2 with Amazon Linux operating system. For other operating systems, you can use respective commands to achieve the results.
In order to pass metrics data from EC2 to AWS Cloudwatch, we will have to create a user with the following IAM role access:
Got confused? Just create a policy (say - cloudwatch-ec2-access) with the above permissions. After that, create a user (cloudwatch-stats-user) and attach the created policy(cloudwatch-ec2-access) to the user. Also, store the generated
AWSSecretKey of this user which will be required at a later stage.
SSH to your EC2 instance and perform the following steps:
I generally prefer to keep everything in a separate folder for clarity, though not mandatory. As we are using Amazon Linux 2 AMI, steps will look similar to this:
# current folder /home/ec2-user $ mkdir cloudwatch_logs $ cd cloudwatch_logs
In order to be able to run the AWS scripts, we will have to install some packages. Also, note that the command may change based on your operating system. Check this link for more information.
sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64
Now that we have installed all the packages, we need to download the Perl scripts provided by AWS.
curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O
unzip CloudWatchMonitoringScripts-1.2.2.zip # remove the zip rm CloudWatchMonitoringScripts-1.2.2.zip # move to the unzipped folder cd aws-scripts-mon
The package for the monitoring scripts contains the following files:
CloudWatchClient.pm– Shared Perl module that simplifies calling Amazon CloudWatch from other scripts.
mon-put-instance-data.pl– Collects system metrics on an Amazon EC2 instance (memory, swap, disk space utilization) and sends them to Amazon CloudWatch.
mon-get-instance-stats.pl– Queries Amazon CloudWatch and displays the most recent utilization statistics for the EC2 instance on which this script is executed.
awscreds.template– File template for AWS credentials that stores your access key ID and secret access key.
LICENSE.txt– Text file containing the Apache 2.0 license.
NOTICE.txt– Copyright notice.
As mentioned in the earlier step, amazon provides a template file
awscreds.template which can be used to create a
conf file to store AWS credentials.
# creates a conf file from the template. Make sure the filename is as is cp awscreds.template awscreds.conf
Next, add your AWS
Access ID and
secret key in this file and save it.
We will now verify if everything is in place and works smoothly with following command:
# change the paths according to your folder structure /home/ec2-user/cloudwatch_logs/aws-scripts-mon/mon-put-instance-data.pl --mem-used-incl-cache-buff --mem-util --mem-used --mem-avail --disk-space-util --disk-space-avail --disk-path=/ --verify --verbose
Congratulations! 👏 You have successfully configured Cloudwatch metrics on your EC2 instance. Now one last step is to add it to cron so it can send metrics after every 5 mins.
# Open the crontab file crontab -e # Add the following line and save it # Cloudwatch Monitoring Metrics (AWS) */5 * * * * /home/ec2-user/cloudwatch_logs/aws-scripts-mon/mon-put-instance-data.pl --mem-used-incl-cache-buff --mem-util --mem-used --mem-avail --disk-space-util --disk-space-avail --disk-path=/ --from-cron
Filesystem, InstanceId, MountPath- All your EC2 disk metrics will be available inside this option.
InstanceId- All your EC2 memory metrics will be available inside this option.
- Whenever you reboot or allocate/deallocate disk space, your EC2 disk filesystem path may change and hence you might require to reconfigure any alarms or dashboards you had created for monitoring EC2.
- If you have launched an EC2 instance from an AMI which had these metrics configured, then the above setup is already present. Just clear the cache via command:
- AWS has recently launched
CloudWatch Agentto collect both system metrics and log files from Amazon EC2 instances. Hence, it is recommended to use
CloudWatch Agentto collect metrics and logs in place of these monitoring scripts. However, there might be certain cases where you would require to use these monitoring scripts and this post might help you to configure it.
I hope you enjoyed the post. See ya! until my next post 😋