CloudWatch Agent on EC2 with Terraform
Although Serverless applications become very popular nowadays, EC2 still commonly used in some uses cases. When coming to EC2 monitoring, AWS comes with native CloudWatch metric for some basic metrics, but exclude memory and disk usage metric. This article going to show you how we can simply enable CloudWatch agents to monitor memory and disk usage metric using Terraform. To make things simple we will provision 1 EC2 using Terraform.
If you want to follow this guide, please make sure you have the following tools installed and configure:-
Prepare Main Terraform Script
Use Visual Studio Code, Create a new working folder and create
main.tf , with fill in the following detail:-
The Terraform script above do 3 things:-
- Create a local variable to load
- Create one EC2 resource with
ap-southeast-1, configure instance profile, and
user_datafrom the local variables.
- Create SSM Parameter resource, and load its value from
Prepare Instance Profile Terraform Script
To make things easier, we will not configure the private key for EC2, but instead, we use Session Manager to login. Session Manager is a fully managed system manager that lets you control your EC2 instance, read here for more detail.
In the same folder create a file
instance_profie.tf and fill in the following content:-
The Terraform script above does 4 things:-
- Create one instance profile, the reference name must be the same as the previous Terraform script.
- Create policy attachment that uses
AmazonEC2RoleForSSMthat allows EC2 to talk to SSM service, and
CloudWatchAgentServerPolicythat allows EC2 to talk to CloudWatch service.
- Create a custom role policy that will allow EC2 to make API call
ssm:GetParameter, the main reason we need to allow this permission again is that we will need CloudWatch agent to load the configuration from SSM service, and the action is using this permission which not include in
- Lastly, the script will create assume role policy for
Create the Cloudwatch Agent Configuration file
At this step, we will create the Cloudwatch Agent Configuration file, the config will instruct the agent on how to pull the logs and metric.
Create a file
cw_agent_config.json under root folder with the following content:-
The JSON above configure the agent to collect the metric for every 10 seconds and collect
mem_acailable_percent metric, and aggregate the metric with few dimensions. You can read here for more on how to configure the Cloudwatch agent.
Create User Data to initialize the EC2
Lastly, we will create a user data file that initializes the EC2 set up, create a file
user_data.sh under root folder with this content:-
What the script above does a few simple steps as below:-
- Output all log to the specified location
- Upgrade the machine with
- Configure Cloudwatch agent with download package from AWS and install it
- Lastly run the cloud watch agent with configuring it to use config from SSM, as if you aware of the first Terraform script, we use a template file to load the file, so we can replace ours SSM config value.
Run and verify your
At this step will be simple execute the following command
terraform apply --auto-approve
And you should see its provision one EC2 successfully in your account. Let the EC2 run for a few minutes to generate enough metric, and navigate to AWS Console, go to Cloudwatch, Metrics, you should see custom metric being populated as below:-
Try to navigate the Namespaces, and browse through the metric, you will get
mem_available_percent metric like below:-
Now you try login to the EC2 using the session manager as below:-
- Click on the
Connectbutton in EC2 console
2. Connect to your instance with
3. Input command
cat /var/log/user-data.log into browser SSH terminal and you will see the Cloudwatch setup log.
Lastly, run the command
terraform destroy --auto-approve to clean up your environment.
In this article we use
Terraform to build EC2 with configuring
Session Manager permission and
Cloudwatch Agent to collect other non-default metrics, with extra configuration, we can use
Cloudwatch Agent to push our custom log to
Cloudwatch Log Group . When you configure
Cloudwatch Agent , suggest only collect metrics that important to you because it does not come with free of charge, please refer here to understand the
If you want to clone the complete source, feel free to get it from here.