Health Check using Terraform !
Consider a use case where EC2 instance is launched and httpd (Apache server) is setup using Terraform using user_data scripts . The question is how to use Terraform to check if the Apache Server is running or not ?
Lot of us are actually using Terraform to create infrastructure and use other tools such Jenkins and or Ansible to configure servers and even have jobs to do functional validation. This requires one to work with different toolsets. However with Terraform check block the validation can be done to certain degree using Terraform itself.
Terraform check is available since v1.5.0
In this post I will share the use case which I have implemented for some of my environments. This is not the best way to do it but is serves the purpose.
My Setup
I am going to use Terraform to achieve following
- Create an EC2 instance (VM in AWS)
- Install Apache Server (httpd) on it
- Check if httpd is running or not
Terraform Code
# providers.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = "ap-south-1"
}
# main.tf
resource "aws_instance" "demo_server" {
ami = "ami-0492447090ced6eb5"
instance_type = "t3.nano"
key_name = "demo"
tags = {
Name = "demo_server"
purpose = "terraform_checks_demo"
author = "amod.kadam"
}
user_data = file("user-data.sh")
}#!/bin/bash
sudo yum update -y
sudo yum install -y httpd.x86_64
sudo systemctl start httpd.service
sudo systemctl enable httpd.service
cd /var/www/html
echo "Hello World" > index.html# outputs.tf
output "public_ip" {
value = aws_instance.demo_server.public_ip
description = "Displays the Public IP of an ec2 instance"
}The above code creates an EC2 instance and using user-data.sh installs httpd and configures the server with index.html file which returns humble Hello World response.
terraform plan
terraform apply -auto-approveIf we look at the output of terraform apply then EC2 instances is created but there is no clue if httpd is running on EC2 instance or not.
Terraform Code with Check block
Now let us introduce the check block in the main.tf to check the status of the httpd server.
# main.tf - added the check block
# check apache status
check "check_apache_status" {
data "http" "apache" {
url = "http://${aws_instance.demo_server.public_ip}"
}
assert {
condition = data.http.apache.status_code == 200
error_message = "Appache response is ${data.http.apache.status_code}"
}
}The check block has a name check_apache_status which uses data block to send the GET request to the provided url.
URL is constructed by getting the public IP of created EC2 instance.
assert block is configured to assert for response code of 200. If the response code is not 200 then error_message is displayed with response code.
If the response code is 200 then it implies httpd server is running. If there is different code then something is not correct.
If we do terraform plan we get the following response.
Terraform sent a GET request to URL (http://65.2.179.156) and the response data is shown in the output with status_code = 200
If we do terraform apply then also we get the same response !
Point to be noted here is that check block gets evaluated during plan as well as apply !
Let us look at the behavior of check block in few scenarios.
Scenario 1 — Stop the httpd Service
I am going to to stop the httpd service by logging into ec2 server.
terraform apply -auto-approve results in following response.
- Terraform apply gave a message Error making request as it is not able to establish connection with server
- Terraform apply still says Apply Complete
- Apply did not fail but gave Warning
- This did not gave any ASSERTION failure as terraform is not able to get any response from the URL ! (Checkout the next scenario !)
- Terraform check block does not block the execution of the terraform resource creation even if the check condition fails !
Scenario 2 — Restart the Service and move index.html
- I have logged into ec2 instance and moved the index.html from /var/www/html folder to $HOME folder
I have modified the url argument of data http block to include index.html
# check apache stataus
check "check_apache_status" {
data "http" "apache" {
url = "http://${aws_instance.demo_server.public_ip}/index.html"
}
assert {
condition = data.http.apache.status_code == 200
error_message = "Appache response is ${data.http.apache.status_code}"
}
}
After doing terraform plan and terraform apply cycle the following was the output.
This time terraform check block resulted in assertion failed and gave the error message as Apache response is 404.
Terraform State Inspection
If you inspect the terraform.state file you should notice that check_results are also captured in the state file.
Scenario 3 — Fix the 404 error by restoring the index.html file
- Restore the index.html back to /var/www/html folder
- Run terraform apply -auto-approve
- Inspect the state file
Note that when the check block assertion is passed then it means it works as it not shown by the terraform ! However the pass status is captured in the terraform state file.
That is all for the Terraform Check block.
Summary
Terraform check block have attempted to fill the gap between post apply and functional validation by providing check semantic.
- Terraform Checks are performed as a last step of terraform plan or apply
- Check blocks are decoupled from the lifecycle of a specific resource or data source
- Check block is a non blocking call for Terraform Execution i.e. failed checks do not block terraform execution
- Check block is the most flexible for validations and can be used to validate input variables, output variables, resources and data sources
- Check block lets you verify assumptions about infrastructure on an ongoing basis instead of just at the time of provisioning
- Health Assessment & Continuous validations can be done using Terraform check block with additional tools in the ecosystem (e.g. Terraform Cloud, automated jobs using Jenkins etc. )
