DevOps Project 2: Three-Tier Infrastructure on AWS with Terraform, Ansible and a Load Balancer
Project Goal
Build a production-style web application platform on AWS using only code. A single command should create the whole environment, and another should remove it.
text
Internet
|
+---------v---------+
| Application Load | public subnets (2 AZs)
| Balancer (HTTP) |
+---------+---------+
|
+---------------v----------------+
| Auto Scaling Group (2-4 EC2) | private subnets (2 AZs)
| nginx + app, configured by |
| Ansible / user data |
+---------------+----------------+
|
+---------v---------+
| RDS PostgreSQL | isolated database subnets
+-------------------+
Remote state: S3 bucket + locking Access: SSM Session Manager (no SSH)
Cost and safety: This project creates paid resources (NAT gateway, ALB, RDS, EC2). Use a sandbox account, set a budget alert, choose small sizes and **run terraform destroy as soon as you finish**. RDS and NAT gateways are the most expensive parts.
Repository Layout
text
aws-three-tier/
bootstrap/ # one-time: S3 state bucket
modules/
network/ # VPC, subnets, NAT, routes
alb/ # load balancer, target group, listener
web/ # launch template + ASG + IAM role (SSM)
database/ # RDS subnet group + instance + secret
envs/
dev/
main.tf backend.tf variables.tf outputs.tf dev.tfvars
ansible/
inventories/aws_ec2.yml
roles/web/...
site.yml
.github/workflows/terraform.yml
Step 1: Network Module
hcl
# modules/network/main.tf
data "aws_availability_zones" "available" { state = "available" }
locals {
azs = slice(data.aws_availability_zones.available.names, 0, var.az_count)
}
resource "aws_vpc" "this" {
cidr_block = var.cidr
enable_dns_support = true
enable_dns_hostnames = true
tags = { Name = "${var.name}-vpc" }
}
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
}
resource "aws_subnet" "public" {
count = var.az_count
vpc_id = aws_vpc.this.id
cidr_block = cidrsubnet(var.cidr, 8, count.index)
availability_zone = local.azs[count.index]
map_public_ip_on_launch = true
tags = { Name = "${var.name}-public-${count.index}" }
}
resource "aws_subnet" "private" {
count = var.az_count
vpc_id = aws_vpc.this.id
cidr_block = cidrsubnet(var.cidr, 8, count.index + 10)
availability_zone = local.azs[count.index]
tags = { Name = "${var.name}-private-${count.index}" }
}
resource "aws_subnet" "db" {
count = var.az_count
vpc_id = aws_vpc.this.id
cidr_block = cidrsubnet(var.cidr, 8, count.index + 20)
availability_zone = local.azs[count.index]
tags = { Name = "${var.name}-db-${count.index}" }
}
resource "aws_eip" "nat" {
domain = "vpc"
}
# One NAT gateway keeps cost low in a lab. Use one per AZ in production.
resource "aws_nat_gateway" "this" {
allocation_id = aws_eip.nat.id
subnet_id = aws_subnet.public[0].id
depends_on = [aws_internet_gateway.this]
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
}
}
resource "aws_route_table" "private" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.this.id
}
}
resource "aws_route_table_association" "public" {
count = var.az_count
subnet_id = aws_subnet.public[count.index].id
route_table_id = aws_route_table.public.id
}
resource "aws_route_table_association" "private" {
count = var.az_count
subnet_id = aws_subnet.private[count.index].id
route_table_id = aws_route_table.private.id
}
hcl
# modules/network/variables.tf and outputs.tf
variable "name" { type = string }
variable "cidr" { type = string }
variable "az_count" {
type = number
default = 2
}
output "vpc_id" { value = aws_vpc.this.id }
output "public_subnet_ids" { value = aws_subnet.public[*].id }
output "private_subnet_ids" { value = aws_subnet.private[*].id }
output "db_subnet_ids" { value = aws_subnet.db[*].id }
Step 2: Load Balancer and Web Tier
hcl
# modules/alb/main.tf
resource "aws_security_group" "alb" {
name_prefix = "${var.name}-alb-"
vpc_id = var.vpc_id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
lifecycle { create_before_destroy = true }
}
resource "aws_lb" "this" {
name = "${var.name}-alb"
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = var.public_subnet_ids
}
resource "aws_lb_target_group" "web" {
name = "${var.name}-tg"
port = 80
protocol = "HTTP"
vpc_id = var.vpc_id
health_check {
path = "/health"
healthy_threshold = 2
unhealthy_threshold = 3
interval = 15
}
}
resource "aws_lb_listener" "http" {
load_balancer_arn = aws_lb.this.arn
port = 80
protocol = "HTTP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.web.arn
}
}
output "alb_dns_name" { value = aws_lb.this.dns_name }
output "target_group_arn" { value = aws_lb_target_group.web.arn }
output "alb_sg_id" { value = aws_security_group.alb.id }
output "alb_arn_suffix" { value = aws_lb.this.arn_suffix }
output "tg_arn_suffix" { value = aws_lb_target_group.web.arn_suffix }
hcl
# modules/web/main.tf
data "aws_ssm_parameter" "ami" {
name = "/aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id"
}
resource "aws_iam_role" "web" {
name_prefix = "${var.name}-web-"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{ Effect = "Allow", Principal = { Service = "ec2.amazonaws.com" }, Action = "sts:AssumeRole" }]
})
}
# SSM lets you open a shell and lets Ansible connect without SSH keys or open ports
resource "aws_iam_role_policy_attachment" "ssm" {
role = aws_iam_role.web.name
policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}
resource "aws_iam_instance_profile" "web" {
name_prefix = "${var.name}-web-"
role = aws_iam_role.web.name
}
resource "aws_security_group" "web" {
name_prefix = "${var.name}-web-"
vpc_id = var.vpc_id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
security_groups = [var.alb_sg_id] # only the load balancer may reach the web tier
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
lifecycle { create_before_destroy = true }
}
resource "aws_launch_template" "web" {
name_prefix = "${var.name}-web-"
image_id = data.aws_ssm_parameter.ami.value
instance_type = var.instance_type
vpc_security_group_ids = [aws_security_group.web.id]
iam_instance_profile { name = aws_iam_instance_profile.web.name }
metadata_options {
http_tokens = "required" # IMDSv2 only
http_put_response_hop_limit = 1
}
user_data = base64encode(templatefile("${path.module}/user_data.sh.tftpl", {
db_host = var.db_host
}))
tag_specifications {
resource_type = "instance"
tags = { Name = "${var.name}-web", Role = "web", Environment = var.environment }
}
}
resource "aws_autoscaling_group" "web" {
name_prefix = "${var.name}-web-"
min_size = var.min_size
max_size = var.max_size
desired_capacity = var.min_size
vpc_zone_identifier = var.private_subnet_ids
target_group_arns = [var.target_group_arn]
health_check_type = "ELB"
launch_template {
id = aws_launch_template.web.id
version = "$Latest"
}
instance_refresh {
strategy = "Rolling"
preferences { min_healthy_percentage = 50 }
}
lifecycle { create_before_destroy = true }
}
resource "aws_autoscaling_policy" "cpu" {
name = "${var.name}-cpu-target"
autoscaling_group_name = aws_autoscaling_group.web.name
policy_type = "TargetTrackingScaling"
target_tracking_configuration {
predefined_metric_specification { predefined_metric_type = "ASGAverageCPUUtilization" }
target_value = 60
}
}
bash
# modules/web/user_data.sh.tftpl (bootstrap: install nginx and a simple health page)
#!/bin/bash
set -euxo pipefail
apt-get update -y
apt-get install -y nginx
echo "ok" > /var/www/html/health
echo "Hello from $(hostname). DB host: ${db_host}" > /var/www/html/index.html
systemctl enable --now nginx
Step 3: Database Module
hcl
# modules/database/main.tf
resource "aws_db_subnet_group" "this" {
name = "${var.name}-db"
subnet_ids = var.db_subnet_ids
}
resource "aws_security_group" "db" {
name_prefix = "${var.name}-db-"
vpc_id = var.vpc_id
ingress {
from_port = 5432
to_port = 5432
protocol = "tcp"
security_groups = [var.web_sg_id] # only the web tier
}
}
resource "aws_db_instance" "this" {
identifier = "${var.name}-pg"
engine = "postgres"
instance_class = "db.t4g.micro"
allocated_storage = 20
db_name = "shop"
username = "shop"
manage_master_user_password = true # AWS creates and rotates the password in Secrets Manager
db_subnet_group_name = aws_db_subnet_group.this.name
vpc_security_group_ids = [aws_security_group.db.id]
storage_encrypted = true
backup_retention_period = 1
multi_az = false # true in production
skip_final_snapshot = true # lab only
deletion_protection = false # true in production
}
output "endpoint" { value = aws_db_instance.this.address }
Circular reference: The web security group id is needed by the database module, and the database host by the web module. Create the web security group in its own module (or at the root) first, then pass its id to both.
Step 4: Wire It Together
hcl
# envs/dev/main.tf
provider "aws" {
region = var.region
default_tags { tags = { Project = "three-tier", Environment = "dev", ManagedBy = "terraform" } }
}
module "network" {
source = "../../modules/network"
name = "tt-dev"
cidr = "10.30.0.0/16"
az_count = 2
}
module "alb" {
source = "../../modules/alb"
name = "tt-dev"
vpc_id = module.network.vpc_id
public_subnet_ids = module.network.public_subnet_ids
}
module "web" {
source = "../../modules/web"
name = "tt-dev"
environment = "dev"
vpc_id = module.network.vpc_id
private_subnet_ids = module.network.private_subnet_ids
alb_sg_id = module.alb.alb_sg_id
target_group_arn = module.alb.target_group_arn
db_host = "pending" # replace with module.database.endpoint after splitting the SG
instance_type = "t3.micro"
min_size = 2
max_size = 4
}
output "app_url" { value = "http://${module.alb.alb_dns_name}" }
bash
cd envs/dev
terraform init
terraform fmt -recursive && terraform validate
terraform plan -out=tfplan
terraform apply tfplan
curl $(terraform output -raw app_url)
Result
The load balancer returns a page from one of the web servers. Refresh several times and the hostname changes as traffic is balanced across instances.
Step 5: Configure Servers with Ansible
User data is fine for a first bootstrap. For ongoing configuration (nginx settings, application deployment, hardening) use Ansible with dynamic inventory and SSM so no SSH keys or open ports are needed.
yaml
# ansible/inventories/aws_ec2.yml
plugin: amazon.aws.aws_ec2
regions: [us-east-1]
filters:
tag:Role: web
instance-state-name: running
hostnames: [instance-id]
keyed_groups:
- key: tags.Role
prefix: role
yaml
# ansible/site.yml
- name: Configure web tier
hosts: role_web
become: true
vars:
ansible_connection: amazon.aws.aws_ssm
ansible_aws_ssm_bucket_name: tt-dev-ansible-ssm # S3 bucket Ansible uses to transfer files
ansible_aws_ssm_region: us-east-1
roles:
- web
yaml
# ansible/roles/web/tasks/main.yml
- name: Install packages
ansible.builtin.apt:
name: [nginx, unattended-upgrades, jq]
state: present
update_cache: true
- name: Deploy nginx site
ansible.builtin.template:
src: site.conf.j2
dest: /etc/nginx/sites-available/default
mode: "0644"
notify: Reload nginx
- name: Deploy index page
ansible.builtin.template:
src: index.html.j2
dest: /var/www/html/index.html
mode: "0644"
- name: Ensure nginx is running
ansible.builtin.service:
name: nginx
state: started
enabled: true
bash
cd ansible
ansible-galaxy collection install amazon.aws community.aws
ansible-inventory -i inventories/aws_ec2.yml --graph
ansible-playbook -i inventories/aws_ec2.yml site.yml --check --diff
ansible-playbook -i inventories/aws_ec2.yml site.yml
Step 6: Monitoring and Alarms
hcl
resource "aws_sns_topic" "alerts" { name = "tt-dev-alerts" }
resource "aws_sns_topic_subscription" "email" {
topic_arn = aws_sns_topic.alerts.arn
protocol = "email"
endpoint = var.alert_email
}
resource "aws_cloudwatch_metric_alarm" "alb_5xx" {
alarm_name = "tt-dev-alb-5xx"
namespace = "AWS/ApplicationELB"
metric_name = "HTTPCode_Target_5XX_Count"
dimensions = { LoadBalancer = module.alb.alb_arn_suffix }
statistic = "Sum"
period = 60
evaluation_periods = 3
threshold = 10
comparison_operator = "GreaterThanThreshold"
treat_missing_data = "notBreaching"
alarm_actions = [aws_sns_topic.alerts.arn]
}
resource "aws_cloudwatch_metric_alarm" "unhealthy_hosts" {
alarm_name = "tt-dev-unhealthy-hosts"
namespace = "AWS/ApplicationELB"
metric_name = "UnHealthyHostCount"
dimensions = { TargetGroup = module.alb.tg_arn_suffix, LoadBalancer = module.alb.alb_arn_suffix }
statistic = "Maximum"
period = 60
evaluation_periods = 2
threshold = 0
comparison_operator = "GreaterThanThreshold"
alarm_actions = [aws_sns_topic.alerts.arn]
}
Step 7: CI Workflow for Terraform
yaml
ame: terraform
on:
pull_request:
paths: ["envs/**", "modules/**"]
push:
branches: [main]
paths: ["envs/**", "modules/**"]
permissions:
id-token: write
contents: read
pull-requests: write
jobs:
plan:
runs-on: ubuntu-latest
defaults: { run: { working-directory: envs/dev } }
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: us-east-1
- run: terraform fmt -check -recursive ../..
- run: terraform init -input=false
- run: terraform validate
- uses: aquasecurity/trivy-action@master
with: { scan-type: config, scan-ref: ., severity: "HIGH,CRITICAL", exit-code: "0" }
- run: terraform plan -input=false -out=tfplan
apply:
needs: plan
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: dev # add required reviewers here
defaults: { run: { working-directory: envs/dev } }
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: us-east-1
- run: terraform init -input=false
- run: terraform apply -input=false -auto-approve
Step 8: Test Resilience
- Instance failure: terminate one web instance in the console. The ASG launches a replacement and the ALB keeps serving
- Scale test: run a load generator (
hey or k6) against the ALB and watch the ASG add instances when CPU rises
- Rolling update: change the user data or AMI, apply, and watch the instance refresh replace servers gradually with no downtime
- Drift: change a security group rule by hand, then run
terraform plan to see it detected
Final Step: Clean Up
bash
cd envs/dev
terraform destroy
# then empty and delete the state bucket created by the bootstrap folder
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=three-tier # confirm nothing remains
aws ec2 describe-nat-gateways --filter Name=state,Values=available
Portfolio and Extension Ideas
- Add HTTPS with ACM and a Route 53 record
- Add a bastion-free admin path with SSM Session Manager and audit logs
- Add a CloudWatch dashboard and an AWS Budget in Terraform
- Add Terragrunt or separate staging and prod environments with different sizes
- Replace EC2 with ECS Fargate or EKS and compare operations
- Document the architecture, cost estimate (Infracost) and teardown steps in the README
Conclusion
You built a three-tier AWS platform entirely from code: network, ALB, auto scaling web tier, database and alarms, configured with Ansible over SSM and checked by a CI workflow. Always finish by destroying lab resources.
Quick Summary: network module; ALB -> ASG -> RDS with SG chaining; SSM not SSH; Ansible dynamic inventory; alarms; CI plan/apply; destroy.