Project 2: Patching and Fleet Management With Systems Manager, Maintenance Windows and Compliance Reporting

AWS SysOps Associate 65 min min read Updated: Nov 17, 2026 Advanced
Project 2: Patching and Fleet Management With Systems Manager, Maintenance Windows and Compliance Reporting
Advanced Topic 2 of 3

Goal

Patch a mixed dev and prod fleet every month with scan first, install later, reboot safely, report automatically and with no SSH access.

Design

ItemChoice
TargetingTags: env=dev|prod, patch-group=web|app, os=al2023|windows
Baselinescorp-linux-security: Security + Bugfix, Critical/Important, auto-approve after 3 days. corp-windows: Critical + Security updates after 5 days
ScheduleScan daily (State Manager) → Install on dev second Tuesday 22:00 → Install on prod the following Saturday 22:00 after dev passes
SafetyMaxConcurrency 10%, MaxErrors 1, ASG-backed servers drained by load balancer, snapshot or AMI before patching prod
ReportingPatch compliance dashboard + Resource Data Sync → S3 → Athena + weekly SNS summary

Step 1: Make Everything Managed

bash aws ssm describe-instance-information --query 'InstanceInformationList[?PingStatus!=`Online`].[InstanceId,PingStatus]' # Quick Setup (Host Management): keeps SSM agent updated, collects inventory, enables patch scans # Or create an association manually: aws ssm create-association --name AWS-UpdateSSMAgent --targets Key=InstanceIds,Values=* --schedule-expression "rate(14 days)"

Step 2: Baselines and Patch Groups

bash aws ssm create-patch-baseline --name corp-linux-security --operating-system AMAZON_LINUX_2023 --approved-patches-compliance-level CRITICAL \ --approval-rules 'PatchRules=[{PatchFilterGroup={PatchFilters=[{Key=CLASSIFICATION,Values=[Security,Bugfix]},{Key=SEVERITY,Values=[Critical,Important]}]},ApproveAfterDays=3,ComplianceLevel=CRITICAL}]' aws ssm register-patch-baseline-for-patch-group --baseline-id pb-0aaa --patch-group web aws ssm get-patch-baseline-for-patch-group --patch-group web aws ec2 create-tags --resources i-0abc --tags Key="Patch Group",Value=web
  • Instances use the baseline of their **Patch Group tag** (enable tag use on instances through instance metadata tags if needed), or the default baseline for the OS
  • Use RejectedPatches for a known bad patch and **ApprovedPatches** to force a hotfix

Step 3: Maintenance Windows

bash WID=$(aws ssm create-maintenance-window --name patch-prod --schedule "cron(0 22 ? * SAT *)" --schedule-timezone "Asia/Kolkata" --duration 4 --cutoff 1 --no-allow-unassociated-targets --query WindowId --output text) TID=$(aws ssm register-target-with-maintenance-window --window-id $WID --resource-type INSTANCE --targets Key=tag:env,Values=prod Key=tag:patch-group,Values=web --query WindowTargetId --output text) aws ssm register-task-with-maintenance-window --window-id $WID --targets Key=WindowTargetIds,Values=$TID --task-arn AWS-RunPatchBaseline --task-type RUN_COMMAND \ --max-concurrency 10% --max-errors 1 --priority 1 --task-invocation-parameters '{"RunCommand":{"Parameters":{"Operation":["Install"],"RebootOption":["RebootIfNeeded"]},"ServiceRoleArn":"arn:aws:iam::111111111111:role/SSMMaintenanceRole","NotificationConfig":{"NotificationArn":"arn:aws:sns:ap-south-1:111111111111:ops-alerts","NotificationEvents":["Failed","TimedOut"],"NotificationType":"Invocation"}}}' aws ssm describe-maintenance-window-executions --window-id $WID

Step 4: Protect Production

  1. Before: create an AMI or snapshot (Automation task priority 0 in the same window using AWS-CreateImage)
  2. During: rolling 10 percent at a time. For ASG instances use standby or rely on ALB health checks with deregistration delay
  3. After: a verification task runs a health check script (curl -fsS localhost/health). Failures raise an OpsItem
  4. Rollback option: replace the instance from the previous AMI (instance refresh) or restore the snapshot. Patches cannot be cleanly uninstalled for every OS

Step 5: Compliance and Reporting

bash aws ssm describe-patch-group-state --patch-group web aws ssm describe-instance-patch-states-for-patch-group --patch-group web --query 'InstancePatchStates[].[InstanceId,InstalledCount,MissingCount,FailedCount,Operation]' --output table aws ssm list-compliance-summaries --filters Key=ComplianceType,Values=Patch aws ssm create-resource-data-sync --sync-name fleet-inventory --s3-destination BucketName=ops-ssm-sync,Region=ap-south-1,SyncFormat=JsonSerDe
sql -- Athena over the Resource Data Sync: instances with missing critical patches SELECT resourceid, patchstate, count(*) AS n FROM aws_complianceitem WHERE compliancetype = 'Patch' AND status = 'NON_COMPLIANT' AND severity = 'CRITICAL' GROUP BY resourceid, patchstate ORDER BY n DESC;
  • Create an EventBridge rule on Maintenance Window Execution State-change and Command Status-change for alerts
  • Export a weekly CSV through a scheduled Lambda to S3 and email the summary
  • Use Explorer for a cross-account dashboard and OpsCenter for failures

Step 6: Immutable Alternative

  • For ASG fleets build a golden AMI weekly with EC2 Image Builder (patched by update-linux component), test, publish AMI id to Parameter Store, then run instance refresh. Patch Manager is then used mostly for scan and compliance
  • This avoids long-lived drift and gives a quick rollback by switching the launch template back

Troubleshooting Patching

ProblemFix
Instance missing from the windowNot Online in SSM, tag mismatch, window target selection wrong
Failed with no patches installedNo internet or repository access: allow Amazon Linux repositories through NAT or S3 endpoint, or configure a mirror
Patches installed but compliance shows non-compliantReboot pending, baseline different from expectation, scan not yet run (Scan after install)
Window never startsTime zone and cron, window disabled, role permissions
Application broken after patchRoll back through AMI or snapshot, pin package versions, test in dev first

Conclusion

A safe patching program uses tags, baselines with approval delays, scans first and installs in rate-controlled maintenance windows (dev before prod), takes backups before changes, verifies health afterward, reports with compliance dashboards and Athena, and moves toward immutable AMIs for autoscaled fleets.

Quick Summary: goal; design; managed nodes; baselines; windows; protection; reporting; immutable approach; troubleshooting.

Get Newsletter

Subscibe to our newsletter and we will notify you about the newest updates on Edugators