Capacity and Scaling Operations: Right-Sizing for Reliability, Quotas, Throttling, Queues and Load Testing

AWS SysOps Associate 45 min min read Updated: Nov 17, 2026 Intermediate
Capacity and Scaling Operations: Right-Sizing for Reliability, Quotas, Throttling, Queues and Load Testing
Intermediate Topic 3 of 3

Scaling Patterns

PatternMeaningAWS features
Vertical scalingBigger instance. Needs a stop (EC2) or a brief outage (RDS)Instance type change, Aurora Serverless v2 (no restart)
Horizontal scalingMore instancesAuto Scaling, ECS service scaling, Lambda concurrency
ScheduledKnown patternsScheduled actions
PredictiveLearn patternsPredictive scaling
Event-based / queue-basedScale on backlogSQS depth per instance
CachingReduce loadCloudFront, ElastiCache, DAX, API Gateway cache
Read scalingOffload readsRead replicas, Aurora replicas

Decouple With Queues

  • SQS buffers spikes between producers and consumers so a slow backend does not drop requests
  • Scale workers on **ApproximateNumberOfMessagesVisible divided by worker count** (backlog per instance), not only on CPU
  • Use DLQs for poison messages and alarm on DLQ depth and ApproximateAgeOfOldestMessage
  • Visibility timeout must be longer than processing time. Use long polling to cut cost. FIFO queues for ordering (lower throughput per group)
bash aws sqs get-queue-attributes --queue-url $Q --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible ApproximateAgeOfOldestMessage aws cloudwatch put-metric-alarm --alarm-name dlq-not-empty --namespace AWS/SQS --metric-name ApproximateNumberOfMessagesVisible \ --dimensions Name=QueueName,Value=orders-dlq --statistic Maximum --period 60 --evaluation-periods 1 --threshold 0 --comparison-operator GreaterThanThreshold --alarm-actions $SNS

Throttling and Backoff

Where it happensSymptomFix
AWS API callsThrottlingException, RequestLimitExceeded, Rate exceededSDK exponential backoff with jitter, cache results, batch calls, reduce polling, use EventBridge instead of polling
API Gateway429 Too Many RequestsRaise throttle limits or usage plan, add caching, SQS buffer
LambdaTooManyRequestsException, Throttles metricRaise account concurrency, reserved concurrency, smooth with SQS
DynamoDBProvisionedThroughputExceededExceptionCapacity, on-demand, key design
KMSThrottlingException on encrypt/decryptData key caching, S3 Bucket Keys, request quota increase
S3503 SlowDownSpread prefixes, retry with backoff
CloudWatch/Logs APIsThrottled PutLogEvents / GetMetricDataBatch, reduce frequency
python import random, time def call_with_backoff(fn, tries=6, base=0.2, cap=10): for n in range(tries): try: return fn() except Exception as e: if n == tries - 1: raise delay = min(cap, base * 2 ** n) time.sleep(random.uniform(0, delay)) # full jitter

Service Quotas

  • Every service has quotas (limits): some adjustable, some hard. Quotas are per Region and per account
  • Service Quotas console/API shows current values and usage, lets you request increases and set automatic quota management
  • Create CloudWatch alarms on quota usage (AWS/Usage namespace) at 80 percent
  • Examples: running On-Demand vCPUs per family, Elastic IPs, VPCs per Region, security groups per ENI, EBS storage, Lambda concurrency, ALB targets, CloudFormation stacks, API rates
  • Request increases before events such as launches. Some take days. Trusted Advisor Service Limits checks warn at 80 percent
bash aws service-quotas list-service-quotas --service-code ec2 --query 'Quotas[?contains(QuotaName,`On-Demand Standard`)].[QuotaName,Value]' aws service-quotas request-service-quota-increase --service-code ec2 --quota-code L-1216C47A --desired-value 512 aws service-quotas list-requested-service-quota-change-history --service-code ec2

Capacity Guarantees

NeedOption
Guarantee EC2 capacity in an AZ for an eventOn-Demand Capacity Reservation (billed whether used or not)
Guarantee GPU/ML capacity for planned datesCapacity Blocks for ML
Avoid InsufficientInstanceCapacityFlexible instance types and multiple AZs, mixed instances ASG, retry later
Pre-provision for fast scale-outWarm pools, pre-baked AMIs, provisioned concurrency

Find the Real Bottleneck

Scaling the web tier will not help if the database, a NAT gateway, connection limits or a third-party API is the limit. Look at each tier during load:

TierSignals
Load balancerRejectedConnectionCount, LCU limits, target response time
Web/appCPU, memory, threads, connections, queue length
DatabaseCPU, connections, IOPS, locks, replica lag
CacheEvictions, CPU, connections
NetworkNAT gateway ErrorPortAllocation, PacketsDropCount, instance bandwidth allowance exceeded (ENA metrics)
Downstream APIsLatency, 429/5xx

Load Testing and Planning

  • Test with tools like Distributed Load Testing on AWS, JMeter, k6 or Locust. Follow AWS testing policy (most testing needs no approval; avoid prohibited activities)
  • Ramp up, hold, then break it to learn limits. Record metrics at each step
  • Plan capacity using CloudWatch trends (weekly peaks, growth rate), Compute Optimizer recommendations and business forecasts. Keep headroom (for example 30 percent)
  • Review capacity monthly and before campaigns

Practice Tasks

  1. Put an SQS queue between an API and a worker ASG and scale the workers on backlog per instance
  2. Create a quota usage alarm for vCPU usage
  3. Write a script that retries with jitter and test it against a throttled API
  4. Load test a staging stack and identify the first bottleneck
  5. Request a service quota increase through the CLI

Conclusion

Scale horizontally with auto scaling and queues, absorb spikes with SQS, handle throttling with exponential backoff and jitter, monitor and raise service quotas ahead of time, reserve capacity when guarantees are needed, and always find the real bottleneck with load testing and per-tier metrics.

Quick Summary: patterns; queues; throttling; quotas; capacity guarantees; bottlenecks; load testing.

Get Newsletter

Subscibe to our newsletter and we will notify you about the newest updates on Edugators