Capacity and Scaling Operations: Right-Sizing for Reliability, Quotas, Throttling, Queues and Load Testing
Scaling Patterns
| Pattern | Meaning | AWS features |
|---|---|---|
| Vertical scaling | Bigger instance. Needs a stop (EC2) or a brief outage (RDS) | Instance type change, Aurora Serverless v2 (no restart) |
| Horizontal scaling | More instances | Auto Scaling, ECS service scaling, Lambda concurrency |
| Scheduled | Known patterns | Scheduled actions |
| Predictive | Learn patterns | Predictive scaling |
| Event-based / queue-based | Scale on backlog | SQS depth per instance |
| Caching | Reduce load | CloudFront, ElastiCache, DAX, API Gateway cache |
| Read scaling | Offload reads | Read replicas, Aurora replicas |
Decouple With Queues
- SQS buffers spikes between producers and consumers so a slow backend does not drop requests
- Scale workers on **
ApproximateNumberOfMessagesVisibledivided by worker count** (backlog per instance), not only on CPU - Use DLQs for poison messages and alarm on DLQ depth and
ApproximateAgeOfOldestMessage - Visibility timeout must be longer than processing time. Use long polling to cut cost. FIFO queues for ordering (lower throughput per group)
bash
aws sqs get-queue-attributes --queue-url $Q --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible ApproximateAgeOfOldestMessage
aws cloudwatch put-metric-alarm --alarm-name dlq-not-empty --namespace AWS/SQS --metric-name ApproximateNumberOfMessagesVisible \
--dimensions Name=QueueName,Value=orders-dlq --statistic Maximum --period 60 --evaluation-periods 1 --threshold 0 --comparison-operator GreaterThanThreshold --alarm-actions $SNS
Throttling and Backoff
| Where it happens | Symptom | Fix |
|---|---|---|
| AWS API calls | ThrottlingException, RequestLimitExceeded, Rate exceeded | SDK exponential backoff with jitter, cache results, batch calls, reduce polling, use EventBridge instead of polling |
| API Gateway | 429 Too Many Requests | Raise throttle limits or usage plan, add caching, SQS buffer |
| Lambda | TooManyRequestsException, Throttles metric | Raise account concurrency, reserved concurrency, smooth with SQS |
| DynamoDB | ProvisionedThroughputExceededException | Capacity, on-demand, key design |
| KMS | ThrottlingException on encrypt/decrypt | Data key caching, S3 Bucket Keys, request quota increase |
| S3 | 503 SlowDown | Spread prefixes, retry with backoff |
| CloudWatch/Logs APIs | Throttled PutLogEvents / GetMetricData | Batch, reduce frequency |
python
import random, time
def call_with_backoff(fn, tries=6, base=0.2, cap=10):
for n in range(tries):
try:
return fn()
except Exception as e:
if n == tries - 1:
raise
delay = min(cap, base * 2 ** n)
time.sleep(random.uniform(0, delay)) # full jitter
Service Quotas
- Every service has quotas (limits): some adjustable, some hard. Quotas are per Region and per account
- Service Quotas console/API shows current values and usage, lets you request increases and set automatic quota management
- Create CloudWatch alarms on quota usage (
AWS/Usagenamespace) at 80 percent - Examples: running On-Demand vCPUs per family, Elastic IPs, VPCs per Region, security groups per ENI, EBS storage, Lambda concurrency, ALB targets, CloudFormation stacks, API rates
- Request increases before events such as launches. Some take days. Trusted Advisor Service Limits checks warn at 80 percent
bash
aws service-quotas list-service-quotas --service-code ec2 --query 'Quotas[?contains(QuotaName,`On-Demand Standard`)].[QuotaName,Value]'
aws service-quotas request-service-quota-increase --service-code ec2 --quota-code L-1216C47A --desired-value 512
aws service-quotas list-requested-service-quota-change-history --service-code ec2
Capacity Guarantees
| Need | Option |
|---|---|
| Guarantee EC2 capacity in an AZ for an event | On-Demand Capacity Reservation (billed whether used or not) |
| Guarantee GPU/ML capacity for planned dates | Capacity Blocks for ML |
Avoid InsufficientInstanceCapacity | Flexible instance types and multiple AZs, mixed instances ASG, retry later |
| Pre-provision for fast scale-out | Warm pools, pre-baked AMIs, provisioned concurrency |
Find the Real Bottleneck
Scaling the web tier will not help if the database, a NAT gateway, connection limits or a third-party API is the limit. Look at each tier during load:
| Tier | Signals |
|---|---|
| Load balancer | RejectedConnectionCount, LCU limits, target response time |
| Web/app | CPU, memory, threads, connections, queue length |
| Database | CPU, connections, IOPS, locks, replica lag |
| Cache | Evictions, CPU, connections |
| Network | NAT gateway ErrorPortAllocation, PacketsDropCount, instance bandwidth allowance exceeded (ENA metrics) |
| Downstream APIs | Latency, 429/5xx |
Load Testing and Planning
- Test with tools like Distributed Load Testing on AWS, JMeter, k6 or Locust. Follow AWS testing policy (most testing needs no approval; avoid prohibited activities)
- Ramp up, hold, then break it to learn limits. Record metrics at each step
- Plan capacity using CloudWatch trends (weekly peaks, growth rate), Compute Optimizer recommendations and business forecasts. Keep headroom (for example 30 percent)
- Review capacity monthly and before campaigns
Practice Tasks
- Put an SQS queue between an API and a worker ASG and scale the workers on backlog per instance
- Create a quota usage alarm for vCPU usage
- Write a script that retries with jitter and test it against a throttled API
- Load test a staging stack and identify the first bottleneck
- Request a service quota increase through the CLI
Conclusion
Scale horizontally with auto scaling and queues, absorb spikes with SQS, handle throttling with exponential backoff and jitter, monitor and raise service quotas ahead of time, reserve capacity when guarantees are needed, and always find the real bottleneck with load testing and per-tier metrics.
Quick Summary: patterns; queues; throttling; quotas; capacity guarantees; bottlenecks; load testing.

