Project 1: Deploy a Three-Tier Application With Ingress, Secrets, Storage, HPA and NetworkPolicy
What You Will Build
text
Internet ─► Ingress (shop.lab.local)
├─ / ─► web (nginx static, 2 pods)
└─ /api ─► api (3 pods, HPA 3..8) ─► db (PostgreSQL StatefulSet, 1 pod + 5Gi PVC)
All in namespace "shop" with: ResourceQuota, default-deny NetworkPolicy + allow rules, PDBs, probes
Use any lab cluster (kind with an ingress controller, minikube, or kubeadm VMs). Each step shows the YAML and the checks. Keep all files in a folder shop/ so you can apply them in order and later put them in Git.
| File | Contents |
00-namespace.yaml | Namespace, quota, limit range |
10-db.yaml | Secret, headless Service, StatefulSet |
20-api.yaml | Deployment, Service, HPA, PDB |
30-web.yaml | Deployment, Service |
40-ingress.yaml | Ingress |
50-netpol.yaml | NetworkPolicies |
Step 1: Namespace, Quota and Defaults
yaml
# 00-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: shop
labels:
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/warn: restricted
---
apiVersion: v1
kind: ResourceQuota
metadata: {name: shop-quota, namespace: shop}
spec:
hard:
requests.cpu: "4"
requests.memory: 8Gi
limits.memory: 12Gi
pods: "40"
persistentvolumeclaims: "5"
---
apiVersion: v1
kind: LimitRange
metadata: {name: defaults, namespace: shop}
spec:
limits:
- type: Container
defaultRequest: {cpu: 50m, memory: 64Mi}
default: {memory: 256Mi}
bash
kubectl apply -f 00-namespace.yaml
kubectl get ns shop --show-labels ; kubectl describe quota -n shop
kubectl config set-context --current --namespace=shop
Step 2: The Database
yaml
# 10-db.yaml
apiVersion: v1
kind: Secret
metadata: {name: db-secret, namespace: shop}
type: Opaque
stringData:
POSTGRES_USER: shopuser
POSTGRES_PASSWORD: change-me-lab-only # lab only: use a secret manager in real life
POSTGRES_DB: shopdb
---
apiVersion: v1
kind: Service
metadata: {name: db, namespace: shop}
spec:
clusterIP: None
selector: {app: db}
ports: [{name: pg, port: 5432}]
---
apiVersion: apps/v1
kind: StatefulSet
metadata: {name: db, namespace: shop}
spec:
serviceName: db
replicas: 1
selector:
matchLabels: {app: db}
template:
metadata:
labels: {app: db, tier: db}
spec:
securityContext: {fsGroup: 999}
containers:
- name: postgres
image: postgres:16
ports: [{containerPort: 5432, name: pg}]
envFrom:
- secretRef: {name: db-secret}
env:
- {name: PGDATA, value: /var/lib/postgresql/data/pgdata}
readinessProbe:
exec: {command: ["pg_isready", "-U", "shopuser", "-d", "shopdb"]}
periodSeconds: 5
livenessProbe:
exec: {command: ["pg_isready", "-U", "shopuser", "-d", "shopdb"]}
initialDelaySeconds: 30
periodSeconds: 10
resources:
requests: {cpu: 200m, memory: 256Mi}
limits: {memory: 512Mi}
volumeMounts:
- {name: data, mountPath: /var/lib/postgresql/data}
volumeClaimTemplates:
- metadata: {name: data}
spec:
accessModes: [ReadWriteOnce]
resources: {requests: {storage: 5Gi}}
bash
kubectl apply -f 10-db.yaml
kubectl get sts,pods,pvc -l app=db
kubectl wait --for=condition=ready pod/db-0 --timeout=120s
kubectl exec db-0 -- psql -U shopuser -d shopdb -c "CREATE TABLE IF NOT EXISTS products(id serial primary key, name text); INSERT INTO products(name) VALUES ('keyboard'),('mouse'); SELECT * FROM products;"
Step 3: The API
To keep the project simple, use a tiny API image you control. Any HTTP service works: here the YAML expects an image myorg/shop-api:1.0 that reads DB_HOST, DB_USER, DB_PASSWORD and exposes /healthz, /ready and /products. For a pure practice run you can replace it with nginx and keep the rest of the YAML.
yaml
# 20-api.yaml
apiVersion: apps/v1
kind: Deployment
metadata: {name: api, namespace: shop}
spec:
selector:
matchLabels: {app: api}
strategy:
type: RollingUpdate
rollingUpdate: {maxSurge: 1, maxUnavailable: 0}
template:
metadata:
labels: {app: api, tier: api}
spec:
securityContext:
runAsNonRoot: true
runAsUser: 10001
seccompProfile: {type: RuntimeDefault}
containers:
- name: api
image: myorg/shop-api:1.0
ports: [{containerPort: 8080, name: http}]
env:
- {name: DB_HOST, value: db-0.db.shop.svc.cluster.local}
- name: DB_USER
valueFrom: {secretKeyRef: {name: db-secret, key: POSTGRES_USER}}
- name: DB_PASSWORD
valueFrom: {secretKeyRef: {name: db-secret, key: POSTGRES_PASSWORD}}
startupProbe:
httpGet: {path: /healthz, port: http}
failureThreshold: 20
periodSeconds: 3
readinessProbe:
httpGet: {path: /ready, port: http}
periodSeconds: 5
livenessProbe:
httpGet: {path: /healthz, port: http}
periodSeconds: 10
resources:
requests: {cpu: 100m, memory: 128Mi}
limits: {memory: 256Mi}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: {drop: [ALL]}
---
apiVersion: v1
kind: Service
metadata: {name: api, namespace: shop}
spec:
selector: {app: api}
ports: [{name: http, port: 80, targetPort: http}]
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: {name: api, namespace: shop}
spec:
scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: api}
minReplicas: 3
maxReplicas: 8
metrics:
- type: Resource
resource: {name: cpu, target: {type: Utilization, averageUtilization: 65}}
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: {name: api, namespace: shop}
spec:
minAvailable: 2
selector:
matchLabels: {app: api}
No replicas field: The Deployment omits replicas on purpose because the HPA owns the number of pods. The HPA minReplicas: 3 sets the starting value.
Step 4: The Web Frontend and Ingress
yaml
# 30-web.yaml
apiVersion: apps/v1
kind: Deployment
metadata: {name: web, namespace: shop}
spec:
replicas: 2
selector:
matchLabels: {app: web}
template:
metadata:
labels: {app: web, tier: frontend}
spec:
containers:
- name: nginx
image: nginxinc/nginx-unprivileged:1.27 # runs as non-root on port 8080
ports: [{containerPort: 8080, name: http}]
readinessProbe:
httpGet: {path: /, port: http}
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {memory: 128Mi}
---
apiVersion: v1
kind: Service
metadata: {name: web, namespace: shop}
spec:
selector: {app: web}
ports: [{name: http, port: 80, targetPort: http}]
yaml
# 40-ingress.yaml (use the IngressClass of YOUR installed controller)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: {name: shop, namespace: shop}
spec:
ingressClassName: traefik # example: change to your controller's class name
rules:
- host: shop.lab.local
http:
paths:
- {path: /api, pathType: Prefix, backend: {service: {name: api, port: {number: 80}}}}
- {path: /, pathType: Prefix, backend: {service: {name: web, port: {number: 80}}}}
bash
kubectl apply -f 20-api.yaml -f 30-web.yaml -f 40-ingress.yaml
kubectl get deploy,svc,hpa,pdb,ingress -n shop
kubectl get ingressclass
echo "<ingress-ip> shop.lab.local" | sudo tee -a /etc/hosts
curl -i http://shop.lab.local/ ; curl -i http://shop.lab.local/api/products
Step 5: Lock Down the Network
yaml
# 50-netpol.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: default-deny, namespace: shop}
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: allow-dns, namespace: shop}
spec:
podSelector: {}
policyTypes: [Egress]
egress:
- to:
- namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: kube-system}}
podSelector: {matchLabels: {k8s-app: kube-dns}}
ports: [{protocol: UDP, port: 53}, {protocol: TCP, port: 53}]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: web-in, namespace: shop}
spec:
podSelector: {matchLabels: {app: web}}
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: ingress-ns}} # your ingress controller namespace
ports: [{protocol: TCP, port: 8080}]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: api-in, namespace: shop}
spec:
podSelector: {matchLabels: {app: api}}
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: ingress-ns}}
ports: [{protocol: TCP, port: 8080}]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: api-to-db, namespace: shop}
spec:
podSelector: {matchLabels: {app: api}}
policyTypes: [Egress]
egress:
- to: [{podSelector: {matchLabels: {app: db}}}]
ports: [{protocol: TCP, port: 5432}]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: db-in, namespace: shop}
spec:
podSelector: {matchLabels: {app: db}}
policyTypes: [Ingress]
ingress:
- from: [{podSelector: {matchLabels: {app: api}}}]
ports: [{protocol: TCP, port: 5432}]
This requires a CNI that enforces policies: Use Calico or Cilium. Adjust the ingress controller namespace and labels to your install. After applying, re-test the site, and use a netshoot pod to prove that web cannot reach db.
Step 6: Verify Everything
bash
kubectl get all,pvc,netpol,hpa,pdb -n shop
kubectl top pods -n shop
# Should work
curl -s http://shop.lab.local/api/products
# Should be blocked by policy
kubectl run probe -n shop --labels=app=web --rm -it --image=nicolaka/netshoot --restart=Never -- nc -zv -w3 db 5432
# Load test and watch autoscaling
kubectl run load -n shop --rm -it --image=busybox:1.36 --restart=Never -- sh -c 'while true; do wget -q -O- http://api/products >/dev/null; done' &
kubectl get hpa api -n shop -w
Step 7: Break It on Purpose
| Test | Expected result | What you learn |
kubectl delete pod -l app=api | New pods start. Site stays up (PDB, replicas) | Self-healing |
kubectl delete pod db-0 | Pod returns, same PVC, data still present | StatefulSet identity and storage |
kubectl set image deploy/api api=myorg/shop-api:does-not-exist | New pods ImagePullBackOff, old pods keep serving | Rolling update safety. Fix with kubectl rollout undo deploy/api |
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data | Pods move. The PDB protects api | Maintenance |
Change the readiness path to /nope | Pods become 0/1 and leave the Service | Probes and Services |
Delete the allow-dns policy | API cannot resolve db-0.db and fails | Policies and DNS |
Step 8: Clean Up
bash
kubectl delete namespace shop # removes everything including the PVC
kubectl get pv # check that the PV was removed (reclaim policy Delete)
Improve It (Homework)
- Put the YAML into Git and install it with Kustomize overlays (dev, prod)
- Add TLS with cert-manager
- Add Prometheus metrics and an alert for API errors
- Back up the database with a CronJob that runs
pg_dump to object storage
- Use a database operator instead of a hand-written StatefulSet
Practice Tasks
- Build the project from the YAML and make the site respond through the Ingress
- Prove the data survives deleting
db-0
- Scale load to make the HPA create more API pods
- Apply the NetworkPolicies and prove that web cannot reach db
- Write a README describing how to deploy and test it
Conclusion
You deployed a real three-tier stack: quotas and defaults, a StatefulSet database with a PVC and Secret, an API with probes, HPA and PDB, a web frontend, Ingress routing and default-deny NetworkPolicies. Breaking it on purpose shows how each safety feature works.
Quick Summary: namespace quota; StatefulSet+PVC+Secret; API probes/HPA/PDB; Ingress host+path; default-deny + allow DNS/web/api/db; failure tests.