Autoscaling in Kubernetes with the Horizontal Pod Autoscaler

Traffic changes through the day. The Horizontal Pod Autoscaler (HPA) adds Pods when load rises and removes them when it falls, so you pay for what you use and stay fast at peak.

Prerequisites

• metrics-server installed in the cluster (most managed clusters include it).
• CPU/memory requests set on your containers; utilisation is measured as a percentage of requests.

Scale on CPU

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300     # wait 5 minutes before scaling down
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60

kubectl get hpa api --watch

Choosing a target

• 50-70% CPU leaves headroom for spikes while new Pods start.
• Set minReplicas to at least 2-3 for availability.
• Slow scale-down avoids "flapping" when traffic is bursty.

Beyond CPU

CPU isn't always the right signal. A queue worker should scale on queue length; an API might scale on requests per second. Options:

• HPA with custom or external metrics via a metrics adapter (for example from Prometheus).
• KEDA, an event-driven autoscaler with ready-made scalers for Kafka, RabbitMQ, SQS, Redis, cron schedules and more. It can also scale workers to zero when idle.

Don't forget the nodes

The HPA adds Pods; something must add machines when they don't fit. Enable the Cluster Autoscaler or Karpenter (or your cloud's equivalent), and make sure new Pods start quickly: small images and fast startup make autoscaling effective.