# Autoscaling Policies

This document captures the implementation and rollout guidance for performance plan item `Autoscaling policies`.

Exposed metrics:

- `/actuator/prometheus` is enabled for scraping.
- `video.clipping.queue.pending`
- `video.clipping.queue.processing`
- `video.clipping.queue.dlq`
- `video.clipping.queue.oldest.pending.age.seconds`

Worker policy:

- Start with one FFmpeg process per worker pod.
- Use `minReplicas: 1` and `maxReplicas: 20`.
- Scale out when either queue depth or queue age indicates backlog:
- `video_clipping_queue_pending > 5` per worker target.
- `video_clipping_queue_oldest_pending_age_seconds > 30`.
- Keep CPU as a secondary safeguard with average utilization near `70%`.

API policy:

- Use a separate HPA from workers.
- Scale API pods on CPU and request latency or concurrent requests if those metrics are available from the ingress/controller layer.
- Start with `minReplicas: 2`, `maxReplicas: 10`, and CPU target near `60%`.

Sizing rationale:

- The worker CPU benchmark and FFmpeg preset benchmark show clip generation is CPU-bound once source staging is local.
- The object-storage bandwidth benchmark shows throughput degradation becomes material below `50 MiB/s`, so queue-age should remain part of the worker policy even when CPU is not yet saturated.
- The visibility-timeout default is conservative, so scaling should respond to queue buildup before stale-claim recovery becomes common.

Suggested worker HPA or KEDA shape:

```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: video-clipping-worker
spec:
  scaleTargetRef:
    name: video-clipping-worker
  minReplicaCount: 1
  maxReplicaCount: 20
  cooldownPeriod: 120
  pollingInterval: 15
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc.cluster.local:9090
        metricName: video_clipping_queue_pending
        threshold: "5"
        query: sum(video_clipping_queue_pending)
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc.cluster.local:9090
        metricName: video_clipping_queue_oldest_pending_age_seconds
        threshold: "30"
        query: max(video_clipping_queue_oldest_pending_age_seconds)
```

Operational note:

- If production uses SQS instead of the database queue, keep the same policy shape but source queue depth and age from native cloud metrics instead of the local Micrometer gauges.