# Queue Visibility Timeout Tuning

This document captures the implementation and sizing guidance for performance plan item `Queue visibility-timeout tuning`.

Implemented behavior:

- Database-backed queue messages now use a configurable visibility timeout via `video-clipping.database-queue.visibility-timeout-ms`.
- The default is `900000` ms (`15` minutes).
- Messages left in `PROCESSING` past the visibility timeout are reclaimed on the next poll and retried.
- This prevents permanently stuck jobs after worker crashes or process termination between claim and completion.

Sizing rule:

- Start with `visibility-timeout-ms = 3x` the observed `p99` end-to-end job runtime for the dominant workload.
- Include source-object materialization, FFmpeg runtime, clip upload, and persistence overhead.
- Keep `retry-backoff-ms` materially smaller than the visibility timeout so normal retries happen before stale-claim recovery becomes the dominant path.

Current baseline rationale:

- The checked-in FFmpeg and object-storage benchmarks show local workloads completing in seconds, not minutes.
- The database queue does not extend visibility during active processing, so the default is intentionally conservative.
- `15` minutes gives room for slower `EXACT` jobs, degraded object-storage bandwidth, and moderate node contention without causing premature duplicate delivery.

How to validate:

```bash
mvn -q -Dtest=DatabaseBackedClipJobQueueAdapterTest test
```

Operational note:

- If production measurements show long-running jobs regularly exceeding the configured timeout, either raise `visibility-timeout-ms` or implement active lease extension before increasing worker concurrency.