Commit Graph

12 Commits

Author SHA1 Message Date
JSLMPR 4d00b5e32b I added an importable Grafana dashboard at dashboards/video-clipping-overview-grafana.json:1 and the companion setup note in docs/dashboards.md:1. The dashboard covers queue backlog and age, DLQ and processing counts, API
request rate and p95 latency, plus CPU and heap usage using the Prometheus and Micrometer metrics already exposed by the service.
2026-07-10 00:14:13 +02:00
JSLMPR 1b9246bafe I added a reproducible cost-model harness in src/test/java/org/example/videoclips/perf/CostModelBenchmarkHarness.java:1 that turns the checked-in FFmpeg and object-storage benchmark baselines into per-source-minute estimates
under explicit planning assumptions. The checked-in companion doc is docs/cost-model-by-video-minute.md:1, and it now includes the modeled baseline table:

  - exact-veryfast-720p: 0.000754 USD / source min
  - exact-medium-720p: 0.000816 USD / source min
  - exact-veryfast-720p-slow-storage: 0.000802 USD / source min
2026-07-09 19:04:23 +02:00
JSLMPR 632959104b • The next step is done and the plan is updated. Autoscaling policies is now marked complete in docs/video-clipping-service-implementation-plan.md:717.
I added autoscaling-ready queue metrics in src/main/java/org/example/videoclips/observability/DatabaseQueueMetricsBinder.java:13, backed by new queue repository counters, and enabled Prometheus scraping via src/main/
  resources/application.properties:3 plus the Prometheus registry dependency in pom.xml:43. The policy itself is documented in docs/autoscaling-policies.md:1, including worker/API scaling rules and a sample KEDA ScaledObject.
  Coverage for the new gauges is in src/test/java/org/example/videoclips/observability/DatabaseQueueMetricsBinderTest.java:14.
2026-07-09 19:00:31 +02:00
JSLMPR 5565298351 • I completed the next plan item and updated the checklist. Queue visibility-timeout tuning is now marked done in docs/video-clipping-service-implementation-plan.md:716.
The main change is in src/main/java/org/example/videoclips/queue/DatabaseBackedClipJobQueueAdapter.java:52: the DB queue now reclaims stale PROCESSING messages after a configurable visibility timeout instead of leaving them
  stuck forever after a worker crash. I added video-clipping.database-queue.visibility-timeout-ms with a default of 900000 ms in src/main/java/org/example/videoclips/config/VideoClippingProperties.java:171 and src/main/
  resources/application.properties:19, plus coverage for stale-claim recovery in src/test/java/org/example/videoclips/queue/DatabaseBackedClipJobQueueAdapterTest.java:91. I also added the tuning note and sizing rule in docs/
  queue-visibility-timeout-tuning.md:1.
2026-07-09 18:37:57 +02:00
JSLMPR f46055c972 • The next plan item is done and the plan is updated. Object-storage bandwidth tests is now marked complete in docs/video-clipping-service-implementation-plan.md:713.
I added a reproducible benchmark harness in src/test/java/org/example/videoclips/perf/ObjectStorageBandwidthBenchmarkHarness.java:17 that exercises ObjectStoragePort materialization and upload paths across a local-disk
  baseline plus throttled 100/50/25 MiB/s profiles. I also checked in the measured baseline and rerun command in docs/object-storage-bandwidth-benchmark-baseline.md:1. On this workspace, the measured throughput came out at
  roughly 1282/825 MiB/s for local disk, 93/89 MiB/s for the 100 MiB/s cap, 48/47 MiB/s for 50 MiB/s, and 24/24 MiB/s for 25 MiB/s on materialize/upload respectively.
2026-07-09 01:33:24 +02:00
JSLMPR 9ffe9bca5c I continued with the next plan step and updated the plan.
I completed Worker CPU/disk benchmarks by adding a reproducible benchmark harness at src/test/java/org/example/videoclips/perf/WorkerBenchmarkHarness.java:1 and checking in the resulting baseline at docs/worker-cpu-disk-
  benchmark-baseline.md:1. The harness measures synchronous worker wall time, current-thread CPU time, clip count, and local/uploaded output bytes for FAST and EXACT stub-processing scenarios, and writes the generated report to
  target/benchmarks/worker-cpu-disk-benchmark.md.

  The plan now marks that item complete in docs/video-clipping-service-implementation-plan.md:711. Verification also stayed green: mvn -q -Dtest=WorkerBenchmarkHarness test passed, and mvn -q test passed with the default suite
  plus the benchmark harness.
2026-07-09 01:25:04 +02:00
JSLMPR e18c2cd9a9 • I completed the next plan step and updated the plan. Security review is now marked done in docs/video-clipping-service-implementation-plan.md:701.
The security hardening I implemented was tenant isolation across the API surface. X-Tenant-Id is now enforced on asset reads, upload completion, asset/job event reads, clip listing, job reads, cancellation, deletion, and
  signed download URL creation. Cross-tenant access returns 403 via the new src/main/java/org/example/videoclips/application/ForbiddenException.java:1 and handler wiring in src/main/java/org/example/videoclips/api/
  ApiExceptionHandler.java:1. I also stopped exposing providerUploadId from the upload-session read response in src/main/java/org/example/videoclips/application/VideoAssetService.java:224, since that provider identifier is
  internal metadata.

  Verification: mvn -q test passed.

  - TenantQuotaControllerTest: 1/1
  - VideoAssetControllerTest: 11/11
  - RetentionCleanupJobTest: 2/2
  - DatabaseBackedClipJobQueueAdapterTest: 2/2

  The production-hardening section of the plan is now fully checked off.
2026-07-09 01:19:43 +02:00
JSLMPR a660b7c1d8 I completed the next step, Full application/problem+json error model, and updated the plan in docs/video-clipping-service-implementation-plan.md:701.
The main change is in src/main/java/org/example/videoclips/api/ApiExceptionHandler.java:1. It now consistently returns problem details for:

  - validation errors
  - malformed JSON
  - unsupported request content type
  - method not allowed
  - quota/conflict/not-found/media-type domain exceptions
  - unexpected internal errors

  I also extended src/test/java/org/example/videoclips/api/VideoAssetControllerTest.java:1 to assert application/problem+json and cover malformed JSON, unsupported request content type, and method-not-allowed responses.
2026-07-09 01:14:38 +02:00
JSLMPR 1737d8b220 I finished the missing part: retryable DB-queue failures no longer become terminal job failures on the first exception. ClipProcessor now supports non-terminal queue retries, the DB queue adapter reschedules retries and only
marks the job FAILED when the message actually moves to DLQ, and retry/DLQ events are recorded at the job level. The key changes are in src/main/java/org/example/videoclips/processing/ClipProcessor.java:37 and src/main/java/
  org/example/videoclips/queue/DatabaseBackedClipJobQueueAdapter.java:18.

  I also kept the queue tests aligned in src/test/java/org/example/videoclips/queue/DatabaseBackedClipJobQueueAdapterTest.java:1, reran mvn -q test, and the suite passed:

  - TenantQuotaControllerTest: 1/1
  - VideoAssetControllerTest: 7/7
  - RetentionCleanupJobTest: 2/2
  - DatabaseBackedClipJobQueueAdapterTest: 2/2

  The plan is updated accordingly in docs/video-clipping-service-implementation-plan.md:701, with Retry and DLQ handling now marked complete. The next unfinished step is Full application/problem+json error model.
2026-07-09 01:10:53 +02:00
JSLMPR 1bd879f1e2 I completed the next plan step, Tenant quotas, and updated the plan to mark it done in docs/video-clipping-service-implementation-plan.md:701.
The implementation adds tenant-aware assets/jobs plus a configurable per-tenant active-job quota. The main enforcement is in src/main/java/org/example/videoclips/application/VideoAssetService.java:165, backed by tenant fields
  added to the domain/persistence model and migration src/main/resources/db/migration/V6__tenant_quota_support.sql:1. Requests can set X-Tenant-Id on asset creation, quota config lives in src/main/java/org/example/videoclips/
  config/VideoClippingProperties.java:39, and quota violations now return 429 via src/main/java/org/example/videoclips/api/ApiExceptionHandler.java:44.
2026-07-09 01:07:07 +02:00
JSLMPR 8548542248 • I updated the milestone section in docs/video-clipping-service-implementation-plan.md:688 to mark what is already done and to show Retry and DLQ handling as in progress rather than untouched.
I also started that next step in code. The database-backed queue now retries failed jobs with configurable backoff and moves messages to a terminal DLQ status after the configured max attempts in src/main/java/org/example/
  videoclips/queue/DatabaseBackedClipJobQueueAdapter.java:48. The new queue settings live in src/main/java/org/example/videoclips/config/VideoClippingProperties.java:133, and I added focused coverage in src/test/java/org/
  example/videoclips/queue/DatabaseBackedClipJobQueueAdapterTest.java:1.

  Verification: mvn -q test passed.

  - VideoAssetControllerTest: 7/7
  - RetentionCleanupJobTest: 2/2
  - DatabaseBackedClipJobQueueAdapterTest: 2/2
2026-07-09 00:59:22 +02:00
JSLMPR 372d2d6fa6 wip 2026-07-08 22:25:12 +02:00