21 KiB

Raw Permalink Blame History

Video Clipping Service Implementation Plan

1. Goal And Scope

Build a production-grade Java Spring Boot microservice system that accepts very large video files and generates 8-second clips.

The service must avoid loading full video files into application memory. The recommended architecture is an API service for metadata, authorization, upload orchestration, and job creation, plus isolated worker services that run FFmpeg asynchronously.

Assumptions:

Clients are web, mobile, or internal services.
Videos can be multi-GB.
Default output clips are MP4.
Object storage is S3-compatible.
Metadata is stored in PostgreSQL.
Processing runs on Kubernetes.
Job processing is asynchronous and queue-driven.

Non-goals:

Browser-based video editing UI.
Real-time livestream clipping.
Complex timelines, effects, captions, subtitles, or DRM.
Full media asset management.

2. Architecture

Recommended flow:

Client
  -> POST /v1/video-assets
  -> API creates asset + upload session + signed multipart URLs
  -> Client uploads video directly to object storage
  -> Client completes upload
  -> API validates upload metadata and creates a processing job
  -> API publishes job message to queue
  -> Worker receives job
  -> Worker downloads or streams source video
  -> Worker probes video with ffprobe
  -> Worker runs FFmpeg
  -> Worker uploads generated clips to object storage
  -> Worker records clips and job status in database
  -> Client polls job status or receives webhook

Service responsibilities:

API service: authentication, authorization, upload-session creation, signed URL generation, request validation, job creation, status reads, clip listing, and cancellation.
Worker service: durable job consumption, FFmpeg execution, progress updates, retry handling, temporary file cleanup, and output upload.
Database: durable metadata for assets, upload sessions, jobs, clips, and job events.
Queue: durable async job dispatch with retry and dead-letter handling.
Object storage: original videos and generated clips.

Use separate deployable API and worker services. The API is network-bound and latency-sensitive; workers are CPU, disk, and storage-bandwidth intensive.

3. Technology Stack

Recommended greenfield stack:

Java: Java 25 LTS for greenfield if all dependencies support it; Java 21 LTS as the conservative fallback.
Spring Boot: Spring Boot 4.1.x for greenfield.
Web stack: Spring Web MVC for REST APIs.
Security: Spring Security OAuth2 Resource Server with JWT.
Database: PostgreSQL with Flyway migrations.
Queue: AWS SQS for AWS deployments, RabbitMQ for cloud-portable work queues, Kafka only if event streaming is already a platform standard.
Storage: S3-compatible object storage using multipart upload.
Processing: FFmpeg and ffprobe installed in the worker image and invoked through a strict process adapter.
Observability: Spring Boot Actuator, Micrometer, OpenTelemetry, Prometheus, Grafana, and structured JSON logs.
Tests: JUnit 5, Spring Boot Test, Testcontainers, realistic video fixtures, and an S3-compatible local test service.

Version notes:

Spring Boot 4.1.0 requires at least Java 17 and is compatible up to Java 26: https://docs.spring.io/spring-boot/system-requirements.html
OpenJDK publishes production-ready JDK 25 builds: https://jdk.java.net/25/
Spring Boot documents Testcontainers integration for integration tests against real backend services: https://docs.spring.io/spring-boot/reference/testing/testcontainers.html

4. API Contract

All APIs are versioned under /v1. Use ISO-8601 UTC timestamps.

Method	Path	Purpose	Auth	Idempotency
`POST`	`/v1/video-assets`	Create asset and upload session	Owner/admin	`Idempotency-Key`
`POST`	`/v1/video-assets/{assetId}/uploads:complete`	Complete uploaded object registration	Owner/admin	Idempotent by upload id
`POST`	`/v1/video-assets/{assetId}/clip-jobs`	Create clipping job	Owner/admin	`Idempotency-Key`
`GET`	`/v1/clip-jobs/{jobId}`	Get job status	Owner/admin	Safe
`GET`	`/v1/clip-jobs/{jobId}/clips`	List generated clips	Owner/admin	Safe
`POST`	`/v1/clips/{clipId}/download-url`	Create signed clip download URL	Owner/admin	Short-lived
`POST`	`/v1/clip-jobs/{jobId}:cancel`	Cancel queued or running job	Owner/admin	Idempotent
`DELETE`	`/v1/video-assets/{assetId}`	Delete asset and generated outputs	Owner/admin	Idempotent

Create Video Asset

POST /v1/video-assets

Request:

{
  "fileName": "match-final.mp4",
  "contentType": "video/mp4",
  "contentLengthBytes": 12884901888,
  "checksumSha256": "7b3e...",
  "clipProfile": "mp4-h264-aac"
}

Validation:

fileName: required, 1..255 chars, stored as metadata only.
contentType: required, allowed video media type.
contentLengthBytes: required, 1 byte through configured tenant max.
checksumSha256: optional initially, required if the client can provide it.
clipProfile: optional, defaults to mp4-h264-aac.

Response:

{
  "assetId": "vid_01JZABCD123",
  "status": "PENDING_UPLOAD",
  "upload": {
    "uploadId": "upl_01JZABCD456",
    "mode": "multipart",
    "partSizeBytes": 104857600,
    "expiresAt": "2026-07-08T18:00:00Z",
    "parts": [
      {
        "partNumber": 1,
        "url": "https://storage.example/upload-part-1"
      }
    ]
  },
  "createdAt": "2026-07-08T16:00:00Z"
}

Invalid request example:

{
  "fileName": "match-final.exe",
  "contentType": "application/octet-stream",
  "contentLengthBytes": 0
}

Expected result: 400 or 415, depending whether the media type or field validation fails first.

Complete Upload

POST /v1/video-assets/{assetId}/uploads:complete

Request:

{
  "uploadId": "upl_01JZABCD456",
  "parts": [
    {
      "partNumber": 1,
      "etag": "\"abc123\""
    }
  ]
}

Response:

{
  "assetId": "vid_01JZABCD123",
  "status": "UPLOADED",
  "uploadedAt": "2026-07-08T16:20:00Z"
}

Create Clip Job

POST /v1/video-assets/{assetId}/clip-jobs

Request:

{
  "segmentDurationSeconds": 8,
  "accuracyMode": "EXACT",
  "outputContainer": "mp4",
  "overwriteExisting": false
}

Validation:

segmentDurationSeconds: required, 1..3600; default product value is 8.
accuracyMode: required, one of FAST, EXACT.
outputContainer: required, initially mp4.
overwriteExisting: optional, default false.

Response:

{
  "jobId": "job_01JZABCDE89",
  "assetId": "vid_01JZABCD123",
  "status": "QUEUED",
  "progressPercent": 0,
  "createdAt": "2026-07-08T16:25:00Z"
}

Invalid request example:

{
  "segmentDurationSeconds": 0,
  "accuracyMode": "FAST"
}

Expected result: 400.

Get Clip Job

GET /v1/clip-jobs/{jobId}

Response:

{
  "jobId": "job_01JZABCDE89",
  "assetId": "vid_01JZABCD123",
  "status": "RUNNING",
  "accuracyMode": "EXACT",
  "segmentDurationSeconds": 8,
  "progressPercent": 42,
  "clipCount": 18,
  "createdAt": "2026-07-08T16:25:00Z",
  "startedAt": "2026-07-08T16:26:12Z",
  "finishedAt": null
}

List Clips

GET /v1/clip-jobs/{jobId}/clips

Response:

{
  "jobId": "job_01JZABCDE89",
  "clips": [
    {
      "clipId": "clip_01JZABC001",
      "clipIndex": 0,
      "startSeconds": 0,
      "durationSeconds": 8,
      "contentType": "video/mp4",
      "contentLengthBytes": 10485760,
      "createdAt": "2026-07-08T16:30:00Z"
    }
  ]
}

5. Validation And Security

Authentication and authorization:

Require JWT auth for all endpoints.
Enforce tenant and owner checks on assets, jobs, and clips.
Support service-to-service roles for internal processing/admin use.

Upload security:

Generate short-lived signed upload URLs.
Use tenant-scoped object prefixes.
Validate object size and checksum after upload completion.
Store original filenames as metadata only.
Never use user-provided filenames as local paths or object keys.

Media validation:

Check declared content type.
Probe actual media with ffprobe.
Reject unsupported containers/codecs.
Reject corrupt, encrypted, or unprobeable files with 422.

Abuse prevention:

Limit asset creation, upload sessions, clip jobs, and signed URL requests by tenant.
Enforce max active jobs per tenant.
Enforce max input duration and max file size.
Use malware scanning for untrusted external uploads.

Sensitive data must not be logged:

Signed URLs.
Authorization headers.
Raw queue payloads containing object keys.
Internal storage credentials.
Full FFmpeg stderr if it contains paths or user metadata.

6. Video Processing Design

FFmpeg supports segmenting with segment_time, but exact cuts depend on keyframe placement. Fast stream-copy segmentation is efficient but usually cuts on keyframes. Exact 8-second boundaries usually require transcoding and forced keyframes. FFmpeg segment muxer docs: https://ffmpeg.org/ffmpeg-formats.html#segment_002c-stream_005fsegment_002c-ssegment

Fast Mode

Fast mode uses stream copy and is best when keyframe-aligned clips are acceptable.

ffmpeg -hide_banner -y -i input.mp4 \
  -map 0 -c copy \
  -f segment -segment_time 8 \
  -reset_timestamps 1 \
  clips/out_%05d.mp4

Tradeoff:

Pros: much faster, less CPU, cheaper.
Cons: clips may not start exactly every 8 seconds if keyframes are not aligned.

Exact Mode

Exact mode transcodes video and forces keyframes every 8 seconds.

ffmpeg -hide_banner -y -i input.mp4 \
  -map 0 \
  -c:v libx264 -preset veryfast -crf 20 \
  -force_key_frames "expr:gte(t,n_forced*8)" \
  -c:a aac -b:a 128k \
  -f segment -segment_time 8 \
  -reset_timestamps 1 \
  clips/out_%05d.mp4

Tradeoff:

Pros: frame-accurate or near frame-accurate segment boundaries.
Cons: much higher CPU cost, longer processing time, possible generational quality loss.

Processing rules:

The final clip may be shorter than 8 seconds.
Record each clip's startSeconds, durationSeconds, and clipIndex.
Upload clips to a staging prefix first, then promote or mark complete after validation.
Clean partial local and remote output on failure unless retained for debugging by policy.
Store summarized FFmpeg/ffprobe failure information, not raw unsafe logs.

Worker reliability:

Jobs must be idempotent.
Workers must tolerate duplicate queue messages.
Use job-state transitions guarded by optimistic locking.
Extend queue visibility while processing long jobs.
Move repeatedly failing jobs to DLQ and mark the job as FAILED.

AWS SQS visibility timeout and retry guidance: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

7. Data Model

`video_assets`

Field	Type	Notes
`id`	varchar	Public id, e.g. `vid_*`
`tenant_id`	varchar	Required
`owner_id`	varchar	Required
`status`	enum	`PENDING_UPLOAD`, `UPLOADED`, `PROCESSING`, `READY`, `FAILED`, `DELETED`
`original_file_name`	varchar	Metadata only
`content_type`	varchar	Declared/validated media type
`content_length_bytes`	bigint	Required
`checksum_sha256`	varchar	Optional or required by policy
`storage_bucket`	varchar	Internal
`storage_key`	varchar	Internal
`created_at`	timestamptz	UTC
`deleted_at`	timestamptz	Nullable

Indexes:

(tenant_id, created_at)
(owner_id, created_at)
(status, created_at)

`upload_sessions`

Field	Type	Notes
`id`	varchar	Public id, e.g. `upl_*`
`asset_id`	varchar	FK
`provider_upload_id`	varchar	S3 multipart upload id
`status`	enum	`OPEN`, `COMPLETED`, `ABORTED`, `EXPIRED`
`part_size_bytes`	bigint	Required
`expires_at`	timestamptz	UTC
`completed_at`	timestamptz	Nullable

`clip_jobs`

Field	Type	Notes
`id`	varchar	Public id, e.g. `job_*`
`asset_id`	varchar	FK
`tenant_id`	varchar	Required for filtering
`status`	enum	`QUEUED`, `RUNNING`, `SUCCEEDED`, `FAILED`, `CANCEL_REQUESTED`, `CANCELLED`
`accuracy_mode`	enum	`FAST`, `EXACT`
`segment_duration_seconds`	int	Default 8
`attempt_count`	int	Retry tracking
`progress_percent`	int	`0..100`
`error_code`	varchar	Nullable
`error_message`	text	Sanitized
`created_at`	timestamptz	UTC
`started_at`	timestamptz	Nullable
`finished_at`	timestamptz	Nullable

Indexes:

(tenant_id, created_at)
(asset_id, created_at)
(status, created_at)

`clips`

Field	Type	Notes
`id`	varchar	Public id, e.g. `clip_*`
`job_id`	varchar	FK
`asset_id`	varchar	FK
`clip_index`	int	Starts at 0
`start_seconds`	numeric	Start position
`duration_seconds`	numeric	May be less than 8 for final clip
`storage_bucket`	varchar	Internal
`storage_key`	varchar	Internal
`content_length_bytes`	bigint	Required
`checksum_sha256`	varchar	Optional
`created_at`	timestamptz	UTC

Constraint:

Unique (job_id, clip_index).

`job_events`

Field	Type	Notes
`id`	bigint	Sequence
`job_id`	varchar	FK
`event_type`	varchar	`QUEUED`, `STARTED`, `PROGRESS`, `FAILED`, etc.
`message`	text	Sanitized
`created_at`	timestamptz	UTC

8. Performance And Scalability

Upload path:

Use direct-to-object-storage upload.
Use multipart upload for large files.
Avoid proxying large video bytes through the API service.
S3 multipart upload allows independent upload and retry of parts: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

Worker scaling:

Scale workers by queue age, queue depth, and CPU saturation.
Start with one FFmpeg process per worker pod, then benchmark.
Use larger worker pods for high-resolution transcoding.
Reserve enough ephemeral storage for input plus output plus overhead.

Backpressure:

Per-tenant max active jobs.
Global max queued jobs.
Queue visibility timeout aligned with expected processing time.
DLQ after a configured retry count.

Benchmark dimensions:

Input size.
Duration.
Resolution.
Codec.
Frame rate mode.
FAST vs EXACT.
Object-storage region and bandwidth.
Worker CPU and disk profile.

9. Error Model

Use application/problem+json for all non-2xx responses.

Validation error example:

{
  "type": "https://api.example.com/problems/validation-error",
  "title": "Invalid request",
  "status": 400,
  "detail": "segmentDurationSeconds must be between 1 and 3600",
  "instance": "/v1/video-assets/vid_01JZABCD123/clip-jobs",
  "errors": [
    {
      "field": "segmentDurationSeconds",
      "reason": "must be between 1 and 3600"
    }
  ]
}

Status mapping:

Status	Meaning
`400`	Malformed JSON or invalid request fields
`401`	Missing or invalid token
`403`	Authenticated but not allowed for tenant/asset/job
`404`	Asset, job, or clip not found
`409`	Invalid state transition, duplicate active job, conflict
`413`	File too large
`415`	Unsupported media type
`422`	Uploaded object is not a processable video
`429`	Rate or quota exceeded
`500`	Unexpected server error
`503`	Storage, queue, or processing capacity temporarily unavailable

10. Testing Strategy

Unit tests:

Request validation.
Domain state transitions.
Idempotency-key behavior.
Object-key generation.
Job progress calculations.

Controller/API tests:

Auth success/failure.
Tenant isolation.
Request and response schemas.
application/problem+json error mapping.
Idempotency behavior.

Integration tests:

PostgreSQL via Testcontainers.
Queue via Testcontainers or local emulator.
S3-compatible object storage via Testcontainers.
Worker duplicate message handling.
DLQ path.

FFmpeg tests:

Short video shorter than 8 seconds.
Video exactly 8 seconds.
Long video with final partial clip.
Video with no audio.
Video with variable frame rate.
Corrupt file.
Unsupported codec.
FAST mode duration tolerance.
EXACT mode boundary accuracy.

Load tests:

Upload-session creation throughput.
Job creation throughput.
Queue-to-worker throughput.
Worker CPU/disk saturation.
Object-storage download/upload throughput.

End-to-end test:

Create asset
 -> upload fixture video
 -> complete upload
 -> create clip job
 -> worker processes job
 -> poll job status
 -> list clips
 -> create signed download URL
 -> verify clip object exists

11. Deployment And Operations

Kubernetes:

Deploy API and worker separately.
API has horizontal autoscaling by CPU/request latency.
Workers scale by queue depth, queue age, and CPU.
Use resource requests and limits.
Define ephemeral storage requests/limits for workers.
Use separate service accounts and least-privilege IAM.

Health checks:

API readiness: database, queue publisher, storage signer availability.
API liveness: process health.
Worker readiness: database, queue consumer, storage, FFmpeg binary check.
Worker liveness: process health, not job success.
Startup probe for slow cold starts.

Kubernetes readiness and liveness probe reference: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Observability:

Expose Actuator health and metrics.
Export metrics through Micrometer.
Use OpenTelemetry tracing for API requests and worker jobs.
Correlate logs by traceId, assetId, jobId, and tenantId.

Spring Boot Actuator and observability references:

Alerts:

Queue age too high.
DLQ messages present.
Job failure rate above threshold.
Worker disk usage high.
FFmpeg exit-code spike.
Object storage errors.
API 5xx rate.
Signed URL generation failures.

Operational jobs:

Abort expired multipart uploads.
Delete expired original videos.
Delete expired clips.
Remove orphaned staging objects.
Redrive DLQ messages after fixes.

12. Project Structure

com.example.videoclips
  api/
    VideoAssetController
    ClipJobController
    dto/
    error/
  application/
    CreateVideoAssetUseCase
    CompleteUploadUseCase
    CreateClipJobUseCase
    CancelClipJobUseCase
    GetClipJobUseCase
  domain/
    VideoAsset
    UploadSession
    ClipJob
    Clip
    JobStatus
    AccuracyMode
  persistence/
    entity/
    repository/
    mapper/
  storage/
    ObjectStoragePort
    S3ObjectStorageAdapter
  queue/
    ClipJobPublisher
    ClipJobConsumer
    ClipJobMessage
  processing/
    FfmpegClipper
    FfprobeAnalyzer
    ProcessRunner
    ClipManifest
  config/
    SecurityConfig
    StorageConfig
    QueueConfig
    ObservabilityConfig
  observability/

Keep the domain independent from Spring framework details. Adapters implement storage, queue, persistence, and FFmpeg process concerns.

13. Implementation Milestones

MVP

Create asset and upload-session API.
Direct S3 multipart upload support.
Complete upload endpoint.
Create clip job endpoint.
Queue publisher and single worker.
FFmpeg FAST mode.
Job status and clip listing.
Basic integration tests.

Production Hardening

EXACT clipping mode.
Idempotency keys.
Retry and DLQ handling.
Tenant quotas.
Full application/problem+json error model.
Cleanup lifecycle jobs.
Security review.

Performance Tuning

Worker CPU/disk benchmarks.
FFmpeg preset benchmarks.
Object-storage bandwidth tests.
Queue visibility-timeout tuning.
Autoscaling policies.
Cost model by video minute.

Operational Readiness

Dashboards.
Alerts.
Runbooks.
Backup and restore validation.
DLQ redrive procedure.
Load-test signoff.

Plan Status

All repository-scoped implementation-plan artifacts are now complete.

Remaining work is execution work outside this repository, primarily:

deploy the documented alerts and dashboard into the target monitoring stack
run a real backup-and-restore drill against infrastructure snapshots
run a real production-like end-to-end load test and record the signoff outcome

14. Open Questions

Must clips be exactly 8 seconds, or is keyframe-aligned clipping acceptable?
What is the maximum input file size?
What input containers and codecs must be supported?
What output codec and container are required?
Which cloud provider and object storage will be used?
Which auth provider will issue JWTs?
What is expected upload volume per minute?
What is expected average and maximum video duration?
How long should originals and generated clips be retained?
Should clip access always use signed URLs?
Are webhook notifications required?
Do jobs need per-tenant priority?
Are videos user-generated and therefore subject to malware/content scanning?

Recommended Default

Use Spring Boot 4.1.x, Java 25 LTS where dependency support is confirmed, PostgreSQL, S3 multipart upload, SQS with DLQ, separate Kubernetes API and worker deployments, FFmpeg in worker containers, FAST and EXACT clipping modes, Spring Boot Actuator/Micrometer/OpenTelemetry, and Testcontainers-backed integration tests.

21 KiB Raw Permalink Blame History

Video Clipping Service Implementation Plan

1. Goal And Scope

2. Architecture

3. Technology Stack

4. API Contract

Create Video Asset

Complete Upload

Create Clip Job

Get Clip Job

List Clips

5. Validation And Security

6. Video Processing Design

Fast Mode

Exact Mode

7. Data Model

video_assets

upload_sessions

clip_jobs

clips

job_events

8. Performance And Scalability

9. Error Model

10. Testing Strategy

11. Deployment And Operations

12. Project Structure

13. Implementation Milestones

MVP

Production Hardening

Performance Tuning

Operational Readiness

Plan Status

14. Open Questions

Recommended Default

21 KiB

Raw Permalink Blame History

`video_assets`

`upload_sessions`

`clip_jobs`

`clips`

`job_events`