# Cinematic Video Editing Service Implementation Plan

## Goal

Extend the video clipping service so it can turn a folder or set of existing clips into a new cinematic promotional video.

Primary use case:

- input: about 35 Porsche clips
- output: one cinematic MP4 video
- style: appealing, exciting, premium automotive promo
- additions: music, sound effects, voiceover praising the Porsche, transitions, timing, and color treatment

The expensive AI model should do high-value creative planning only. The service should do the heavy media work with deterministic tools such as FFmpeg. A lower-cost coding model should be able to implement this plan milestone by milestone.

## Core Principle

Do not send full-resolution video through an AI model.

Use the service to extract compact metadata and preview assets:

- durations
- codecs
- dimensions
- frame thumbnails
- contact sheets
- low-resolution preview proxies
- optional audio loudness data
- optional scene or motion scores

Then use AI to generate:

- creative direction
- shot selection
- edit decision list
- voiceover script
- music and sound effect cue list
- final render instructions

The service renders the final video from the plan.

## Non-Goals

- Browser-based nonlinear editor UI.
- Manual timeline editing interface.
- Real-time collaborative editing.
- Full professional color grading system.
- Training custom AI models.
- Passing raw full-resolution video to an LLM.

## Proposed Workflow

```text
Input clips
  -> analyze media with ffprobe and ffmpeg
  -> extract thumbnails and low-resolution proxies
  -> build compact clip manifest
  -> AI creates cinematic storyboard and edit decision list
  -> service validates the plan
  -> service generates or imports voiceover, music, and SFX assets
  -> service renders final MP4 with ffmpeg
  -> service stores render output and manifest
```

## Local AI Director Workflow

The service should support a local workflow where the video editing service runs on the user's machine and an AI coding/chat agent such as Codex or Claude acts as the director.

This mode should not require the AI agent to directly process video files. The service prepares a project folder containing metadata, thumbnails, contact sheets, and a strict prompt. The AI agent reviews those artifacts, writes an `edit-plan.json`, and asks the service to render the result.

Recommended local folder flow:

```text
input/editing/source/
  porsche-drive-001.mp4
  porsche-drive-002.mp4

service scheduler
  -> detects a new source folder or batch
  -> creates output/edit-projects/<project-id>/
  -> analyzes clips
  -> extracts thumbnails and contact sheets
  -> writes analysis.json
  -> writes ai-director-prompt.md
  -> marks project as WAITING_FOR_DIRECTOR

AI director
  -> reads ai-director-prompt.md
  -> reviews contact-sheets/ and thumbnails/
  -> selects the strongest shots
  -> writes edit-plan.json
  -> optionally writes voiceover-script.txt and audio shopping list

service renderer
  -> validates edit-plan.json
  -> renders final.mp4
  -> writes render-manifest.json
```

Suggested local command:

```bash
mvn spring-boot:run -Dspring-boot.run.profiles=cinematic-editing-local
```

Then place a folder of clips here:

```text
input/editing/source/porsche-session-001/
```

The service should produce:

```text
output/edit-projects/<project-id>/ai-director-prompt.md
output/edit-projects/<project-id>/analysis.json
output/edit-projects/<project-id>/contact-sheets/
output/edit-projects/<project-id>/thumbnails/
```

The AI director should only need those generated artifacts to make editing decisions. It should not need to inspect full-resolution clips unless the user explicitly asks for manual review of a specific moment.

### AI Director Responsibilities

The AI director should:

- judge clips from thumbnails, contact sheets, metadata, and optional proxies
- identify the best hero shots, detail shots, motion shots, and ending shot
- choose clip order and trim ranges
- write a strict `edit-plan.json`
- write or update voiceover lines
- suggest music and SFX cues
- keep the final timeline within the configured target duration

The AI director should not:

- render video
- run heavy FFmpeg commands unless troubleshooting
- modify source clips
- invent clip IDs or timestamps outside `analysis.json`
- depend on private absolute paths in viewer-facing text

### Service Responsibilities In AI Director Mode

The service should:

- watch a configurable local source directory for new project folders
- claim one project folder at a time
- ignore folders ending in `.tmp`, `.part`, `.download`, or `.processing`
- create deterministic project IDs from folder names with numeric suffixes on collision
- extract enough thumbnails for visual judgment
- create contact sheets that are easy for an AI vision model to inspect
- write a director prompt with exact instructions and JSON schema
- wait for `edit-plan.json`
- validate the plan before rendering
- render final output deterministically

This mode allows a strong AI model to do creative direction once, then a cheaper model or the service itself can execute the mechanical steps.

## Output Targets

Default final render:

- container: MP4
- video codec: H.264
- audio codec: AAC
- resolution: same as source if all clips match, otherwise configurable default `1920x1080`
- frame rate: same as source if all clips match, otherwise configurable default `30`
- audio sample rate: `48000`
- max duration: configurable, default `60` seconds

For Porsche promo videos, default creative structure:

```text
0-5s: hook, hero reveal, engine or impact sound
5-15s: exterior beauty shots and badge/details
15-30s: driving motion, acceleration, road energy
30-45s: interior, craftsmanship, handling, lifestyle shots
45-55s: strongest montage sequence
55-60s: final hero shot and short tagline
```

## Data Model

Add package:

```text
src/main/java/org/example/videoclips/editing
```

Suggested domain records:

```java
record EditProject(
        String id,
        String name,
        EditProjectStatus status,
        Path inputDirectory,
        Path outputDirectory,
        Instant createdAt,
        Instant updatedAt
) {}

enum EditProjectStatus {
    CREATED,
    ANALYZING,
    ANALYZED,
    WAITING_FOR_DIRECTOR,
    PLANNING,
    PLANNED,
    RENDERING,
    RENDERED,
    FAILED
}

record ClipAnalysis(
        String clipId,
        Path sourcePath,
        double durationSeconds,
        String videoCodec,
        String audioCodec,
        int width,
        int height,
        double frameRate,
        List<Path> thumbnails,
        Path contactSheet,
        Path proxyPath,
        double motionScore,
        double brightnessScore,
        double sharpnessScore
) {}

record EditDecision(
        String clipId,
        double sourceStartSeconds,
        double sourceEndSeconds,
        double timelineStartSeconds,
        double timelineEndSeconds,
        String transitionIn,
        String transitionOut,
        double playbackSpeed,
        String visualTreatment,
        String reason
) {}

record AudioCue(
        String type,
        String assetKey,
        double timelineStartSeconds,
        double timelineEndSeconds,
        double gainDb,
        String notes
) {}

record VoiceoverLine(
        String text,
        double timelineStartSeconds,
        double timelineEndSeconds,
        String delivery
) {}

record EditPlan(
        String projectId,
        String style,
        double targetDurationSeconds,
        List<EditDecision> decisions,
        List<AudioCue> audioCues,
        List<VoiceoverLine> voiceover,
        String renderProfile,
        String summary
) {}
```

For the first implementation, these can be JSON files on disk. Do not start with JPA unless project persistence is required by the user later.

## API Contract

Add REST endpoints under `/v1/edit-projects`.

### Create Edit Project

`POST /v1/edit-projects`

Request:

```json
{
  "name": "porsche-cinematic",
  "inputDirectory": "./output/clips/20240708_173213",
  "targetDurationSeconds": 60,
  "style": "cinematic-porsche-promo",
  "voiceoverEnabled": true,
  "musicEnabled": true,
  "soundEffectsEnabled": true
}
```

Response:

```json
{
  "projectId": "edit_01JZABCDE",
  "status": "CREATED"
}
```

Validation:

- `name`: required, filesystem-safe after normalization
- `inputDirectory`: required and must exist
- `targetDurationSeconds`: `15..600`
- `style`: required, initially allow `cinematic-porsche-promo`

### Analyze Clips

`POST /v1/edit-projects/{projectId}:analyze`

Behavior:

- scans project input directory
- accepts valid video files only
- runs `ffprobe`
- extracts thumbnails
- creates contact sheets
- optionally creates low-resolution proxy files
- writes `analysis.json`

Response:

```json
{
  "projectId": "edit_01JZABCDE",
  "status": "ANALYZED",
  "clipCount": 35,
  "analysisPath": "./output/edit-projects/edit_01JZABCDE/analysis.json"
}
```

### Generate Storyboard Prompt

`POST /v1/edit-projects/{projectId}:storyboard-prompt`

Behavior:

- reads `analysis.json`
- produces a compact AI prompt
- includes clip summaries, thumbnail references, timing constraints, and output schema
- writes `storyboard-prompt.md`

Response:

```json
{
  "projectId": "edit_01JZABCDE",
  "promptPath": "./output/edit-projects/edit_01JZABCDE/storyboard-prompt.md"
}
```

### Save Edit Plan

`PUT /v1/edit-projects/{projectId}/plan`

Request:

```json
{
  "style": "cinematic-porsche-promo",
  "targetDurationSeconds": 60,
  "decisions": [],
  "audioCues": [],
  "voiceover": [],
  "renderProfile": "mp4-h264-aac-1080p",
  "summary": "Cinematic Porsche promo edit."
}
```

Behavior:

- validates the AI-generated edit plan
- ensures all clip IDs exist
- ensures source time ranges are inside clip durations
- ensures timeline ranges do not overlap incorrectly
- writes `edit-plan.json`

### Render Edit

`POST /v1/edit-projects/{projectId}:render`

Behavior:

- reads `edit-plan.json`
- renders the final video with FFmpeg
- mixes music, SFX, and voiceover if available
- writes final MP4 and `render-manifest.json`

Response:

```json
{
  "projectId": "edit_01JZABCDE",
  "status": "RENDERED",
  "outputPath": "./output/edit-projects/edit_01JZABCDE/final.mp4",
  "durationSeconds": 60
}
```

### Read Project

`GET /v1/edit-projects/{projectId}`

Return current status, paths, timestamps, and error details if failed.

## File Layout

For local filesystem implementation:

```text
output/edit-projects/<project-id>/
  project.json
  analysis.json
  storyboard-prompt.md
  edit-plan.json
  render-manifest.json
  final.mp4
  thumbnails/
    <clip-id>_0001.jpg
    <clip-id>_0002.jpg
  contact-sheets/
    <clip-id>.jpg
  proxies/
    <clip-id>.mp4
  audio/
    voiceover.wav
    music.wav
    sfx/
  inbox/
    edit-plan.json
```

## Configuration

Add YAML config under `video-clipping.editing`.

```yaml
video-clipping:
  editing:
    enabled: true
    project-directory: ./output/edit-projects
    ffmpeg-binary: ffmpeg
    ffprobe-binary: ffprobe
    thumbnail-count-per-clip: 5
    contact-sheet-columns: 5
    proxy-enabled: true
    proxy-width: 640
    target-duration-seconds: 60
    output-width: 1920
    output-height: 1080
    output-frame-rate: 30
    audio-sample-rate: 48000
    video-bitrate: 12000k
    audio-bitrate: 192k
    local-director:
      enabled: true
      source-directory: ./input/editing/source
      working-directory: ./input/editing/working
      processed-directory: ./input/editing/processed
      rejected-directory: ./input/editing/rejected
      poll-interval-ms: 5000
      director-prompt-file-name: ai-director-prompt.md
      expected-plan-file-name: edit-plan.json
      auto-render-when-plan-appears: false
```

Local director defaults:

- `auto-render-when-plan-appears=false` is safer for early development because the user can inspect `edit-plan.json` before rendering.
- When set to `true`, the service should render automatically after a valid plan appears in the project `inbox/` folder.
- The scheduler should process one project folder at a time, using the same repeat-prevention pattern as the input folder scheduler.

## FFmpeg Commands

### Probe Clip

```bash
ffprobe -v error \
  -show_entries format=duration \
  -show_entries stream=codec_type,codec_name,width,height,r_frame_rate \
  -of json \
  clip_00001.mp4
```

### Extract Thumbnail

Use timestamps spread across clip duration.

```bash
ffmpeg -hide_banner -y \
  -ss 3.2 \
  -i clip_00001.mp4 \
  -frames:v 1 \
  -q:v 2 \
  thumbnails/clip_00001_0001.jpg
```

### Create Contact Sheet

```bash
ffmpeg -hide_banner -y \
  -i clip_00001.mp4 \
  -vf "fps=1/2,scale=320:-1,tile=5x5" \
  -frames:v 1 \
  contact-sheets/clip_00001.jpg
```

### Create Proxy

```bash
ffmpeg -hide_banner -y \
  -i clip_00001.mp4 \
  -vf "scale=640:-2" \
  -c:v libx264 -preset veryfast -crf 28 \
  -c:a aac -b:a 96k \
  proxies/clip_00001.mp4
```

### Render Final Video

First implementation should use a concat-based render:

1. Trim each selected segment into an intermediate file.
2. Normalize resolution, frame rate, pixel format, and audio.
3. Concatenate intermediates.
4. Mix music, SFX, and voiceover.
5. Write final MP4.

This is simpler and easier for a lower-cost coding model than building one huge `filter_complex` command.

Intermediate segment command:

```bash
ffmpeg -hide_banner -y \
  -ss <sourceStart> \
  -to <sourceEnd> \
  -i <sourceClip> \
  -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,format=yuv420p" \
  -r 30 \
  -c:v libx264 -preset veryfast -crf 18 \
  -c:a aac -b:a 192k -ar 48000 \
  <work>/segment_0001.mp4
```

Concat command:

```bash
ffmpeg -hide_banner -y \
  -f concat \
  -safe 0 \
  -i concat.txt \
  -c copy \
  <work>/timeline-video.mp4
```

Final mix command:

```bash
ffmpeg -hide_banner -y \
  -i <work>/timeline-video.mp4 \
  -i audio/music.wav \
  -i audio/voiceover.wav \
  -filter_complex "[1:a]volume=0.25[music];[2:a]volume=1.0[voice];[0:a][music][voice]amix=inputs=3:duration=first:dropout_transition=2[a]" \
  -map 0:v -map "[a]" \
  -c:v copy \
  -c:a aac -b:a 192k \
  final.mp4
```

Add SFX later after the basic music and voiceover mix works.

## AI Storyboard Prompt Requirements

The generated prompt should instruct the AI to output strict JSON only.

Required input context:

- project style
- target duration
- clip IDs
- duration per clip
- dimensions and frame rate
- extracted thumbnail paths or descriptions
- any motion, brightness, and sharpness scores
- known constraints

Required output schema:

```json
{
  "style": "cinematic-porsche-promo",
  "targetDurationSeconds": 60,
  "summary": "string",
  "decisions": [
    {
      "clipId": "clip_00001",
      "sourceStartSeconds": 0.0,
      "sourceEndSeconds": 2.5,
      "timelineStartSeconds": 0.0,
      "timelineEndSeconds": 2.5,
      "transitionIn": "cut",
      "transitionOut": "crossfade",
      "playbackSpeed": 1.0,
      "visualTreatment": "high contrast warm cinematic grade",
      "reason": "hero front angle"
    }
  ],
  "voiceover": [
    {
      "text": "This Porsche turns precision into emotion.",
      "timelineStartSeconds": 3.0,
      "timelineEndSeconds": 6.0,
      "delivery": "confident cinematic narrator"
    }
  ],
  "audioCues": [
    {
      "type": "music",
      "assetKey": "cinematic-driving-bed",
      "timelineStartSeconds": 0.0,
      "timelineEndSeconds": 60.0,
      "gainDb": -12.0,
      "notes": "build energy toward final montage"
    }
  ]
}
```

Prompt rules:

- Never reference clip paths directly in final viewer-facing text.
- Keep total timeline duration within the requested target duration.
- Use at least 12 clips if enough usable clips exist.
- Avoid using any single source clip for more than 20% of the final runtime.
- Prefer a strong first 5 seconds.
- End with the strongest exterior or motion shot.
- Voiceover should praise the car without sounding exaggerated or fake.

## AI Director Prompt File

In local director mode, generate `ai-director-prompt.md` for Codex, Claude, or another AI instance.

The prompt must include:

- project ID and style
- target duration
- path to `analysis.json`
- paths to contact sheets and thumbnails
- exact output location for the plan
- strict JSON schema for `edit-plan.json`
- validation rules
- instruction to judge shots from thumbnails/contact sheets
- instruction to produce a cinematic Porsche promo edit

Suggested prompt content:

```md
You are the director for a cinematic Porsche promo edit.

Use only the project artifacts in this folder:

- `analysis.json`
- `contact-sheets/`
- `thumbnails/`
- optional `proxies/`

Your task:

1. Review the contact sheets and thumbnails.
2. Pick the strongest shots.
3. Create a cinematic edit plan with a strong opening, rising energy, and final hero shot.
4. Write voiceover lines that praise the Porsche in a premium but believable tone.
5. Add music and SFX cues.
6. Write strict JSON to `inbox/edit-plan.json`.

Do not render video. Do not modify source clips. Do not invent clip IDs.
```

The service should write a matching `director-readme.md` with short human instructions:

```text
Open ai-director-prompt.md in Codex or Claude. Let the AI inspect the generated thumbnails and contact sheets. When it writes inbox/edit-plan.json, call the render endpoint or enable auto-render.
```

## Local Director Scheduler

Add a scheduler similar to the input folder scheduler, but for edit project folders.

States:

```text
source -> working -> processed
source -> working -> rejected
```

Folder example:

```text
input/editing/source/porsche-session-001/
  clip_00000.mp4
  clip_00001.mp4
```

Claimed folder:

```text
input/editing/working/porsche-session-001/
```

Project output:

```text
output/edit-projects/porsche-session-001/
```

Scheduler behavior:

1. scan `local-director.source-directory`
2. pick the first valid project folder in deterministic order
3. move it to `working-directory`
4. create an edit project
5. analyze clips
6. generate `ai-director-prompt.md`
7. write status `WAITING_FOR_DIRECTOR`
8. move the source folder to `processed-directory` after project creation succeeds
9. leave the project output folder ready for the AI director

Plan pickup behavior:

1. scan project `inbox/` folders for `edit-plan.json`
2. validate the plan
3. if `auto-render-when-plan-appears=true`, render automatically
4. otherwise expose the project as `PLANNED` and wait for `POST /v1/edit-projects/{projectId}:render`

Tests:

- ignores temporary project folders
- processes one project folder at a time
- creates project artifacts and director prompt
- does not reprocess a processed folder
- rejects empty folders
- validates plan pickup
- respects `auto-render-when-plan-appears`

## Voiceover And Audio

First implementation can support imported audio assets only:

- user places `music.wav` and `voiceover.wav` under project `audio/`
- service mixes those files into the final render

Second implementation can add generated voiceover:

- create `voiceover-script.txt`
- call an external TTS provider or OpenAI audio model outside the core renderer
- save `voiceover.wav`
- keep generation behind an interface so it can be mocked in tests

Suggested interface:

```java
interface VoiceoverGenerator {
    Path generateVoiceover(String projectId, List<VoiceoverLine> lines);
}
```

Suggested first adapter:

```text
NoopVoiceoverGenerator
```

It writes the script only and requires an imported audio file.

## Milestones

1. [x] Add editing configuration under `video-clipping.editing`.
2. [x] Add filesystem project store for `project.json`, `analysis.json`, `edit-plan.json`, and render output.
3. [x] Add edit project domain records and status enum.
4. [ ] Add `POST /v1/edit-projects` and `GET /v1/edit-projects/{projectId}`.
5. [ ] Add clip discovery for project input directories.
6. [ ] Add `ffprobe` analysis and validation for all input clips.
7. [ ] Add thumbnail extraction.
8. [ ] Add contact sheet generation.
9. [ ] Add optional proxy generation.
10. [ ] Add `analysis.json` writing and reading.
11. [ ] Add storyboard prompt generation from `analysis.json`.
12. [ ] Add local AI director prompt generation with thumbnail/contact-sheet instructions.
13. [ ] Add local director source-folder scheduler.
14. [ ] Add project `inbox/` plan pickup.
15. [ ] Add strict JSON edit plan schema and validation.
16. [ ] Add `PUT /v1/edit-projects/{projectId}/plan`.
17. [ ] Add simple segment rendering without transitions.
18. [ ] Add concat-based final timeline rendering.
19. [ ] Add music and voiceover audio mixing.
20. [ ] Add render manifest with command metadata and output details.
21. [ ] Add basic transition support: cut, crossfade, fade in, fade out.
22. [ ] Add SFX cue support.
23. [ ] Add optional generated voiceover adapter interface.
24. [ ] Add structured logs and metrics for analysis, planning, and rendering durations.
25. [ ] Add integration test with generated fixture clips and a short edit plan.
26. [ ] Add documentation for running a local cinematic edit with Codex or Claude as director.
27. [ ] Run `mvn verify` and update this checklist.

## Milestone Details

### Milestone 1: Editing Configuration

Files:

- `src/main/java/org/example/videoclips/config/VideoClippingProperties.java`
- `src/main/resources/application.yml`

Add nested properties class:

```java
private final Editing editing = new Editing();

public Editing getEditing() {
    return editing;
}
```

Add fields matching the YAML in this plan.

Tests:

- add a focused config binding test or extend an existing Spring context test
- verify defaults bind correctly

Acceptance criteria:

- application starts with editing config present
- editing can be disabled with `video-clipping.editing.enabled=false`

### Milestone 2: Filesystem Project Store

Files:

- `src/main/java/org/example/videoclips/editing/EditProjectStore.java`
- `src/main/java/org/example/videoclips/editing/FileSystemEditProjectStore.java`

Responsibilities:

- create project directories
- read/write JSON files with Jackson
- normalize project IDs and reject path traversal
- expose methods for project, analysis, plan, and manifest paths

Tests:

- creates expected folder structure
- rejects invalid project IDs
- round-trips `project.json`

Acceptance criteria:

- no direct filesystem writes from controllers
- all project path creation goes through the store

### Milestone 3: Domain Records

Files:

- `src/main/java/org/example/videoclips/editing/EditProject.java`
- `src/main/java/org/example/videoclips/editing/EditProjectStatus.java`
- `src/main/java/org/example/videoclips/editing/ClipAnalysis.java`
- `src/main/java/org/example/videoclips/editing/EditPlan.java`
- `src/main/java/org/example/videoclips/editing/EditDecision.java`
- `src/main/java/org/example/videoclips/editing/AudioCue.java`
- `src/main/java/org/example/videoclips/editing/VoiceoverLine.java`

Tests:

- JSON serialization for representative records

Acceptance criteria:

- records serialize with stable camelCase JSON
- no business logic in records beyond validation helpers if needed

### Milestone 4: Project API

Files:

- `src/main/java/org/example/videoclips/api/EditProjectController.java`
- `src/main/java/org/example/videoclips/api/dto/CreateEditProjectRequest.java`
- `src/main/java/org/example/videoclips/api/dto/EditProjectResponse.java`

Tests:

- create project success
- reject missing input directory
- reject invalid target duration
- get project success
- get missing project returns `404`

Acceptance criteria:

- API returns project ID and status
- project state is persisted to `project.json`

### Milestone 5-10: Analysis Pipeline

Files:

- `src/main/java/org/example/videoclips/editing/EditProjectAnalyzer.java`
- `src/main/java/org/example/videoclips/editing/FfmpegClipInspector.java`
- `src/main/java/org/example/videoclips/editing/ThumbnailExtractor.java`
- `src/main/java/org/example/videoclips/editing/ContactSheetGenerator.java`
- `src/main/java/org/example/videoclips/editing/ProxyGenerator.java`

Implementation notes:

- reuse patterns from `FolderVideoValidator` and `FolderFfmpegClipper`
- keep process execution behind package-private interfaces for tests
- sanitize process output before logging
- process clips in deterministic filename order

Tests:

- analysis ignores non-video files
- invalid videos are reported in analysis errors
- command construction is correct
- output JSON includes all analyzed clips
- proxy generation can be disabled

Acceptance criteria:

- `analysis.json` is enough for an AI prompt without reading raw video
- failure of one invalid clip does not fail the entire project if at least one valid clip exists

### Milestone 11: Storyboard Prompt

Files:

- `src/main/java/org/example/videoclips/editing/StoryboardPromptGenerator.java`

Tests:

- prompt includes clip IDs, durations, target duration, style, and JSON schema
- prompt excludes raw absolute system paths unless explicitly configured

Acceptance criteria:

- generated prompt can be pasted into an AI model
- model output requirements are strict JSON

### Milestone 12: Local AI Director Prompt

Files:

- `src/main/java/org/example/videoclips/editing/AiDirectorPromptGenerator.java`

Behavior:

- generate `ai-director-prompt.md`
- include project style, target duration, clip metadata, contact sheet paths, thumbnail paths, and strict output schema
- write human-readable `director-readme.md`
- instruct the AI to write `inbox/edit-plan.json`

Tests:

- prompt contains `analysis.json`
- prompt contains `contact-sheets/` and `thumbnails/`
- prompt contains exact output path `inbox/edit-plan.json`
- prompt instructs the AI not to render video

Acceptance criteria:

- a user can open the prompt in Codex or Claude and get a valid edit plan without additional explanation

### Milestone 13: Local Director Source-Folder Scheduler

Files:

- `src/main/java/org/example/videoclips/editing/LocalDirectorScheduler.java`
- `src/main/java/org/example/videoclips/editing/EditProjectDirectoryInitializer.java`

Behavior:

- create configured source, working, processed, rejected, and project output directories on startup
- scan `video-clipping.editing.local-director.source-directory`
- claim one project folder by moving it to working
- ignore hidden and temporary folders
- create an edit project
- analyze the clips
- generate `ai-director-prompt.md`
- set project status to `WAITING_FOR_DIRECTOR`
- move the claimed source folder to processed after successful analysis

Tests:

- creates all local director directories
- selects project folders in deterministic order
- ignores `.tmp`, `.part`, `.download`, and `.processing` folders
- processes one project folder per scan
- writes project prompt and analysis files
- rejects empty project folders

Acceptance criteria:

- dropping a folder into `input/editing/source` produces a ready-to-direct project under `output/edit-projects`

### Milestone 14: Plan Inbox Pickup

Files:

- `src/main/java/org/example/videoclips/editing/EditPlanInboxScanner.java`

Behavior:

- scan project `inbox/` folders for `edit-plan.json`
- ignore partially written files with `.tmp`, `.part`, or `.download` suffix
- validate the plan
- save normalized plan to project root
- mark status `PLANNED`
- render automatically only when `auto-render-when-plan-appears=true`

Tests:

- ignores temporary plan files
- rejects invalid plan files
- accepts valid plan files
- does not render when auto-render is disabled
- invokes renderer when auto-render is enabled

Acceptance criteria:

- Codex or Claude can write `inbox/edit-plan.json` and the service can pick it up without a manual API call

### Milestone 15-16: Edit Plan Validation

Files:

- `src/main/java/org/example/videoclips/editing/EditPlanValidator.java`
- `src/main/java/org/example/videoclips/api/dto/SaveEditPlanRequest.java`

Validation rules:

- all clip IDs exist in `analysis.json`
- `sourceStartSeconds >= 0`
- `sourceEndSeconds > sourceStartSeconds`
- source range is inside clip duration
- timeline ranges are non-negative
- timeline ranges do not overlap unless the transition requires overlap
- final duration is within target duration tolerance
- playback speed is between `0.25` and `4.0`
- only supported transitions are accepted

Tests:

- accepts valid plan
- rejects unknown clip ID
- rejects out-of-range source timestamps
- rejects overlapping timeline decisions
- rejects unsupported transition

Acceptance criteria:

- renderer never receives an invalid plan

### Milestone 17-20: Render Pipeline

Files:

- `src/main/java/org/example/videoclips/editing/EditRenderer.java`
- `src/main/java/org/example/videoclips/editing/FfmpegEditRenderer.java`
- `src/main/java/org/example/videoclips/editing/RenderManifest.java`

Implementation sequence:

1. create working directory
2. render each selected decision into normalized segment MP4
3. write `concat.txt`
4. concatenate segments into `timeline-video.mp4`
5. mix audio when audio files exist
6. write `final.mp4`
7. write `render-manifest.json`

Tests:

- command construction for segment render
- command construction for concat
- command construction for audio mix
- render fails if no decisions exist
- manifest is written on success

Acceptance criteria:

- final MP4 exists after render
- render manifest includes input clip IDs, output path, duration, and command summaries

### Milestone 21-22: Transitions And SFX

Start simple.

Supported transitions:

- `cut`
- `fade-in`
- `fade-out`
- `crossfade`

Supported SFX cues:

- imported WAV files only
- cue has asset key, start time, gain, and optional fade

Tests:

- unsupported transition rejected
- SFX file missing is a validation error
- final mix includes SFX input when cue exists

Acceptance criteria:

- basic cinematic polish is possible without needing a full timeline engine

### Milestone 23: Voiceover Generation Hook

Files:

- `src/main/java/org/example/videoclips/editing/VoiceoverGenerator.java`
- `src/main/java/org/example/videoclips/editing/NoopVoiceoverGenerator.java`

Behavior:

- first version writes `voiceover-script.txt`
- does not call external APIs
- later adapters can call TTS providers

Tests:

- script file generated from voiceover lines
- noop adapter does not fail when TTS is disabled

Acceptance criteria:

- service supports the workflow without requiring paid audio generation

### Milestone 24: Logs And Metrics

Logs:

- `event=edit_project_created`
- `event=local_director_project_claimed`
- `event=edit_analysis_started`
- `event=edit_analysis_completed`
- `event=ai_director_prompt_generated`
- `event=storyboard_prompt_generated`
- `event=edit_plan_inbox_detected`
- `event=edit_plan_saved`
- `event=edit_render_started`
- `event=edit_render_completed`
- `event=edit_render_failed`

Metrics:

- `video.clipping.edit.projects.created`
- `video.clipping.edit.analysis.duration`
- `video.clipping.edit.render.duration`
- `video.clipping.edit.render.failures`

Tests:

- metrics binder test if custom metrics are added
- at minimum, unit tests should cover event-producing paths

Acceptance criteria:

- an operator can see which stage failed and how long rendering took

### Milestone 25: Integration Test

Create generated fixture clips with FFmpeg:

- three 3-second clips
- different colors or test patterns
- simple audio tones

Test flow:

1. place a folder of generated clips in local director source
2. run the local director scheduler
3. assert `ai-director-prompt.md` and `analysis.json` exist
4. write a short valid `inbox/edit-plan.json`
5. run the plan inbox scanner
6. render final video
7. probe final video
8. assert duration is greater than zero

Skip test when `ffmpeg` or `ffprobe` is unavailable.

Acceptance criteria:

- proves the end-to-end renderer works on a machine with FFmpeg installed

## Lower-Cost Model Implementation Instructions

Follow these rules when assigning this plan to a cheaper coding model:

- Implement exactly one milestone at a time.
- Run the relevant tests after each milestone.
- Update the checklist in this file after each completed milestone.
- Commit after each milestone if the repository is in a clean, test-passing state.
- Do not refactor unrelated packages.
- Reuse existing patterns from the folder scheduler for process execution, logging, and tests.
- Keep the first renderer deterministic and simple.
- Do not add external AI, TTS, or music-provider calls until the filesystem workflow works.
- For local director mode, implement file handoff first. Do not embed Codex, Claude, or another model inside the Spring service.

Suggested prompt for the lower-cost coding model:

```md
You are implementing `docs/cinematic-video-editing-service-plan.md`.

Implement the next unchecked milestone only. Keep the change scoped. Use existing patterns from the folder scheduler package where possible. Add tests for the milestone. Run the relevant Maven test command. Update the milestone checkbox only when tests pass. Do not implement future milestones early.
```

Suggested prompt for Codex or Claude acting as the local AI director:

```md
You are the AI director for this local edit project.

Read `ai-director-prompt.md`, `analysis.json`, and the generated contact sheets and thumbnails. Create a cinematic Porsche promo edit plan. Write strict JSON to `inbox/edit-plan.json`. Do not render video and do not modify the source clips.
```

## Local Runbook

Use this runbook after the local AI director milestones are implemented.

### Prerequisites

Install:

- Java 21
- Maven
- FFmpeg
- ffprobe
- Codex, Claude, or another AI instance that can read local project files and images

Verify tools:

```bash
java -version
mvn -version
ffmpeg -version
ffprobe -version
```

### Start The Service

Run the service with the local cinematic editing profile:

```bash
mvn spring-boot:run -Dspring-boot.run.profiles=cinematic-editing-local
```

Expected startup behavior:

- creates `input/editing/source`
- creates `input/editing/working`
- creates `input/editing/processed`
- creates `input/editing/rejected`
- creates `output/edit-projects`
- logs that local director mode is enabled

### Add Source Clips

Create one project folder under the local director source directory:

```text
input/editing/source/porsche-session-001/
```

Place the clips inside that folder:

```text
input/editing/source/porsche-session-001/clip_00000.mp4
input/editing/source/porsche-session-001/clip_00001.mp4
input/editing/source/porsche-session-001/clip_00002.mp4
```

Recommended copy flow for large files:

1. copy the folder as `porsche-session-001.tmp`
2. wait until all files are fully copied
3. rename it to `porsche-session-001`

The scheduler should ignore temporary folders ending in `.tmp`, `.part`, `.download`, or `.processing`.

### Wait For Analysis

The service should claim the folder and move it through:

```text
input/editing/source/porsche-session-001
input/editing/working/porsche-session-001
input/editing/processed/porsche-session-001
```

After analysis, find the project output:

```text
output/edit-projects/porsche-session-001/
```

Expected files:

```text
output/edit-projects/porsche-session-001/project.json
output/edit-projects/porsche-session-001/analysis.json
output/edit-projects/porsche-session-001/ai-director-prompt.md
output/edit-projects/porsche-session-001/director-readme.md
output/edit-projects/porsche-session-001/contact-sheets/
output/edit-projects/porsche-session-001/thumbnails/
output/edit-projects/porsche-session-001/inbox/
```

Expected status:

```text
WAITING_FOR_DIRECTOR
```

### Ask The AI Director To Edit

Open Codex, Claude, or the selected AI instance in the project output folder.

Use this prompt:

```md
You are the AI director for this local edit project.

Read `ai-director-prompt.md`, `analysis.json`, and the generated contact sheets and thumbnails. Create a cinematic Porsche promo edit plan. Write strict JSON to `inbox/edit-plan.json`. Do not render video and do not modify the source clips.
```

The AI should inspect:

- `analysis.json`
- `contact-sheets/*.jpg`
- `thumbnails/*.jpg`
- optional `proxies/*.mp4`

The AI should write:

```text
output/edit-projects/porsche-session-001/inbox/edit-plan.json
```

### Validate And Render

If `auto-render-when-plan-appears=false`, trigger rendering manually:

```bash
curl -X POST http://localhost:8080/v1/edit-projects/porsche-session-001:render
```

If `auto-render-when-plan-appears=true`, the service should render automatically after it detects and validates:

```text
output/edit-projects/porsche-session-001/inbox/edit-plan.json
```

Expected final output:

```text
output/edit-projects/porsche-session-001/final.mp4
output/edit-projects/porsche-session-001/render-manifest.json
```

### Review Output

Check:

- final video exists
- duration is close to target duration
- music and voiceover are present if configured
- no black frames at the beginning or end
- no invalid clip references in `render-manifest.json`

Optional probe:

```bash
ffprobe -v error \
  -show_entries format=duration \
  -show_entries stream=codec_type,codec_name,width,height,r_frame_rate \
  -of json \
  output/edit-projects/porsche-session-001/final.mp4
```

### Troubleshooting

No project appears in `output/edit-projects`:

- confirm the service is running with `cinematic-editing-local`
- confirm the source folder is not still named `.tmp`, `.part`, `.download`, or `.processing`
- confirm the folder contains at least one valid video file
- check logs for `event=local_director_project_claimed`

Analysis fails:

- run `ffprobe` manually against one source clip
- check whether the input file is still being copied
- move bad files out of the project folder and try again

AI cannot decide from thumbnails:

- increase `thumbnail-count-per-clip`
- enable proxies with `proxy-enabled=true`
- regenerate analysis

`edit-plan.json` is rejected:

- confirm all `clipId` values exist in `analysis.json`
- confirm source timestamps are inside each clip duration
- confirm timeline timestamps do not overlap except for supported transitions
- confirm the JSON is strict JSON with no markdown wrapper

Render fails:

- check `render-manifest.json` if it exists
- check FFmpeg logs
- confirm all referenced source clips are still available under the project
- temporarily disable music, SFX, and voiceover to isolate video rendering

Final video has no audio:

- confirm the source clips have audio or imported audio files exist
- confirm `music.wav` and `voiceover.wav` are valid WAV files if used
- check the audio mix command in the render manifest

Final video is too slow to render:

- lower output resolution
- disable proxies only after analysis is complete
- keep the first renderer concat-based before adding expensive transitions

## Cost Strategy

The service should reduce expensive AI usage by producing compact artifacts.

Recommended AI input:

- `analysis.json`
- contact sheet images
- optional short low-resolution proxy snippets
- target duration and style

Avoid AI input:

- full-resolution video files
- every frame from every clip
- raw uncompressed audio

Expected token ranges:

- metadata-only storyboard: `5k-15k`
- metadata plus contact sheets: `30k-100k`
- dense frame analysis: `150k-400k+`

The target architecture should keep normal cinematic edit planning in the `30k-100k` range before rendering.

## Completion Criteria

The feature is complete when:

- a user can create an edit project from an existing folder of clips
- a user can drop a folder into `input/editing/source` and get a ready-to-direct project locally
- `ai-director-prompt.md` tells Codex or Claude exactly how to judge thumbnails and write `inbox/edit-plan.json`
- the service can pick up an AI-generated `edit-plan.json` from the project inbox
- the service can analyze clips and generate `analysis.json`
- the service can generate a storyboard prompt
- a valid edit plan can be saved
- the service can render one final MP4
- music and voiceover can be mixed into the output
- a realistic FFmpeg integration test passes
- the plan checklist is fully checked