# Cinematic Editing Runbook

This guide explains how to turn a folder of video clips into one edited cinematic MP4 using the local video editing service and an AI director such as Codex or Claude.

The service does the heavy video work with FFmpeg. The AI director reviews generated thumbnails, contact sheets, metadata, and optional proxies, then writes an `edit-plan.json`. The service validates that plan and renders the final video.

## 1. Prerequisites

Install these tools on the machine running the service:

- Java 21
- Maven
- FFmpeg
- ffprobe
- Codex, Claude, or another AI instance that can read local files and images

Verify the tools:

```bash
java -version
mvn -version
ffmpeg -version
ffprobe -version
```

## 2. Start The Service

From the repository root, start the service with the local cinematic editing profile:

```bash
mvn spring-boot:run -Dspring-boot.run.profiles=cinematic-editing-local
```

This profile disables the normal 8-second folder scheduler and enables the cinematic editing workflow.

Expected startup behavior:

- The service creates `input/editing/source`.
- The service creates `input/editing/working`.
- The service creates `input/editing/processed`.
- The service creates `input/editing/rejected`.
- The service creates `output/edit-projects`.
- The service logs that local director mode is enabled.

Useful config defaults:

```yaml
video-clipping:
  editing:
    project-directory: ./output/edit-projects
    thumbnail-count-per-clip: 5
    proxy-enabled: true
    target-duration-seconds: 600
    output-width: 1920
    output-height: 1080
    output-frame-rate: 30
    video-bitrate: 12000k
    audio-bitrate: 192k
    local-director:
      source-directory: ./input/editing/source
      poll-interval-ms: 5000
      auto-render-when-plan-appears: false
      require-approval-before-render: true
      approval-file-name: approved.flag
```

## 3. Prepare A Source Project Folder

You can use either a project folder or loose clips directly under `input/editing/source`.

The clearest layout is one folder per edit project under `input/editing/source`.

Example:

```text
input/editing/source/porsche-session-001/
```

Put your source clips in that folder:

```text
input/editing/source/porsche-session-001/clip_00000.mp4
input/editing/source/porsche-session-001/clip_00001.mp4
input/editing/source/porsche-session-001/clip_00002.mp4
```

For the Porsche example with 35 existing 8-second clips, put all 35 files in the same project folder:

```text
input/editing/source/porsche-session-001/clip_00000.mp4
input/editing/source/porsche-session-001/clip_00001.mp4
...
input/editing/source/porsche-session-001/clip_00034.mp4
```

The scheduler also accepts loose video clips directly in `input/editing/source`:

```text
input/editing/source/clip_00000.mp4
input/editing/source/clip_00001.mp4
input/editing/source/clip_00002.mp4
```

When loose clips are found and no project folder is waiting, the service claims them as one generated project named `source-clips`.

Recommended flow for large files or many clips:

1. Copy the folder as `porsche-session-001.tmp`.
2. Wait until all clips are fully copied.
3. Rename the folder to `porsche-session-001`.

The scheduler ignores temporary project folders ending in `.tmp`, `.part`, `.download`, or `.processing`.

## 4. Wait For Service Analysis

The local director scheduler scans every 5 seconds by default.

When it finds the project folder, it moves the folder through this lifecycle:

```text
input/editing/source/porsche-session-001
input/editing/working/porsche-session-001
input/editing/processed/porsche-session-001
```

After analysis succeeds, the generated project appears here:

```text
output/edit-projects/porsche-session-001/
```

Expected generated files and folders:

```text
output/edit-projects/porsche-session-001/project.json
output/edit-projects/porsche-session-001/analysis.json
output/edit-projects/porsche-session-001/ai-director-prompt.md
output/edit-projects/porsche-session-001/director-readme.md
output/edit-projects/porsche-session-001/contact-sheets/
output/edit-projects/porsche-session-001/thumbnails/
output/edit-projects/porsche-session-001/proxies/
output/edit-projects/porsche-session-001/inbox/
```

The expected project status at this point is:

```text
WAITING_FOR_DIRECTOR
```

Check project state through the API:

```bash
curl http://localhost:8080/v1/edit-projects/porsche-session-001
```

## 5. Ask Codex Or Claude To Act As Director

Open Codex, Claude, or the selected AI instance in this folder:

```text
output/edit-projects/porsche-session-001/
```

Model choice matters mostly for creative judgment, not rendering. Use a stronger model when the footage is ambiguous, repetitive, emotionally important, or needs a premium commercial feel. Use a cheaper model when the generated thumbnails/contact sheets are clear and the edit style is straightforward.

Recommended split:

- Use a stronger model for the first director pass on high-value edits, because it chooses shots, pacing, voiceover, text overlays, and audio moments.
- Use a cheaper model for JSON cleanup, schema repair, or small plan revisions after the creative direction is already clear.
- Do not ask any model to process full-resolution video directly unless necessary. Let the service provide thumbnails, contact sheets, metadata, and proxies.

Give the AI this prompt:

```md
You are the AI director for this local edit project.

Read `ai-director-prompt.md`, `analysis.json`, and the generated contact sheets and thumbnails. Create a cinematic Porsche promo edit plan. Write strict JSON to `inbox/edit-plan.json`. Do not render video and do not modify the source clips.
```

The AI director should inspect:

- `ai-director-prompt.md`
- `analysis.json`
- `contact-sheets/*.jpg`
- `thumbnails/*.jpg`
- `proxies/*.mp4` if more visual context is needed

After the plan is imported, the service also runs an asset materialization stage:

- It reuses matching generated music, SFX, and voiceover assets from the shared cache folders.
- It copies reusable assets into the project-local `audio/` folder so the renderer can use them.
- It writes reusable asset requests under `assets/requests/` when a needed asset is missing.
- It waits to render if required SFX assets are still missing.

The AI director must write this file:

```text
output/edit-projects/porsche-session-001/inbox/edit-plan.json
```

Do not put markdown fences around the JSON. The file must contain strict JSON only.

## 6. What The Edit Plan Must Contain

The generated `edit-plan.json` must match the service schema.

Minimal shape:

```json
{
  "style": "cinematic-porsche-promo",
  "targetDurationSeconds": 60,
  "renderProfile": "mp4-h264-aac-1080p",
  "summary": "A cinematic Porsche promo with a strong opening, detail shots, driving energy, and a final hero shot.",
  "decisions": [
    {
      "clipId": "clip_00000",
      "sourceStartSeconds": 0.0,
      "sourceEndSeconds": 3.5,
      "timelineStartSeconds": 0.0,
      "timelineEndSeconds": 3.5,
      "transitionIn": "fade",
      "transitionOut": "cut",
      "playbackSpeed": 1.0,
      "visualTreatment": "warm high-contrast cinematic grade",
      "reason": "Opening hero angle establishes the car immediately."
    }
  ],
  "audioCues": [],
  "voiceover": []
}
```

Rules the AI director must follow:

- Every `clipId` must exist in `analysis.json`.
- `sourceStartSeconds` and `sourceEndSeconds` must be inside the source clip duration.
- `sourceEndSeconds` must be greater than `sourceStartSeconds`.
- `timelineEndSeconds` must be greater than `timelineStartSeconds`.
- Timeline decisions should be ordered and non-overlapping.
- `playbackSpeed` should usually stay near `1.0` unless the plan intentionally uses slow motion or speed ramp style.
- `audioCues` and `voiceover` must be present, even if they are empty arrays.

## 7. Add Optional Music, Voiceover, And SFX

The renderer can mix optional WAV files into the final output.

Shared cache locations:

- Music: `input/highlights/assets/music/`
- SFX: `input/highlights/assets/sfx/`
- Voiceover cache: `output/highlight-projects/_voiceover-cache/`

If the service or an AI worker generates an asset there, later projects can reuse it automatically.

Place optional assets here before rendering:

```text
output/edit-projects/porsche-session-001/audio/music.wav
output/edit-projects/porsche-session-001/audio/voiceover.wav
output/edit-projects/porsche-session-001/audio/sfx/engine-rev.wav
```

For SFX cues, the `assetKey` is the filename without `.wav`.

Example:

```json
{
  "type": "sfx",
  "assetKey": "engine-rev",
  "timelineStartSeconds": 5.2,
  "timelineEndSeconds": 6.4,
  "gainDb": -3.0,
  "notes": "Short engine accent on acceleration shot."
}
```

For narration, the default `noop` voiceover provider writes a script file but does not synthesize audio. If the plan includes `voiceover` lines, record or generate the narration and place it here before rendering:

```text
output/edit-projects/porsche-session-001/audio/voiceover.wav
```

## 8. Render The Final Video

By default, the local profile has:

```yaml
auto-render-when-plan-appears: false
```

That means you manually trigger rendering after the AI writes `inbox/edit-plan.json`:

```bash
curl -X POST http://localhost:8080/v1/edit-projects/porsche-session-001:render
```

If you set `VIDEO_EDITING_LOCAL_DIRECTOR_AUTO_RENDER=true`, the service renders automatically after it detects and validates:

```text
output/edit-projects/porsche-session-001/inbox/edit-plan.json
```

By default, automatic rendering is still gated by human review. To approve a validated plan for automatic rendering, create:

```text
output/edit-projects/porsche-session-001/inbox/approved.flag
```

On the next inbox scan, the service renders the already imported `edit-plan.json`. To disable the approval gate for local experiments, set:

```bash
VIDEO_EDITING_LOCAL_DIRECTOR_REQUIRE_APPROVAL=false
```

## 9. Find The Edited Video

After rendering completes, the final output is:

```text
output/edit-projects/porsche-session-001/final.mp4
```

The renderer also publishes each final rendered segment clip that was concatenated into the full video:

```text
output/edit-projects/porsche-session-001/rendered-clips/clip_0001.mp4
output/edit-projects/porsche-session-001/rendered-clips/clip_0002.mp4
output/edit-projects/porsche-session-001/rendered-clips/clip_0003.mp4
```

The render manifest is:

```text
output/edit-projects/porsche-session-001/render-manifest.json
```

The QA report is:

```text
output/edit-projects/porsche-session-001/qa-report.json
```

The manifest is useful for debugging because it records the selected clips, rendered segment clip paths, output path, duration, and FFmpeg command summaries.

The QA report is useful for acceptance because it records:

- final output existence
- rendered segment clip existence
- duration consistency with the edit timeline
- required asset resolution
- text overlay timeline and placement safety
- audio mastering presence
- FFmpeg command completion
- black frame probe result
- long silence probe result
- audio clipping probe result

Check the final file with ffprobe:

```bash
ffprobe -v error \
  -show_entries format=duration \
  -show_entries stream=codec_type,codec_name,width,height,r_frame_rate \
  -of json \
  output/edit-projects/porsche-session-001/final.mp4
```

## 10. Review The Result

Review these points before accepting the edited video:

- `final.mp4` exists.
- Duration is close to the requested target duration.
- The opening shot is strong and starts cleanly.
- There are no black frames at the beginning or end.
- Cut timing feels intentional.
- Music, voiceover, and SFX are present if configured.
- `render-manifest.json` does not contain invalid clip references.
- `qa-report.json` has no failed `ERROR` checks.

Current renderer limitation:

- Crossfade-style transitions are represented as paired boundary fades in the concat renderer. They are not true overlapping FFmpeg `xfade` transitions yet.

## 11. Troubleshooting

No project appears in `output/edit-projects`:

- Confirm the service is running with the `cinematic-editing-local` profile.
- Confirm the source folder is under `input/editing/source`.
- Confirm the source folder is not still named `.tmp`, `.part`, `.download`, or `.processing`.
- Confirm the folder contains at least one valid video file.
- Check logs for `event=local_director_project_claimed`.

Analysis fails:

- Run `ffprobe` manually against one source clip.
- Confirm the input file is not still being copied.
- Move bad files out of the project folder and try again.
- Check `input/editing/rejected` for rejected project folders.

The AI cannot decide from thumbnails:

- Increase `VIDEO_EDITING_THUMBNAIL_COUNT_PER_CLIP`.
- Keep `VIDEO_EDITING_PROXY_ENABLED=true`.
- Ask the AI to inspect the generated proxy clips.
- Delete the failed project output and rerun analysis if you intentionally changed analysis settings.

`edit-plan.json` is rejected:

- Confirm every `clipId` exists in `analysis.json`.
- Confirm source timestamps are inside each clip duration.
- Confirm timeline timestamps are ordered and non-overlapping.
- Confirm the JSON has no markdown wrapper.
- Confirm `audioCues` and `voiceover` are arrays, even when empty.

Render fails:

- Check application logs for `event=edit_render_failed`.
- Check `render-manifest.json` if it exists.
- Check `qa-report.json` if it exists.
- Confirm all referenced source clips are still available under the project.
- Temporarily remove music, SFX, and voiceover assets to isolate video rendering.

QA report has failed checks:

- `output_exists`: render did not publish `final.mp4`; check application logs.
- `duration_matches_timeline`: inspect `edit-plan.json` for timeline gaps or invalid final timestamp.
- `required_assets_resolved`: add the missing SFX or voiceover file, or remove the cue from the edit plan.
- `text_overlays_safe`: adjust overlay placement or timeline.
- `black_frames`: inspect the affected render and replace black source sections or trim the decision.
- `long_silence`: add music, reduce silent sections, or confirm intentional silence.
- `audio_clipping`: lower music, SFX, or voiceover gain and render again.

Final video has no audio:

- Confirm the source clips have audio.
- Confirm `audio/music.wav` and `audio/voiceover.wav` are valid WAV files if used.
- Confirm SFX cue `assetKey` values match files under `audio/sfx`.
- Check the audio mix command in `render-manifest.json`.

Rendering is too slow:

- Lower `VIDEO_EDITING_OUTPUT_WIDTH` and `VIDEO_EDITING_OUTPUT_HEIGHT`.
- Lower `VIDEO_EDITING_VIDEO_BITRATE`.
- Keep the first edit shorter while testing.
- Add music and voiceover only after the video-only render works.

## 12. End-To-End Quick Path

Use this condensed checklist when the service is already working:

1. Start the service:

```bash
mvn spring-boot:run -Dspring-boot.run.profiles=cinematic-editing-local
```

2. Create the source project:

```text
input/editing/source/porsche-session-001/
```

3. Put all source clips in that folder.

Alternative: put the clips directly in `input/editing/source`; the service will create a `source-clips` edit project.

4. Wait for:

```text
output/edit-projects/porsche-session-001/ai-director-prompt.md
output/edit-projects/porsche-session-001/analysis.json
```

5. Run Codex or Claude in:

```text
output/edit-projects/porsche-session-001/
```

6. Tell the AI:

```md
Read `ai-director-prompt.md`, `analysis.json`, contact sheets, thumbnails, and proxies. Write the cinematic edit plan to `inbox/edit-plan.json`. Do not render video.
```

7. Add optional audio assets under:

```text
output/edit-projects/porsche-session-001/audio/
```

8. Render:

```bash
curl -X POST http://localhost:8080/v1/edit-projects/porsche-session-001:render
```

9. Open the edited video:

```text
output/edit-projects/porsche-session-001/final.mp4
```