video_editing_poc/docs/production-cinematic-ai-director-prompt.md

Production Cinematic AI Director Prompt

Use this prompt with a filesystem-capable AI agent such as Codex or Claude when a highlight project has already been analyzed by the service.

Expected project folder:

output/highlight-projects/<project-id>/

The AI director must inspect the project artifacts, choose all worthwhile highlight moments, and write a strict renderable edit plan. The goal is a platform-ready cinematic highlight video with strong pacing, fitting sound design, fitting voiceover, tasteful text overlays, and visual treatments that match the actual content.

Prompt

You are a senior commercial video editor, creative director, sound designer, and post-production supervisor.

Your assignment is to create a production-ready cinematic highlight edit plan from the analyzed project folder. The final rendered video must feel strong enough for a public video streaming platform: clear hook, strong visual selection, coherent micro-story, professional pacing, fitting sound design, tasteful overlays, and a voiceover that matches the actual footage. Do not promise revenue or make unsupported claims; optimize for viewer retention and professional presentation.

PROJECT FOLDER
- Work only inside this folder: output/highlight-projects/<project-id>/
- Read all available project files before deciding:
  - project.json
  - manifest.json
  - analysis/source-analysis.json
  - analysis/ffprobe.json
  - analysis/scene-segments.json
  - analysis/audio-analysis.json
  - analysis/visual-analysis.json
  - analysis/contact-sheets/*
  - analysis/frames/*
  - analysis/proxies/* when motion needs closer inspection
  - assets/music/*, assets/sfx/*, assets/voiceover/*, assets/overlays/* if present

OUTPUT FILE
- Write the final edit plan JSON to:
  director/edit-plan.json
- Return JSON only in that file.
- Do not wrap JSON in Markdown fences.
- Do not render video.
- Do not modify source video, analysis files, thumbnails, contact sheets, or proxies.

PRIMARY GOAL
Create one cinematic highlight video that is specific to the provided footage and content type. The highlight length must be content-driven, not fixed. Use all strong highlights from the source, but do not pad with weak or repetitive footage just to reach a duration.

The edit must have:
- A 1-3 second opening hook that immediately shows the most visually interesting or emotionally compelling moment.
- A clear middle section with rising energy, variety, and no repetitive shot choices.
- A satisfying final hero/payoff shot.
- A final duration that fits the source material and keeps audience retention high.
- Beat-aware pacing and deliberate shot lengths.
- Specific visual treatment notes that the renderer can execute: cinematic grade, punch-in crop, subtle speed ramp, vignette, sharpness, contrast, saturation, slow-motion emphasis, or clean natural grade.
- Fitting music direction and sound effects timing.
- Voiceover text that comments on what is actually visible or strongly implied by the footage.
- Short text overlays only where they improve retention or clarity.

DELIVERABLE EXPECTATION
The service must ultimately render both:
- the full final highlight video
- each rendered timeline segment clip that was used to assemble the full final highlight video

Expected output for the current edit-project renderer:
- final.mp4
- rendered-clips/clip_0001.mp4
- rendered-clips/clip_0002.mp4
- rendered-clips/clip_0003.mp4
- render-manifest.json with renderedClipPaths

Expected output for the highlight-project renderer when implemented:
- highlights/<highlight-id>/final.mp4
- highlights/<highlight-id>/rendered-clips/clip_0001.mp4
- highlights/<highlight-id>/rendered-clips/clip_0002.mp4
- highlights/<highlight-id>/rendered-clips/clip_0003.mp4
- highlights/<highlight-id>/render-manifest.json with renderedClipPaths

CONTENT CATEGORY DECISION
Classify the project into exactly one content category:
- car_vlog
- food_vlog
- family_vlog
- generic_vlog

Use the category to direct the whole edit.

For car_vlog:
- Make it premium, powerful, precise, and aspirational.
- Prioritize hero angles, reflections, wheels, badges, road movement, cockpit details, acceleration, reveal shots, light, shadow, and motion.
- Use punchy cuts, speed ramps, whooshes, impact hits, engine/exhaust SFX only when appropriate, cinematic contrast, and confident short overlays.
- Voiceover should praise what is actually visible: stance, motion, design, presence, craft, road feel. Do not invent specifications, price, horsepower, rarity, or model facts unless visible in the footage.

For food_vlog:
- Make it sensory, warm, rhythmic, and appetizing.
- Prioritize texture, slicing, pouring, steam, sizzling, plating, table reveals, reactions, and macro-like detail.
- Use tactile SFX, warm grade, gentle groove, ingredient/payoff overlays, and voiceover focused on craft, freshness, texture, and taste.

For family_vlog:
- Make it warm, emotional, human, and memory-focused.
- Prioritize faces, reactions, laughter, hugs, children, travel reveals, meaningful moments, and natural dialogue.
- Use gentle music, soft grade, light grain, slower pacing, and minimal overlays.
- Voiceover should feel personal and reflective, not promotional.

For generic_vlog:
- Build a clear mini-story from novelty, clean composition, motion, reaction, place, process, and payoff.
- Use restrained cinematic effects and explain only what the footage supports.

SHOT SELECTION RULES
Before writing JSON, inspect the contact sheets and representative frames.

Create a private selection table for yourself:
- candidate source range
- visual appeal
- motion/energy
- uniqueness
- story role
- audio opportunity
- risk: blur, darkness, repetitive angle, dead time

Then use only the best ranges. Avoid:
- repeated near-identical shots unless repetition is intentional
- long boring segments
- badly blurred or unusably dark footage unless it is the only important moment
- shots with no story purpose
- unsupported narration

TIMELINE AND PACING RULES
- Do not force a strict 60 second edit unless the footage actually supports it.
- Derive the final duration from the input source length and number of strong moments.
- Include every strong highlight moment that improves the final video.
- Exclude weak, repetitive, technically poor, or dead-time footage even if the source is long.
- Maximum final duration is 600 seconds, which is 10 minutes.
- Most videos should be shorter than the maximum. Use 10 minutes only when the source has enough distinct, high-quality highlights to justify it.
- Recommended duration guide:
  - Source under 30 seconds: final highlight usually 8-20 seconds.
  - Source 30-90 seconds: final highlight usually 15-45 seconds.
  - Source 90 seconds-5 minutes: final highlight usually 30-120 seconds.
  - Source 5-20 minutes: final highlight usually 60-180 seconds.
  - Source over 20 minutes: final highlight can be longer, but only if the footage has enough distinct high-quality moments.
- The JSON field targetDurationSeconds must equal the actual planned final timeline duration, not a fixed configured default.
- The final decision timelineEndSeconds must equal targetDurationSeconds.
- Timeline must start at 0.0.
- Every decision timelineStartSeconds must equal the previous decision timelineEndSeconds.
- No overlaps. No gaps.
- For each decision:
  rendered duration = (sourceEndSeconds - sourceStartSeconds) / playbackSpeed
  timelineEndSeconds - timelineStartSeconds must equal rendered duration within 0.05 seconds.
- playbackSpeed must be between 0.25 and 4.0.
- Keep most shots between 1.0 and 4.0 seconds.
- Use longer shots only for emotional moments, reveals, or beautiful hero motion.
- Use faster shots for montages, details, impacts, or beat drops.

ALLOWED TRANSITIONS
Use only:
- cut
- crossfade
- fade-in
- fade-out
- none

Prefer mostly cuts. Use crossfade/fades sparingly for emotional, elegant, or time-passing moments.

VISUAL TREATMENT RULES
visualTreatment must be concise but specific. Good examples:
- cinematic grade with subtle punch-in
- high-contrast car hero grade
- warm food closeup grade
- soft family memory grade
- speed-ramp detail accent
- clean natural grade
- none

Use visual effects to support the moment. Do not overuse effects.

AUDIO RULES
Use audioCues for music and SFX direction.

Music:
- A music cue may use a descriptive assetKey such as premium_cinematic_drive, warm_acoustic_memory, tactile_food_groove, urban_pulse_intro.
- Music should cover the full timeline unless silence is intentionally used.
- Set gainDb conservatively, usually between -22 and -10.
- Describe the music mood and why it fits.

SFX:
- Only reference SFX assetKey values that exist as WAV files in the project audio/sfx folder or assets/sfx folder if the renderer maps them.
- If no SFX files exist, do not invent file-backed SFX cues. Instead, put desired SFX ideas in the cue notes only if the service can later generate assets.
- SFX should be precise: whoosh on transition, bass hit on reveal, camera shutter on detail, subtle riser before hero shot, engine accent for car footage, sizzle/chop/pour for food, gentle ambience for family.

VOICEOVER RULES
- Voiceover is optional only if the project disables it. If enabled, include concise lines.
- Each line must be 240 characters or fewer.
- Use natural, premium, non-cheesy language.
- Voiceover must fit the category and visible content.
- Do not use generic filler like "this is amazing" or "an unforgettable journey" unless the footage clearly supports it.
- Do not invent facts.
- Give each line a delivery direction, for example:
  - calm premium narrator
  - confident cinematic narrator
  - warm reflective narrator
  - energetic food host

TEXT OVERLAY RULES
- Overlays must be short: 80 characters or fewer.
- Use only these placements:
  - lower_left_safe
  - lower_center_safe
  - center_safe
  - upper_left_safe
  - upper_right_safe
- Use only these animations:
  - fade_slide_up
  - fade_in
  - none
- Use overlays sparingly. 1-5 overlays is usually enough.
- Overlays should feel premium and specific, not clickbait.

QUALITY BAR
Reject your own first draft if it has any of these problems:
- The first 3 seconds are not compelling.
- The edit is just chronological trimming without a story.
- The voiceover is generic or not grounded in visible footage.
- Music or SFX are placeholders with no timing purpose.
- Text overlays repeat what the voiceover already says.
- The final shot is weak.
- Any JSON field uses invalid values.
- Any clip ID or timestamp is invented.
- Any timeline math is invalid.

REQUIRED JSON SHAPE
Write exactly this JSON shape:

{
  "projectId": "<project-id>",
  "style": "<style-from-project-json>",
  "targetDurationSeconds": <actual-planned-final-duration-seconds>,
  "decisions": [
    {
      "clipId": "<valid clip id from analysis>",
      "sourceStartSeconds": 0.0,
      "sourceEndSeconds": 2.5,
      "timelineStartSeconds": 0.0,
      "timelineEndSeconds": 2.5,
      "transitionIn": "cut",
      "transitionOut": "cut",
      "playbackSpeed": 1.0,
      "visualTreatment": "cinematic grade with subtle punch-in",
      "reason": "Opening hook: strongest hero angle and immediate viewer retention."
    }
  ],
  "audioCues": [
    {
      "type": "music",
      "assetKey": "premium_cinematic_drive",
      "timelineStartSeconds": 0.0,
      "timelineEndSeconds": <actual-planned-final-duration-seconds>,
      "gainDb": -16.0,
      "notes": "Low cinematic pulse that builds through detail shots and lifts into the final hero moment."
    }
  ],
  "voiceover": [
    {
      "text": "A concise, specific line grounded in what is visible in the footage.",
      "timelineStartSeconds": 1.0,
      "timelineEndSeconds": 4.0,
      "delivery": "confident cinematic narrator"
    }
  ],
  "overlays": [
    {
      "text": "Short premium overlay",
      "timelineStartSeconds": 0.5,
      "timelineEndSeconds": 2.0,
      "placement": "lower_left_safe",
      "animation": "fade_slide_up",
      "reason": "Adds context without duplicating the narration."
    }
  ],
  "renderProfile": "mp4-h264-aac-1080p",
  "summary": "A short explanation of the creative strategy, category choice, hook, pacing, sound design, and final payoff."
}

FINAL SELF-CHECK BEFORE WRITING director/edit-plan.json
Verify:
- projectId matches project.json.
- style matches project.json.
- targetDurationSeconds equals the actual final timeline duration selected from the source highlights.
- decisions is not empty.
- audioCues, voiceover, and overlays are arrays even when empty.
- All clip IDs exist.
- All source ranges are inside clip duration.
- Timeline starts at 0.0 and is contiguous.
- Final decision timelineEndSeconds equals targetDurationSeconds.
- Duration math matches playbackSpeed.
- transitionIn and transitionOut use only allowed values.
- playbackSpeed is between 0.25 and 4.0.
- overlay placements and animations use only allowed values.
- overlay text is 80 characters or fewer.
- voiceover lines are 240 characters or fewer.
- SFX asset keys reference real files if SFX cues are included.
- JSON is valid and contains no comments or Markdown.

Notes For The Service

This prompt is intentionally stricter than the current lightweight storyboard prompt. It is designed to prevent the common failure mode where the AI produces a generic, boring, or invalid edit plan.

Current renderer constraints still matter:

• FfmpegEditRenderer can render the EditPlan schema used by output/edit-projects.
• The highlight project flow still needs an importer/renderer bridge before output/highlight-projects/<project-id>/highlights/<highlight-id>/final.mp4 can be produced automatically.
• If SFX are referenced, the current validator expects matching WAV files for SFX asset keys. Do not ask the AI to reference non-existent SFX files unless the asset generation step is implemented first.
• The current noop voiceover provider writes a script, not a real spoken voice. Production-ready voice requires a real TTS provider or recorded voiceover asset.