jsl
/
video_editing_poc
1
0
Fork 1
Code Issues Pull Requests Packages Projects Releases Wiki Activity
video_editing_poc/AGENTS.md

3.8 KiB
Raw Permalink Blame History

AGENTS.md — onboarding for any AI assistant (Codex, Claude, etc.)

This file is the portable entry point for any AI or human working on this repo. Everything below is plain Markdown/Python in the repo — nothing depends on a specific assistant. (Claude Code additionally auto-loads .claude/skills + .claude/memory, but those are just Markdown you can read directly.)

What this project is

A local, offline service that turns one source video into a cinematic highlight: it selects the moment, auto-directs a story-structured montage, generates music/SFX/voiceover with local models, renders with a cinematic grade, and masters the audio. No external AI services, no runtime downloads. Java 21 / Spring Boot.

Read these first (source of truth, in order)

  1. README.md — overview, quality rules, models/licenses, limitations.
  2. docs/cinematic-quality-rules.md — the R1–R14 source-adaptive rendering/director rules (all implemented; R13/R14 opt-in).
  3. docs/RUNBOOK-highlight-e2e.md — exact commands: source → rendered final.mp4.
  4. docs/LOCAL-MODELS.md — local model runtime + version traps (Intel-Mac reference).
  5. docs/cinematic-highlight-poc-plan.md — milestone log / roadmap.
  6. .claude/memory/ai-handover.md — latest state, full commit list, honest status. Always git log/git status first — the repo is the live truth.

Build / test / run

./mvnw -B verify        # build + all tests + coverage (needs JDK 21 and ffmpeg/ffprobe on PATH)

Run the highlight pipeline via the localpoc profile — see the runbook (step-by-step). Key entry code: src/main/java/org/example/videoclips/editing/ (HighlightSourceScheduler, HighlightMontageDirector = Tier-1 director, HighlightVisionDirector = Tier-2 VLM director, HighlightFfmpegRenderer). Local model workers: tools/local_asset_worker.py, tools/vision_caption.py.

Hard constraints (do not violate)

  • No automatic dependency/model downloads at runtime; models load offline from models/ + the local cache.
  • No external AI services in the media path.
  • No unlicensed assets; no placeholder silence/tones passed off as generated audio (fail closed instead).
  • No rendering without an explicit approval flag.
  • Do not run tools/run_local_*_worker.sh in a certified environment (their auto modes install/download).

Honest status (do not overclaim)

  • Cinematic quality ruleset R1–R14 complete (R10 24fps, R11 filmic grade + LUT hook, R12 motion blur, R13 beat-synced cuts, R14 subject-tracking reframe; R13/R14 opt-in). mvn verify green (288 tests). Also fixed a 444→420p playback-compat defect. Production hardening started (CI, README, render-approval gate, Dockerfile, Maven Wrapper).
  • The video output is a technically-clean cinematic draft, NOT certified production-ready. Blockers: no blinded human creative review (Gate B) has passed; source is 576p; music is generic musicgen-small; audio models are CC-BY-NC (non-commercial). A passing test / valid MP4 is not proof of cinematic quality.

Environment gotcha (2026-07-24)

If the app dies with Operation not permitted on a socket bind or /var/folders/.../T, the sandbox restricted the default $TMPDIR/network: run the app JVM with -Dspring-boot.run.jvmArguments="-Djava.io.tmpdir=<writable>" and Maven with -DargLine="-Djava.io.tmpdir=<writable>". HF downloads: export HF_HUB_DISABLE_XET=1.

Conventions

Small, reviewable, test-backed commits. Follow existing patterns. Don't change production-facing defaults without noting old→new value and rationale. Don't commit generated media/models (.gitignore / .dockerignore already exclude output/, input/, models/, .venv-*).