diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..b5c1f2b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,60 @@ +# AGENTS.md — onboarding for any AI assistant (Codex, Claude, etc.) + +This file is the portable entry point for any AI or human working on this repo. Everything below is plain +Markdown/Python in the repo — nothing depends on a specific assistant. (Claude Code additionally auto-loads +`.claude/skills` + `.claude/memory`, but those are just Markdown you can read directly.) + +## What this project is + +A **local, offline** service that turns one source video into a cinematic highlight: it selects the moment, +auto-directs a story-structured montage, generates music/SFX/voiceover with local models, renders with a +cinematic grade, and masters the audio. No external AI services, no runtime downloads. Java 21 / Spring Boot. + +## Read these first (source of truth, in order) + +1. [`README.md`](README.md) — overview, quality rules, models/licenses, limitations. +2. [`docs/cinematic-quality-rules.md`](docs/cinematic-quality-rules.md) — the R1–R9 source-adaptive rendering/director rules (all implemented). +3. [`docs/RUNBOOK-highlight-e2e.md`](docs/RUNBOOK-highlight-e2e.md) — exact commands: source → rendered `final.mp4`. +4. [`docs/LOCAL-MODELS.md`](docs/LOCAL-MODELS.md) — local model runtime + version traps (Intel-Mac reference). +5. [`docs/cinematic-highlight-poc-plan.md`](docs/cinematic-highlight-poc-plan.md) — milestone log / roadmap. +6. `.claude/memory/ai-handover.md` — latest state, full commit list, honest status. **Always `git log`/`git status` first** — the repo is the live truth. + +## Build / test / run + +```bash +./mvnw -B verify # build + all tests + coverage (needs JDK 21 and ffmpeg/ffprobe on PATH) +``` + +Run the highlight pipeline via the `localpoc` profile — see the runbook (step-by-step). Key entry code: +`src/main/java/org/example/videoclips/editing/` (`HighlightSourceScheduler`, `HighlightMontageDirector` = +Tier-1 director, `HighlightVisionDirector` = Tier-2 VLM director, `HighlightFfmpegRenderer`). Local model +workers: `tools/local_asset_worker.py`, `tools/vision_caption.py`. + +## Hard constraints (do not violate) + +- No automatic dependency/model downloads at runtime; models load offline from `models/` + the local cache. +- No external AI services in the media path. +- No unlicensed assets; no placeholder silence/tones passed off as generated audio (fail closed instead). +- No rendering without an explicit approval flag. +- Do not run `tools/run_local_*_worker.sh` in a certified environment (their `auto` modes install/download). + +## Honest status (do not overclaim) + +- Cinematic quality ruleset **R1–R9 complete**; `mvn verify` green (271 tests). Production hardening **started** + (CI, README, render-approval gate, Dockerfile, Maven Wrapper). +- The video output is a **technically-clean cinematic draft, NOT certified production-ready.** Blockers: no + blinded human creative review (Gate B) has passed; source is 576p; music is generic `musicgen-small`; audio + models are **CC-BY-NC** (non-commercial). A passing test / valid MP4 is **not** proof of cinematic quality. + +## Environment gotcha (2026-07-24) + +If the app dies with `Operation not permitted` on a socket bind or `/var/folders/.../T`, the sandbox +restricted the default `$TMPDIR`/network: run the app JVM with +`-Dspring-boot.run.jvmArguments="-Djava.io.tmpdir="` and Maven with +`-DargLine="-Djava.io.tmpdir="`. HF downloads: `export HF_HUB_DISABLE_XET=1`. + +## Conventions + +Small, reviewable, test-backed commits. Follow existing patterns. Don't change production-facing defaults +without noting old→new value and rationale. Don't commit generated media/models (`.gitignore` / `.dockerignore` +already exclude `output/`, `input/`, `models/`, `.venv-*`). diff --git a/docs/gate-b-review-bowling.md b/docs/gate-b-review-bowling.md new file mode 100644 index 0000000..5401fca --- /dev/null +++ b/docs/gate-b-review-bowling.md @@ -0,0 +1,46 @@ +# Gate-B review — bowling highlight (production-readiness scorecard) + +Sample: `output/localpoc/highlight-projects/bowling_strike/final.mp4` (auto-directed, R1–R9 pipeline). +This is the artifact that answers **"is the output production-ready?"** — it is production-ready only when +BOTH gates below pass. Fill Gate B by watching/listening; do not relax thresholds to make it pass. + +## Gate A — technical (objective, measured 2026-07-24) → **PASS** + +| Check | Threshold | Measured | Pass | +|---|---|---|---| +| Container / video codec | MP4 / H.264 | mp4 / h264 | ✅ | +| Audio codec / sample rate | AAC / 48 kHz | aac / 48000 (stereo) | ✅ | +| Resolution (portrait source) | portrait, no distortion | 1080×1920 | ✅ | +| Frame rate | 30 fps | 30/1 | ✅ | +| Integrated loudness | −16 LUFS ±1 | −16.1 LUFS | ✅ | +| True peak | ≤ −1.5 dBTP | −2.8 dBFS | ✅ | +| Black / long-silence (unintended) | none | none detected | ✅ | +| Audio present through end (no early cutoff) | audio ≈ video length | 9.8 s audio / 9.67 s video | ✅ | + +## Gate B — human creative (blinded, subjective) → **PENDING** + +Score each 0–4 (0 unusable · 2 acceptable · 3 good · 4 excellent). **Pass = average ≥ 3.5, every dimension ≥ 3, +factual errors = 0, and preference over the deterministic baseline > 50%.** At least 3 reviewers; randomize A/B +order; keep disagreements. + +| Dimension | What to judge | Score (0–4) | Notes | +|---|---|---|---| +| Highlight selection | Is the *right* moment chosen, with good in/out points? | ☐ | | +| Story / structure | Clear build → payoff; nothing feels arbitrary or missing | ☐ | | +| Pacing & transitions | Cut lengths, slow-mo ease, dissolves feel intentional | ☐ | | +| Visual craft | Framing, push-in, grade, exposure, overlay | ☐ | | +| Sound design | Music fit + build, SFX, source-under-score balance | ☐ | | +| Voiceover (if any) | Intelligibility, timing, script faithfulness | ☐ N/A | none in this cut | +| Factual grounding | Overlay/narration assert only what's on screen | ☐ | | +| **Preference vs baseline** | Prefer this over the plain deterministic cut? (Y/N per reviewer) | ☐ | | + +Reviewers: __________ Date: __________ +Average: ____ Min dimension: ____ Factual errors: ____ Preference >50%: ____ + +## Verdict + +- **Gate A:** PASS (measured above). +- **Gate B:** PENDING a human review — until it passes, the output is a technically-clean cinematic **draft**, + **not** production-ready. +- Independent of the gates, commercial release is additionally blocked by model licensing (MusicGen/AudioLDM2 + are CC-BY-NC) and the 576p source ceiling — see [`README.md`](README.md) Limitations.