From 31efe330811bf9467a37c51623b31ac07ed3893f Mon Sep 17 00:00:00 2001 From: JSLMPR Date: Fri, 24 Jul 2026 12:17:08 +0200 Subject: [PATCH] Add CI workflow and a production README - .github/workflows/ci.yml: build + test on every push/PR (JDK 21 + ffmpeg, mvn -B verify, uploads surefire reports). Closes the "no CI" gap. - README.md: honest overview of the local offline highlight pipeline, the R1-R9 cinematic quality rules, how to build/test/run, the local models and their (non-commercial) licenses, constraints, and limitations. Explicitly states it is a PoC, not production-hardened. Closes the "no README" gap. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01MPuJXQyAeWpFcTtcnxo1UN --- .github/workflows/ci.yml | 51 +++++++++++++++++++++ README.md | 98 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 149 insertions(+) create mode 100644 .github/workflows/ci.yml create mode 100644 README.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..f64aab9 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,51 @@ +name: CI + +on: + push: + branches: ["**"] + pull_request: + branches: ["**"] + +# Cancel superseded runs on the same ref. +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + build: + name: Build & test (Java 21) + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up JDK 21 + uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: "21" + cache: maven + + # Several integration tests shell out to ffmpeg/ffprobe; the local AI models are NOT needed for tests + # (asset workers are mocked), so no Python venv or model download is required in CI. + - name: Install ffmpeg + run: | + sudo apt-get update + sudo apt-get install -y ffmpeg + ffmpeg -version | head -n 1 + ffprobe -version | head -n 1 + + # ubuntu-latest ships a recent Maven. (A pinned Maven Wrapper is a future hardening step; the wrapper + # plugin couldn't be provisioned offline in this environment.) + - name: Build and test + run: mvn -B -ntp verify + + - name: Upload test reports + if: always() + uses: actions/upload-artifact@v4 + with: + name: surefire-reports + path: target/surefire-reports/ + if-no-files-found: ignore + retention-days: 7 diff --git a/README.md b/README.md new file mode 100644 index 0000000..b69a85d --- /dev/null +++ b/README.md @@ -0,0 +1,98 @@ +# Video Editing Service — local cinematic highlight generator + +Turn a single source video into a cinematic highlight **entirely with local, offline models**: it selects the +moment, cuts a story-structured montage, generates the music/SFX/voiceover, applies a cinematic grade, and +masters the audio — no external AI services, no runtime downloads. + +> **Status:** a working, source-adaptive **proof of concept**. The cinematic quality ruleset (R1–R9, below) is +> complete and content-agnostic. It is **not yet production-hardened** (no auth on the render endpoint, +> no containers/PostgreSQL/no-egress certification, and the local models are non-commercially licensed — see +> [Limitations](#limitations)). Do not deploy as-is. + +## What it does + +For one source clip the single-source highlight pipeline runs, fully offline: + +``` +ingest ─► analyze (ffprobe, scenes, audio, frames) + ─► candidates + category + ─► DIRECTOR (auto): + Tier 1 measure motion (YDIF) + audio (RMS) ─► story-structured shot list + Tier 2 local VLM (moondream2) captions beats ─► semantic payoff selection + overlay + music mood + ─► generate assets (Piper voice · MusicGen music · AudioLDM2 SFX) + ─► render (portrait/landscape-aware, exposure-normalized, push-in, crossfades, slow-mo ramp, + bold overlay, ducked source under a swelling score) + ─► master loudness ─► QA probes ─► final.mp4 +``` + +Rendering is gated: it stays off by default and requires an explicit approval flag per project. + +## Cinematic quality rules (R1–R9) + +Every rule is **source-adaptive** — it measures the source and adapts, rather than hard-coding constants. +Full detail in [`docs/cinematic-quality-rules.md`](docs/cinematic-quality-rules.md). + +| # | Rule | Measure → adapt | +|---|---|---| +| R1 | Exposure | frame luma → normalize; grade never crushes the subject | +| R2 | Orientation | source rotation → portrait/landscape output, no distortion | +| R3 | Audio balance + loudness | score leads, source ducked; measured loudness corrected to −16 LUFS | +| R4 | Duration | any length, story-driven | +| R5 | Motion push-in | per-shot motion (YDIF) → adaptive in-shot `zoompan` | +| R6 | Transitions | cross-dissolves between beats + ease into slow-motion | +| R7 | Overlays | bold, outlined, animated entrance | +| R8 | Music dynamics | volume swell builds into the payoff | +| R9 | Show the action | Tier-1 measured **+ Tier-2 VLM caption-driven** selection | + +## Build & test + +Requires **JDK 21** and **ffmpeg/ffprobe** on the PATH. + +```bash +mvn -B verify # compile, run all tests, JaCoCo gate +``` + +CI (`.github/workflows/ci.yml`) runs this on every push. + +## Run the highlight pipeline (local PoC) + +The `localpoc` Spring profile wires the pipeline to pre-provisioned local model paths, isolates its +input/output trees, keeps rendering disabled + approval-required, and never starts a network-capable +bootstrap. The end-to-end command sequence (stage source → analyze → auto-direct → approve → render) is +documented step-by-step in the operator runbook. Outputs land under +`output/localpoc/highlight-projects//final.mp4`. + +Local models used (each with a provenance sidecar under `models/`): + +| Model | Role | License | +|---|---|---| +| Piper (`en_US-lessac-medium`) | voiceover | MIT / Blizzard dataset | +| MusicGen small | music | **CC-BY-NC** | +| AudioLDM2 | SFX | **CC-BY-NC-SA** | +| moondream2 | Tier-2 vision director | Apache-2.0 | +| YOLOv8n (optional CV) | visual analysis | **AGPL-3.0** | + +## Non-negotiable constraints + +- No automatic dependency/model downloads at runtime; models load offline from `models/` + the local cache. +- No external AI services in the media path. +- No unlicensed assets; no placeholder silence/tones passed off as generated audio (the pipeline fails closed). +- No rendering without an explicit approval flag. + +## Limitations + +- **Licensing:** MusicGen (CC-BY-NC), AudioLDM2 (CC-BY-NC-SA) and YOLOv8 (AGPL) are **non-commercial/copyleft**. + Commercial use requires swapping in commercially-licensed models/assets. +- **Not production-hardened:** no Spring Security/authN, no container/K8s/deployment manifests, REST persistence + defaults to in-memory, `POST /v1/edit-projects/{projectId}:render` has no approval gate, and no-egress + operation is not yet certified. +- **VLM quality:** on distant/small subjects the small local VLM is only weakly discriminative; a stronger + model or closer framing improves Tier-2 selection. +- A director can only cut what was filmed — it cannot show a moment the camera never captured. + +## Repository map + +- `src/main/java/org/example/videoclips/editing/` — highlight analysis, two-tier director, renderer, QA. +- `tools/` — local model workers (`local_asset_worker.py`, `vision_caption.py`). +- `src/main/resources/application-localpoc.yml` — the opt-in PoC profile. +- `docs/` — the PoC plan, the cinematic quality rules, acceptance review.