Add CI workflow and a production README
- .github/workflows/ci.yml: build + test on every push/PR (JDK 21 + ffmpeg, mvn -B verify, uploads surefire reports). Closes the "no CI" gap. - README.md: honest overview of the local offline highlight pipeline, the R1-R9 cinematic quality rules, how to build/test/run, the local models and their (non-commercial) licenses, constraints, and limitations. Explicitly states it is a PoC, not production-hardened. Closes the "no README" gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPuJXQyAeWpFcTtcnxo1UN
This commit is contained in:
parent
2552a7cf06
commit
31efe33081
|
|
@ -0,0 +1,51 @@
|
||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: ["**"]
|
||||||
|
pull_request:
|
||||||
|
branches: ["**"]
|
||||||
|
|
||||||
|
# Cancel superseded runs on the same ref.
|
||||||
|
concurrency:
|
||||||
|
group: ci-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
name: Build & test (Java 21)
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 30
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up JDK 21
|
||||||
|
uses: actions/setup-java@v4
|
||||||
|
with:
|
||||||
|
distribution: temurin
|
||||||
|
java-version: "21"
|
||||||
|
cache: maven
|
||||||
|
|
||||||
|
# Several integration tests shell out to ffmpeg/ffprobe; the local AI models are NOT needed for tests
|
||||||
|
# (asset workers are mocked), so no Python venv or model download is required in CI.
|
||||||
|
- name: Install ffmpeg
|
||||||
|
run: |
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y ffmpeg
|
||||||
|
ffmpeg -version | head -n 1
|
||||||
|
ffprobe -version | head -n 1
|
||||||
|
|
||||||
|
# ubuntu-latest ships a recent Maven. (A pinned Maven Wrapper is a future hardening step; the wrapper
|
||||||
|
# plugin couldn't be provisioned offline in this environment.)
|
||||||
|
- name: Build and test
|
||||||
|
run: mvn -B -ntp verify
|
||||||
|
|
||||||
|
- name: Upload test reports
|
||||||
|
if: always()
|
||||||
|
uses: actions/upload-artifact@v4
|
||||||
|
with:
|
||||||
|
name: surefire-reports
|
||||||
|
path: target/surefire-reports/
|
||||||
|
if-no-files-found: ignore
|
||||||
|
retention-days: 7
|
||||||
|
|
@ -0,0 +1,98 @@
|
||||||
|
# Video Editing Service — local cinematic highlight generator
|
||||||
|
|
||||||
|
Turn a single source video into a cinematic highlight **entirely with local, offline models**: it selects the
|
||||||
|
moment, cuts a story-structured montage, generates the music/SFX/voiceover, applies a cinematic grade, and
|
||||||
|
masters the audio — no external AI services, no runtime downloads.
|
||||||
|
|
||||||
|
> **Status:** a working, source-adaptive **proof of concept**. The cinematic quality ruleset (R1–R9, below) is
|
||||||
|
> complete and content-agnostic. It is **not yet production-hardened** (no auth on the render endpoint,
|
||||||
|
> no containers/PostgreSQL/no-egress certification, and the local models are non-commercially licensed — see
|
||||||
|
> [Limitations](#limitations)). Do not deploy as-is.
|
||||||
|
|
||||||
|
## What it does
|
||||||
|
|
||||||
|
For one source clip the single-source highlight pipeline runs, fully offline:
|
||||||
|
|
||||||
|
```
|
||||||
|
ingest ─► analyze (ffprobe, scenes, audio, frames)
|
||||||
|
─► candidates + category
|
||||||
|
─► DIRECTOR (auto):
|
||||||
|
Tier 1 measure motion (YDIF) + audio (RMS) ─► story-structured shot list
|
||||||
|
Tier 2 local VLM (moondream2) captions beats ─► semantic payoff selection + overlay + music mood
|
||||||
|
─► generate assets (Piper voice · MusicGen music · AudioLDM2 SFX)
|
||||||
|
─► render (portrait/landscape-aware, exposure-normalized, push-in, crossfades, slow-mo ramp,
|
||||||
|
bold overlay, ducked source under a swelling score)
|
||||||
|
─► master loudness ─► QA probes ─► final.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
Rendering is gated: it stays off by default and requires an explicit approval flag per project.
|
||||||
|
|
||||||
|
## Cinematic quality rules (R1–R9)
|
||||||
|
|
||||||
|
Every rule is **source-adaptive** — it measures the source and adapts, rather than hard-coding constants.
|
||||||
|
Full detail in [`docs/cinematic-quality-rules.md`](docs/cinematic-quality-rules.md).
|
||||||
|
|
||||||
|
| # | Rule | Measure → adapt |
|
||||||
|
|---|---|---|
|
||||||
|
| R1 | Exposure | frame luma → normalize; grade never crushes the subject |
|
||||||
|
| R2 | Orientation | source rotation → portrait/landscape output, no distortion |
|
||||||
|
| R3 | Audio balance + loudness | score leads, source ducked; measured loudness corrected to −16 LUFS |
|
||||||
|
| R4 | Duration | any length, story-driven |
|
||||||
|
| R5 | Motion push-in | per-shot motion (YDIF) → adaptive in-shot `zoompan` |
|
||||||
|
| R6 | Transitions | cross-dissolves between beats + ease into slow-motion |
|
||||||
|
| R7 | Overlays | bold, outlined, animated entrance |
|
||||||
|
| R8 | Music dynamics | volume swell builds into the payoff |
|
||||||
|
| R9 | Show the action | Tier-1 measured **+ Tier-2 VLM caption-driven** selection |
|
||||||
|
|
||||||
|
## Build & test
|
||||||
|
|
||||||
|
Requires **JDK 21** and **ffmpeg/ffprobe** on the PATH.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mvn -B verify # compile, run all tests, JaCoCo gate
|
||||||
|
```
|
||||||
|
|
||||||
|
CI (`.github/workflows/ci.yml`) runs this on every push.
|
||||||
|
|
||||||
|
## Run the highlight pipeline (local PoC)
|
||||||
|
|
||||||
|
The `localpoc` Spring profile wires the pipeline to pre-provisioned local model paths, isolates its
|
||||||
|
input/output trees, keeps rendering disabled + approval-required, and never starts a network-capable
|
||||||
|
bootstrap. The end-to-end command sequence (stage source → analyze → auto-direct → approve → render) is
|
||||||
|
documented step-by-step in the operator runbook. Outputs land under
|
||||||
|
`output/localpoc/highlight-projects/<project>/final.mp4`.
|
||||||
|
|
||||||
|
Local models used (each with a provenance sidecar under `models/`):
|
||||||
|
|
||||||
|
| Model | Role | License |
|
||||||
|
|---|---|---|
|
||||||
|
| Piper (`en_US-lessac-medium`) | voiceover | MIT / Blizzard dataset |
|
||||||
|
| MusicGen small | music | **CC-BY-NC** |
|
||||||
|
| AudioLDM2 | SFX | **CC-BY-NC-SA** |
|
||||||
|
| moondream2 | Tier-2 vision director | Apache-2.0 |
|
||||||
|
| YOLOv8n (optional CV) | visual analysis | **AGPL-3.0** |
|
||||||
|
|
||||||
|
## Non-negotiable constraints
|
||||||
|
|
||||||
|
- No automatic dependency/model downloads at runtime; models load offline from `models/` + the local cache.
|
||||||
|
- No external AI services in the media path.
|
||||||
|
- No unlicensed assets; no placeholder silence/tones passed off as generated audio (the pipeline fails closed).
|
||||||
|
- No rendering without an explicit approval flag.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- **Licensing:** MusicGen (CC-BY-NC), AudioLDM2 (CC-BY-NC-SA) and YOLOv8 (AGPL) are **non-commercial/copyleft**.
|
||||||
|
Commercial use requires swapping in commercially-licensed models/assets.
|
||||||
|
- **Not production-hardened:** no Spring Security/authN, no container/K8s/deployment manifests, REST persistence
|
||||||
|
defaults to in-memory, `POST /v1/edit-projects/{projectId}:render` has no approval gate, and no-egress
|
||||||
|
operation is not yet certified.
|
||||||
|
- **VLM quality:** on distant/small subjects the small local VLM is only weakly discriminative; a stronger
|
||||||
|
model or closer framing improves Tier-2 selection.
|
||||||
|
- A director can only cut what was filmed — it cannot show a moment the camera never captured.
|
||||||
|
|
||||||
|
## Repository map
|
||||||
|
|
||||||
|
- `src/main/java/org/example/videoclips/editing/` — highlight analysis, two-tier director, renderer, QA.
|
||||||
|
- `tools/` — local model workers (`local_asset_worker.py`, `vision_caption.py`).
|
||||||
|
- `src/main/resources/application-localpoc.yml` — the opt-in PoC profile.
|
||||||
|
- `docs/` — the PoC plan, the cinematic quality rules, acceptance review.
|
||||||
Loading…
Reference in New Issue