Improve highlight render: filmic grade + true-peak limiter

Phase 3 quality fixes for the highlight renderer (multi-clip FfmpegEditRenderer
left untouched):

- cinematicVisualFilter: replace the weak fixed eq with a deliberate filmic grade
  (curves medium_contrast S-curve + teal-orange colorbalance + eq + unsharp +
  vignette). Richer blue, warm highlights, tonal contrast. Still one uniform look;
  beat/category-specific grading is future work (needs a validated grade enum).
- audio mix: add a brickwall limiter (alimiter limit=0.72) after loudnorm. The
  first render clipped at 0.0 dBFS true peak; measured re-render now lands
  -1.7/-2.7/-2.8 dBFS per highlight (all within the -1.5 dBTP gate), integrated
  loudness -16.3 LUFS.

Test updated; mvn -o verify green (245 tests, 0 failures).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
This commit is contained in:
JSLMPR 2026-07-22 09:38:47 +02:00
parent 588c652a2f
commit 0fcfca9283
3 changed files with 52 additions and 10 deletions

View File

@ -47,21 +47,52 @@ numpy<2, transformers must be 4.x, audiocraft/xformers unusable here, `HF_HUB_DI
**Phase 1 COMPLETE (2026-07-21).**
### Phase 2 — One approved end-to-end highlight (STOP before render for explicit approval)
- [ ] 2.1 Opt-in `localpoc` Spring profile (base/production defaults untouched): venv python, model paths,
offline flags, no bootstrap auto-start, heuristic fallback off, isolated PoC I/O dirs, render disabled.
- [ ] 2.2 Reprocess the DJI source so `category.json` + `highlight-candidates.json` are written.
- [ ] 2.3 Hand-author a director `edit-plan.json` grounded in persisted candidates (manual, local-only).
- [ ] 2.4 Explicit approval for the specific project → render → final + manifest + QA report.
- [ ] 2.5 Measure technical QA + structured human creative review.
- [x] 2.1 Opt-in `localpoc` Spring profile added (`application-localpoc.yml`): venv python, model paths,
offline, no bootstrap auto-start, heuristic visual, isolated PoC I/O dirs, render OFF, approval REQUIRED.
- [x] 2.2 DJI source processed under `localpoc` (app started, ~4 min 4K analysis): wrote `category.json`
(GENERIC_VLOG, conf 0.25) + `highlight-candidates.json` (3 windows: 27-39 s, 36-48 s, 81-93 s).
Actual footage = blue Porsche Taycan aerial orbit (EV car reveal).
- [x] 2.3 Hand-authored `director/edit-plan.json`: 3 x 9 s highlights (opening_hook / rising_energy /
hero_payoff), grounded in the visible car; passes ALL validator constraints (containment, speed 1.0,
duration bounds, VO budget, category match). Manual + local-only (no external AI).
- [x] 2.4 **RENDERED (operator-approved 2026-07-21).** render-enabled override + approved.flag → local models
generated 3 VO (Piper) + 3 music (MusicGen) + SFX (AudioLDM2) → 3 highlights rendered → project
`final.mp4` assembled (27.3 s, 51.7 MB). Flow ~8.6 min. All real local-model assets, no placeholders.
- [~] 2.5 QA measured (honest):
**PASS** — 1920x1080 / H.264 / 30 fps / AAC 48 kHz; duration 27.3 s; integrated loudness -15.8 LUFS
(target -16 ±1); no black frames; no long silence; overlays render & are grounded ("First light.",
"Silent power."); VO intelligible.
**FAIL/weak** — true peak: highlight_001 = 0.0 dBFS (clips; target ≤ -1.5 dBTP); h2/h3 = -1.4 dBFS
(loudnorm TP limiting not enforced tightly). Visual "treatment" is essentially PASSTHROUGH — no visible
cinematic grade/reframe (renderer maps any treatment to a generic look); overlays are plain white
captions. LRA 16.9 (high). Pipeline's own qa-report passed because its grade/clip checks are structural.
**Verdict: genuine end-to-end local PoC, NOT yet "genuinely cinematic."** Visual grade + true-peak
limiting are the top Phase 3 items.
### Phase 3 — Iterate to quality
- [ ] Improve selection, pacing, visual treatment, voice, music/SFX fit; keep experiments reproducible.
Ranked from the first render's evidence:
- [~] P3.1 Stronger filmic grade in `HighlightFfmpegRenderer.cinematicVisualFilter`: S-curve (curves
medium_contrast) + teal-orange colorbalance + eq + unsharp + vignette (replaces the weak fixed eq).
Verified visually (richer blue, warm highlights, contrast, vignette). NOTE: still ONE uniform look —
not yet beat/category-specific (that needs a validated grade enum plumbed via EditDecision). Only the
highlight renderer changed; multi-clip FfmpegEditRenderer untouched.
- [x] P3.2 True-peak safety: added `alimiter=level=disabled:limit=0.72` after loudnorm in the audio mix.
VERIFIED by re-render: project true peak 0.0 dBFS (clipping) → **-1.7 dBFS**; per-highlight -1.7/-2.7/
-2.8 (all <= -1.5 dBTP gate); integrated loudness -16.3 LUFS. (First pass at limit=0.79 gave -1.3 on
h1, 0.2 dB over — tightened to 0.72 for inter-sample margin.) Test added.
- [ ] P3.3 Overlay styling: premium type, safe-area, subtle animation instead of plain white captions.
- [ ] P3.4 Real CV visual analysis (resident model, no bootstrap script) to improve candidate selection.
- [ ] P3.5 Music/SFX creative fit + ducking review; consider longer inference / better prompts.
- [ ] P3.6 Freeze acceptance thresholds + blinded human creative review vs baseline before declaring success.
## Milestone log
- 2026-07-21: Models provisioned & individually proven (Piper, MusicGen, AudioLDM2). Plan approved. Phase 1 started.
- 2026-07-21: **Phase 1 complete.** Worker rewritten (audiocraft→transformers MusicGen + diffusers AudioLDM2),
models materialized to `models/` with license sidecars, all 3 asset kinds generate 48 kHz mono real audio
through the worker, `mvn -o verify` green (245/0/0/0). Paused for review before Phase 2.
- 2026-07-21: Phase 1 committed (a95d1fa). **Phase 2 up to approval gate:** localpoc profile added, DJI source
processed → candidates + category, grounded 3-highlight director plan authored & pre-validated.
STOPPED — awaiting explicit render approval for dji_21230510013241_0091_d.
## Deferred (until output-quality gate passes)
Production hardening: Spring Security/OIDC, PostgreSQL/Testcontainers, containers/K8s, CI/CD, distributed

View File

@ -321,6 +321,11 @@ public class HighlightFfmpegRenderer {
.append(properties.getLoudnessTargetI())
.append(":TP=").append(properties.getLoudnessTruePeak())
.append(":LRA=").append(properties.getLoudnessRange());
// Brickwall safety limiter after loudnorm. loudnorm's true-peak control is predictive and can
// overshoot when several loud sources are mixed. alimiter caps SAMPLE peak, and inter-sample
// true peak runs ~0.5-1.0 dB higher, so hold the sample ceiling at 0.72 lin (~-2.85 dBFS) to
// keep measured true peak comfortably within the -1.5 dBTP gate.
filters.append(",alimiter=level=disabled:limit=0.72");
if (!sourceAudioPresent) {
filters.append(",apad");
}
@ -517,9 +522,13 @@ public class HighlightFfmpegRenderer {
if (visualTreatment == null || visualTreatment.isBlank() || "none".equalsIgnoreCase(visualTreatment)) {
return "";
}
return ",eq=contrast=1.12:saturation=1.18:brightness=-0.015"
+ ",unsharp=5:5:0.55:3:3:0.25"
+ ",vignette=PI/7";
// Deliberate filmic grade: S-curve contrast, teal shadows / warm highlights (cinematic
// "teal-orange"), gentle saturation and gamma lift, light sharpen, and a soft vignette.
return ",curves=preset=medium_contrast"
+ ",colorbalance=rs=-0.04:bs=0.06:rm=0.01:bm=-0.01:rh=0.06:bh=-0.05"
+ ",eq=contrast=1.12:saturation=1.12:gamma=0.97:brightness=-0.012"
+ ",unsharp=5:5:0.4:3:3:0.2"
+ ",vignette=PI/6";
}
private List<VoiceoverInput> voiceoverInputs(EditPlan plan, Path audioDirectory) {

View File

@ -115,6 +115,8 @@ class HighlightFfmpegRendererTest {
assertThat(command).noneMatch(argument -> argument.contains("[0:a]"));
assertThat(command).anySatisfy(argument -> assertThat(argument)
.contains("amix=inputs=1:duration=longest")
.contains("loudnorm=I=")
.contains(",alimiter=level=disabled:limit=0.72")
.contains(",apad[a]"));
assertThat(command).contains("-shortest");
}