Single-pass loudnorm in the mix is only ~+/-2 LUFS accurate, so the auto-rendered
output could land quiet (e.g. -18.7 LUFS vs the -16 target). After the mix,
probeIntegratedLoudness measures the file, loudnessGainDb computes the corrective
gain, and masterLoudness applies it with a brickwall limiter for true peak. No-ops
when already on target or when the measurement is implausible; handles MusicGen
loudness variance. loudnessGainDb unit-tested.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPuJXQyAeWpFcTtcnxo1UN
Rendering (whole service, driven by source measurements, not constants):
- Orientation-aware geometry: FfmpegClipInspector reads display rotation and
stores effective dims; HighlightFfmpegRenderer.outputGeometry renders portrait
sources portrait and skips the 2.39 letterbox on portrait (landscape unchanged).
- Dynamic exposure: probeSourceLuma measures the frames; exposureNormalizationFilter
maps the mean toward a target; the grade is now exposure-preserving (no crushed
subjects: bowling final went ~63 -> ~100 mean luma).
- Motion-adaptive in-shot push-in (zoompan), amount from per-shot YDIF.
- Audio mix ducks source audio under the generated score so it leads.
- Highlight duration is no longer capped (validator + config).
Automatic director (plans were hand-authored before):
- Tier 1 HighlightMontageDirector: composes the montage from measured motion (YDIF)
and audio-energy (RMS) curves -- setup, continuous action/tension, slow-mo payoff
on the audio climax, resolution button, camera-whip tail trimmed.
- Tier 2 HighlightVisionDirector + tools/vision_caption.py: a local, offline
vision-language model (moondream2) captions the payoff frame and augments the
montage with a semantic overlay ("STRIKE") and scene-informed music; fails soft.
- Wired into the scheduler behind auto-director-enabled / vision-director-enabled
(on in the localpoc profile).
Docs: cinematic-quality-rules.md (R1-R5, R9 both tiers), poc-plan milestones.
Tests: mvn -o verify -> 262 passing, 0 failures.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPuJXQyAeWpFcTtcnxo1UN
A montage edits many short shots pulled from ANYWHERE in the source (not a few
fixed contiguous highlight windows), sequenced establishing -> quick detail cuts ->
slow-motion hero, over one continuous music bed -- much closer to how a car film is
actually cut.
- MontagePlan model + director/montage.json shot list (per-shot source time,
duration, punch-in zoom, speed).
- HighlightDirectorFlowService.processMontage: builds one EditPlan from the shot
list (bypassing the highlight-window validator), one continuous music cue,
distributed voiceover, and titles; reuses asset prep + worker + renderer.
- Renderer: explicit per-shot framing via a "zoom=" token (falls back to the
progressive punch-in); pin each segment to its exact target duration with -t so
frame-quantization drift cannot accumulate across many short shots.
Verified on the DJI source: 18 shots, 16.7 s, -16.5 LUFS, TP -2.8 dBTP, QA green.
The edit now reads as a real montage (varied framing + detail cuts + hero) rather
than slow pans. mvn -o verify green (248). Remaining ceiling is the source footage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Turn each beat from one long pan into an edit:
- Flow splits every beat into contiguous cuts (opening 2, rising 3, hero 2) with
hard cuts between and sequential timeline positions; playback speed is preserved
so the hero stays slow-motion across its cuts.
- Renderer frames each cut with a tighter center crop that fills the frame and
hides the mundane location, plus a progressive punch-in (each successive cut of a
beat steps tighter), so the sequence reads as deliberate.
- Widen the duration QA tolerance to the frozen acceptance value (0.25 s): multi-cut
and slow-motion accumulate small per-segment frame-quantization drift.
Verified: 7 cuts (2/3/2), final 27.4 s, TP -2.7 dBTP. mvn -o verify green (248).
Known follow-up: integrated loudness runs low (~-18.7 LUFS) on the sparse music-bed
mix; single-pass loudnorm undershoots -- needs two-pass or a louder bed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Add afade in (0.12s) / out (0.25s) to each narration line so it eases in and out
instead of hard-cutting -- the hard cut read as un-cinematic. Voice remains
present and leading (measured onset ramp -18 -> -17 dB).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Root cause of "no voiceover": in audioMixCommand the [voice] label was used both
as the sidechaincompress key AND as an amix input. FFmpeg does NOT auto-split a
reused label, so the ducking sidechain consumed the voice entirely and the amix
reference got no audio -- the narration never made it into any final mix (every
render played ducked music with a silent hole where the voice should be).
Fix: explicitly `asplit=2[voice_key][voice_mix]` so one copy keys the duck and one
copy stays in the mix. Measured: voice window went from ~-30..-53 dB (silent) to
~-16 dB (present, leading the bed). Revert the compensating +8 dB over-boost back
to unity; music bed lowered to -14 dB so narration leads cleanly.
Also make SFX direction OPTIONAL in the validator (matching the earlier
voiceover-optional change) so a clean music+narration edit with no sound effects
validates. Regression test added for the asplit. mvn -o verify green (248 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Push the highlight output toward a genuinely cinematic look after a human review
found it functional-but-not-cinematic:
- HighlightFfmpegRenderer: add a 2.39:1 letterbox and subtle film grain to each
graded segment; raise overlay placement above the letterbox bar. (A time-based
crop-zoom push-in was prototyped and removed: FFmpeg crop cannot use the `t`
variable for width/height; a zoompan push-in is a possible follow-up.)
- HighlightDirectorPlanValidator: make voiceover OPTIONAL so a music-driven edit
can carry no narration (lines still validated when present).
Combined with a no-voiceover, driving-orchestral, slow-motion-hero director plan,
this yields a letterboxed, richly graded, music-led cinematic cut. mvn -o verify
green (247 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
The local-cv provider was never exercised end to end (its bootstrap was
prohibited), and hid a latent bug: the JDK HttpClient defaulted to HTTP/2 and
negotiated an h2c cleartext upgrade that the HTTP/1.1-only worker (uvicorn/h11)
mishandled by dropping the request body, so every call returned HTTP 422. Pin the
client to HTTP/1.1.
With this fix the resident YOLOv8 worker (run offline against the existing
yolov8n.pt, no bootstrap script) classifies the sample source as CAR_VLOG at 0.95
with measured OpenCV blur/exposure and a real car label, replacing the previous
filename-keyword GENERIC_VLOG fallback. Provider remains opt-in via runtime
override; the committed localpoc profile keeps the heuristic default.
Also ignore yolov*.pt.license.txt (provenance for the git-ignored weights).
mvn -o verify green (247 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
The highlight renderer now selects a filmic grade per story beat, read from the
EditPlan style key (safeKey: style_<category>_<storyPurpose>), so no change to the
shared EditDecision or its serialization is needed:
- opening_hook: calmer, cooler, softer contrast (linear_contrast, gentle vignette)
- rising_energy / default: balanced base grade (medium_contrast)
- hero_payoff: richest, warmer, stronger S-curve and deeper vignette (strong_contrast)
Threads plan.style() into segmentCommand -> cinematicVisualFilter. Multi-clip
FfmpegEditRenderer untouched. Test added; mvn -o verify green (247 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Replace the plain hard-cut white caption with a refined 48px caption, a soft drop
shadow, a thin subtle border, and a smooth alpha fade in and out (0.4s ramps)
within each overlay timeline window. Highlight renderer only; the multi-clip
renderer and overlay placement/safe-area logic are unchanged. Test added;
mvn -o verify green (246 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Phase 3 quality fixes for the highlight renderer (multi-clip FfmpegEditRenderer
left untouched):
- cinematicVisualFilter: replace the weak fixed eq with a deliberate filmic grade
(curves medium_contrast S-curve + teal-orange colorbalance + eq + unsharp +
vignette). Richer blue, warm highlights, tonal contrast. Still one uniform look;
beat/category-specific grading is future work (needs a validated grade enum).
- audio mix: add a brickwall limiter (alimiter limit=0.72) after loudnorm. The
first render clipped at 0.0 dBFS true peak; measured re-render now lands
-1.7/-2.7/-2.8 dBFS per highlight (all within the -1.5 dBTP gate), integrated
loudness -16.3 LUFS.
Test updated; mvn -o verify green (245 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
Activated with --spring.profiles.active=localpoc. Points only at pre-provisioned
local model paths (Piper voice, MusicGen, AudioLDM2), disables bootstrap
auto-start, uses heuristic visual analysis, isolates PoC input/output dirs, and
keeps render disabled + director approval required. Base/production defaults are
untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bg76sLc43Wc3j5ZcLkboYR
JSLMPR