forked from jsl/video_editing_poc
d89a28e084
Choosing WHICH moment is the highlight is a question of meaning, not motion or loudness, and there is no generic rule in measurement alone: on the real bowling clip a camera turn-away has the highest motion AND is louder than the celebration. Positional bands / thresholds only move which video breaks. So responsibilities are now split, with zero per-video constants: - HighlightMontageDirector.candidatePeaks: measurement PROPOSES the intensity (motion+audio) local maxima, strongest-first, min-separated. No opinion on which is the highlight; no band, no threshold. - HighlightVisionDirector.rankDecisiveMoment: the vision model JUDGES each candidate by highlight-worthiness (a celebration/goal outranks a loud turn-away or an "about to..." build; anticipation is not the payoff). Highest score wins; falls back to the strongest peak only if the model declines. - composeMontageAt: builds the action segment (measured onset -> chosen peak -> measured resolution that sweeps in the outcome+reaction) around the choice. - MomentChooser interface makes the judge a drop-in: a stronger local VLM plugs in with no director changes. Removes the previous positional-band / semantic-weight heuristics. Honest, verified ceiling (documented in R15): the judge is only as good as its eyes. moondream2 perceives some actions (soccer: "kicking a soccer ball" -> the goal is chosen correctly) but not others -- on distant portrait bowling footage it describes every frame as "standing"/"walking" and never sees the celebration (a posture prompt collapsed to a constant "Standing still"). When it can't discriminate, candidates tie and it falls back to the loudest peak. This is a model-capability limit, not a design flaw; the fix is a stronger VLM (drop-in). mvn verify: 293 tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| LOCAL-MODELS.md | ||
| RUNBOOK-highlight-e2e.md | ||
| alerts.md | ||
| autoscaling-policies.md | ||
| backup-and-restore-validation.md | ||
| cinematic-editing-runbook.md | ||
| cinematic-highlight-acceptance-review.md | ||
| cinematic-highlight-operator-checklist.md | ||
| cinematic-highlight-poc-plan.md | ||
| cinematic-quality-rules.md | ||
| cinematic-video-editing-service-plan.md | ||
| cost-model-by-video-minute.md | ||
| current_state.md | ||
| dashboards.md | ||
| dlq-redrive-procedure.md | ||
| ffmpeg-preset-benchmark-baseline.md | ||
| gate-b-review-bowling.md | ||
| grafana-dashboard-creation-prompt.md | ||
| highlight-rendering-gap-closure-plan.md | ||
| input-folder-scheduler-plan.md | ||
| load-test-signoff.md | ||
| local-cinematic-asset-worker-prompt.md | ||
| local-cv-visual-analysis-guide.md | ||
| object-storage-bandwidth-benchmark-baseline.md | ||
| production-cinematic-ai-director-prompt.md | ||
| production-cinematic-highlight-editing-plan.md | ||
| queue-visibility-timeout-tuning.md | ||
| runbooks.md | ||
| video-clipping-service-implementation-plan.md | ||
| video-clipping-service-planning-prompt.md | ||
| worker-cpu-disk-benchmark-baseline.md | ||