Choosing WHICH moment is the highlight is a question of meaning, not motion or
loudness, and there is no generic rule in measurement alone: on the real bowling
clip a camera turn-away has the highest motion AND is louder than the celebration.
Positional bands / thresholds only move which video breaks. So responsibilities
are now split, with zero per-video constants:
- HighlightMontageDirector.candidatePeaks: measurement PROPOSES the intensity
(motion+audio) local maxima, strongest-first, min-separated. No opinion on which
is the highlight; no band, no threshold.
- HighlightVisionDirector.rankDecisiveMoment: the vision model JUDGES each
candidate by highlight-worthiness (a celebration/goal outranks a loud turn-away
or an "about to..." build; anticipation is not the payoff). Highest score wins;
falls back to the strongest peak only if the model declines.
- composeMontageAt: builds the action segment (measured onset -> chosen peak ->
measured resolution that sweeps in the outcome+reaction) around the choice.
- MomentChooser interface makes the judge a drop-in: a stronger local VLM plugs in
with no director changes.
Removes the previous positional-band / semantic-weight heuristics.
Honest, verified ceiling (documented in R15): the judge is only as good as its
eyes. moondream2 perceives some actions (soccer: "kicking a soccer ball" -> the
goal is chosen correctly) but not others -- on distant portrait bowling footage it
describes every frame as "standing"/"walking" and never sees the celebration
(a posture prompt collapsed to a constant "Standing still"). When it can't
discriminate, candidates tie and it falls back to the loudest peak. This is a
model-capability limit, not a design flaw; the fix is a stronger VLM (drop-in).
mvn verify: 293 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
d89a28e084