video_editing_poc/.claude/skills/video-editing-research-methodology/SKILL.md

---
name: video-editing-research-methodology
description: "Load when turning an observation or hunch in this video-editing repository into a controlled experiment, preregistering numeric predictions, designing baselines/counterfactuals/ablations, assigning adversarial review, managing a disabled experiment flag, replicating offline on macOS and Linux, deciding whether a result is a candidate for adoption, or retiring an inconclusive or disproven idea without losing the negative evidence."
---

# Video Editing Research Methodology

## Purpose

Turn a hunch into an auditable result without weakening production controls. Require one causal
**mechanism** (the rules by which a cause produces an outcome) to explain positive observations,
negative observations, boundary cases, and failures. Accept it only after an assigned reviewer tries
to refute it with a prediction that differs from competing explanations.

Use repository facts verified on **2026-07-21**. Treat numeric targets from planning documents or
sibling skills as candidate thresholds until `video-editing-change-control` approves and freezes them.

## Route The Work

| Need | Use this skill? | Exact sibling |
|---|---:|---|
| Form a hypothesis, preregister numbers, run a research lifecycle, or retire an idea | Yes | This skill |
| Select an open problem or make an external novelty claim | No | `video-editing-research-frontier` or `video-editing-external-positioning` |
| Derive a metric, invariant, causal discriminator, or uncertainty calculation | No | `video-editing-proof-and-analysis-toolkit` |
| Execute the current single-source cinematic-quality program | No | `video-editing-cinematic-highlights-campaign` |
| Define release evidence, goldens, or the acceptance portfolio | No | `video-editing-validation-and-qa` |
| Diagnose an unexplained live failure | No | `video-editing-debugging-playbook`, then `video-editing-failure-archaeology` |
| Change code, dependencies, configuration, defaults, runtime, or promotion status | No | `video-editing-change-control` |
| Look up an existing property or add a configuration axis | No | `video-editing-config-and-flags` |

Do not use this skill for routine implementation, incident triage, release certification, or current-campaign execution; use the exact sibling named in the table.

Do not label missing authentication, offline packaging, fail-closed behavior, reproducible builds,
measured QA, or deployment controls as research. Those are engineering obligations. Use research only
when two or more plausible mechanisms need a controlled comparison.

## Define The Terms Once

| Term | Definition in this project |
|---|---|
| Hypothesis | A falsifiable claim that names a mechanism and predicts measured outcomes before data collection. |
| Baseline | The frozen current or simplest approved method against which a candidate is compared. |
| Counterfactual | An otherwise equivalent case in which the proposed cause is absent or changed. |
| Ablation | A counterfactual made by removing exactly one candidate component while holding the rest fixed. |
| Negative control | A case where the mechanism predicts no effect; it detects leakage, confounding, and false positives. |
| Slice | A preregistered subgroup such as source category, speech presence, lighting, codec, platform, or failure mode. |
| Holdout | Data frozen before tuning and opened only for the confirmatory run. |
| Uncertainty | A stated interval or error bound around an estimate, produced by a preregistered method. |
| Replication | A repeat on independently provisioned runtime or unseen sources, not a rerun of the same warm process. |
| Adversarial reviewer | An independent person assigned before results to construct the strongest counterexample and try to falsify the mechanism. |
| Candidate | A replicated research result eligible for change-control review; it is not production-ready. |
| Retirement | A documented decision to stop an idea while preserving its prediction, evidence, and revisit condition. |
| nDCG | Normalized discounted cumulative gain, a ranking metric that rewards relevant candidates more when they appear earlier. |

## Enforce The Research Perimeter

Stop before experimentation when any item fails:

- [ ] Run with network access denied. Do not call external AI services, model hubs, package indexes,
  remote telemetry, or loopback model services during certified experiments.
- [ ] Pre-provision every dependency and model. Record immutable version, SHA-256, source, license,
  approved use, platform, runtime loader, and resource envelope. Never bootstrap or download at runtime.
- [ ] Use only rights-cleared footage, voices, music, sound effects, fonts, LUTs, and model outputs.
  Record consent or voice rights where applicable. Generated output is not automatically licensed.
- [ ] Reject missing capabilities. Never replace requested voiceover, music, or SFX with silence,
  tones, host TTS, generic media, another model, or another analysis mechanism.
- [ ] Obtain explicit render approval before FFmpeg creates a candidate render. Analysis permission is
  not render permission.
- [ ] Leave every production-facing packaged default unchanged. Activate research only in an isolated,
  approved environment with a dedicated experiment control whose safe default is off.
- [ ] Keep research inputs and results out of production queues, tenant data, normal application logs,
  and uncontrolled metrics. Preserve privacy and retention classifications.
- [ ] Route any executable code, flag, schema, fixture, dependency, or behavior change through
  `video-editing-change-control` before use.

Do not run `tools/run_local_cv_worker.sh`, `tools/run_local_asset_worker.sh`, or ordinary application
startup as an experiment. On the verified baseline, those paths can install packages, resolve model
weights, scan inputs, or render. Inspect first:

```bash
rg -n 'pip install|get_pretrained|write_silence|write_fallback_tone|fallbackTone' tools src/main
rg -n 'auto-start:|render-enabled:|require-director-approval:|fallback-to-heuristic:' src/main/resources/application.yml
```

## Use The Research Lifecycle

Never skip a gate. A failed gate produces a negative result or retirement record, not an improvised
change to the method.

| Stage | Required artifact | Gate to advance |
|---:|---|---|
| 0. Observe | Observation record with evidence label | The symptom is reproducible or explicitly marked historical/open. |
| 1. Hypothesize | Preregistered experiment record | Mechanism, alternatives, numeric predictions, controls, slices, and stop rules are frozen. |
| 2. Isolate | Approved disabled experiment control and evidence location | Default is off; scope, owner, expiry, cleanup, and production isolation are verified. |
| 3. Calibrate | Frozen baseline on development data | Metric definitions, denominators, uncertainty method, and data exclusions are fixed. |
| 4. Discriminate | Positive, negative, boundary, counterfactual, and ablation results | Competing mechanisms make different predictions and the observations select among them. |
| 5. Replicate | Unseen-source and platform replication packet | Result repeats offline on supported macOS development and target Linux/VPS runtime. |
| 6. Refute | Signed adversarial review | Independent reviewer cannot produce an unexplained counterexample within the preregistered scope. |
| 7. Decide | Adoption candidate or retirement record | Evidence is complete; no result is silently left behind an expiring flag. |
| 8. Govern | Change-control decision | Adopt through normal gates or remove/disable and preserve the negative ledger. |

### Stage 0: Record The Observation

Distinguish an observation from an explanation. Record the exact source commit, input identity,
effective configuration, environment, and command that exposed it. Label a generated file, plan
checkbox, or hard-coded QA boolean `NOT EVIDENCE` when it cannot support the claim.

Before inventing a new explanation, search the established history:

```bash
git log --oneline --decorate --all
rg -n -i 'TODO|FIXME|not implemented|execution evidence pending|fallback|silence|tone' docs src tools
```

Use `video-editing-failure-archaeology` to determine whether the mechanism is already fixed, active,
partial, historical, or open. Do not rerun a retired path unchanged.

### Stage 1: Preregister A Numeric Hypothesis

Write and freeze the experiment record before inference, rendering, or opening the holdout. Make the
hypothesis predict numbers, not adjectives. Include expected direction, minimum effect, allowed
regression, denominator, interval method, and expected values for every negative control.

When no trustworthy baseline exists, run a labeled **exploratory baseline study** first. Use its data
only to design a later confirmatory study. Do not set a threshold from a dataset and claim success on
that same dataset.

For highlight work, name metrics such as nDCG@3, Recall@3 at a stated temporal IoU, false-positive
rate on no-highlight footage, per-category effects, and blinded preference. For audio/render work,
name probe-derived duration, loudness, true peak, silence/tone rejection, A/V synchronization,
technical failure rate, and blinded rubric dimensions. Get formulas and current campaign floors from
the proof toolkit and cinematic campaign; do not duplicate or silently revise them here.

### Stage 2: Isolate With A Safe Experiment Control

The verified repository has no dedicated experiment registry. Current scheduler, fallback, model,
and render properties are operational controls, not research assignment controls. Do not repurpose
them and do not treat `render-enabled` or `fallback-to-heuristic` as experiment flags.

Introduce any experiment control only through change control. Require this contract:

| Field | Required value |
|---|---|
| Identifier | Stable, descriptive, and unique; never a user or media identifier. |
| Owner | Named accountable team/person and independent reviewer. |
| Safe default | `false` in every packaged configuration and `matchIfMissing`; missing means disabled. |
| Scope | Approved fixtures/environments only; never implicit all-traffic assignment. |
| Start and expiry | UTC dates with a short expiry and automated or reviewed removal checkpoint. |
| Activation | Explicit test/experiment configuration outside production defaults. |
| Observability | Bounded, low-cardinality experiment ID; no source path, prompt, secret, or personal data. |
| Failure | Fail the experiment; never fall back to baseline and label the result candidate. |
| Removal | Delete or permanently disable after adoption/retirement; owner verifies no stale path remains. |

Do not create a speculative flag without a funded experiment, owner, decision date, and deletion
condition. A flag is not approval to render or bypass asset/model preflight.

### Stage 3: Freeze Baselines, Inputs, And Analysis

Use a source-disjoint development/holdout split. Compute a corpus-manifest SHA-256 and split SHA-256.
Freeze the baseline commit, candidate commit, toolchain, feature schema, random seeds, render plan,
models, assets, licenses, and analysis script before the confirmatory run.

| Design element | Required practice | Common invalid result |
|---|---|---|
| Baseline | Run the same inputs, candidate budget, render profile, and reviewer protocol. | Candidate gets more clips, assets, compute, or review time. |
| Counterfactual | Change only the claimed cause and verify output equivalence elsewhere. | Two renders differ in timing, encoding, or mix beyond the intended factor. |
| Ablation | Remove one component per variant; preregister multiple-comparison handling. | A kitchen-sink comparison cannot assign causality. |
| Negative control | Include shuffled timestamps, no-highlight footage, silence/tone, missing model, or unrelated audio as appropriate. | Positive-only examples hide leakage or fallback. |
| Slice | Report each preregistered category/platform/failure slice and its denominator. | Aggregate improvement hides a harmed category. |
| Uncertainty | Report confidence interval or error bound and method; include reviewer disagreement. | A point estimate or one clip is called a result. |
| Exclusion | Freeze objective exclusion rules before results; publish every exclusion count/reason. | Failed outputs disappear from the denominator. |

### Stage 4: Execute Once, Preserve Everything

Run only the exact approved commands in the record. Capture stdout/stderr, exit codes, timestamps,
resource measurements, raw per-item results, failed outputs, and hashes. Do not edit the hypothesis,
threshold, split, or exclusion rule after seeing results. Record deviations and classify the run
`invalid`; preregister a replacement instead of repairing history.

Add every failed prediction, neutral ablation, negative control, and platform difference to the
negative-result ledger. Preserve a false start even when a later variant works.

### Stage 5: Replicate Offline On macOS And Linux

Use the same committed source and immutable corpus, split, dependency, model, and asset manifests.
Provision them before execution through the approved artifact process. Deny egress for the whole
process tree. Record OS version, architecture, Java, Maven, Python runtime if applicable, FFmpeg,
ffprobe, CPU/GPU, memory, precision, thread/concurrency settings, seeds, and hashes.

Define replication before running as either:

- **Byte replication:** identical artifact SHA-256 is required; or
- **Metric-bounded replication:** deterministic bytes are not expected, so every preregistered metric
  must remain within an approved tolerance and every categorical gate must agree.

Do not average platform disagreement away. Mark the result platform-specific or inconclusive when
one supported platform fails. A warm-cache repeat on one machine is not replication.

### Stage 6: Assign Adversarial Refutation

Assign the reviewer before results. The author, experiment operator, fixture annotator, and promoter
must not be the sole adversarial reviewer. Blind media variants and randomize order where practical.

Require the reviewer to:

1. State the strongest competing mechanism.
2. Identify one case where the proposed mechanism predicts an effect and the competitor does not.
3. Identify one negative case where the proposed mechanism predicts no effect.
4. Look for data leakage, source overlap, hidden fallbacks, changed budgets, reviewer unblinding,
   selective exclusions, uncontrolled randomness, and platform-specific behavior.
5. Recompute a sample from raw data and verify hashes/licenses for a sample of inputs/models/assets.
6. Explain every positive, negative, boundary, and failed observation with the proposed mechanism.
7. Return `supported`, `refuted`, or `inconclusive`; never return “looks good.”

Refutation is not an approval gate substitute. A supported result still goes through change control,
validation, security, operations, licensing, and render approval.

### Stage 7: Adopt Or Retire

Classify the idea exactly once at the decision date:

| Status | Meaning | Required next action |
|---|---|---|
| `candidate` | First controlled result met its preregistered gate. | Replicate; do not promote. |
| `replicated-candidate` | Held-out and platform replication passed adversarial review. | Submit the eight-part decision packet to change control. |
| `inconclusive` | Evidence cannot distinguish mechanisms or uncertainty crosses the gate. | Retire temporarily or fund a newly preregistered discriminator. |
| `refuted` | A prediction failed or a counterexample broke the mechanism. | Retire; remove/disable the experiment path. |
| `retired` | Work stopped for evidence, cost, rights, safety, operability, or priority reasons. | Preserve the ledger, stop reason, and revisit condition. |

For an adoption candidate, provide: requirement; selected approach; alternatives; benefits/trade-offs;
operational consequences; security implications; verification; and revisit conditions. Then follow
`video-editing-change-control`. Research status never changes a production default.

## Use These Records

Keep each record in the approved review artifact or a repository docs-of-record location established
through change control. Do not invent a new production path or store secrets, proprietary model
weights, personal data, or unlicensed media in the record.

### Experiment Record Template

```text
Experiment ID / title / UTC date:
Owner / operator / independent adversarial reviewer:
Observation and evidence label:
Question:
Mechanism:
Strongest competing mechanisms:
Baseline commit and exact method:
Candidate commit and safe-disabled experiment control:
Flag owner / safe default=false / start / expiry / deletion condition:
Corpus-manifest hash / split hash / annotation version / licenses:
Dependency, binary, model, asset hashes and approved-use records:
macOS and Linux runtime/hardware/resource envelope:
Primary metric, denominator, formula, predicted baseline and candidate numbers:
Minimum effect / allowed regression / uncertainty method / alpha or interval:
Slice predictions and minimum sample counts:
Positive / boundary / negative controls and predicted numbers:
Counterfactuals / one-factor ablations / multiple-comparison method:
Randomization / blinding / seeds / exclusion rules:
Exact offline commands and egress-denial evidence:
Render approval reference, if rendering is required:
Stop rules / retirement rule / revisit condition:
Raw evidence location and retention/classification:
Observed results, deviations, failed predictions, and uncertainty (after run):
Replication outcome by platform (after run):
Disposition and change-control reference (after review):
```

### Adversarial Review Template

```text
Reviewer / UTC date / independence conflicts:
Mechanism reviewed:
Strongest alternative:
Differentiating positive prediction:
Required negative prediction:
Counterexample attempted and exact command/data:
Leakage/fallback/blinding/exclusion/platform checks:
Raw result recomputation and hash/license sample:
Unexplained positive, negative, boundary, or failure observations:
Verdict: supported | refuted | inconclusive
Required follow-up and change-control blockers:
```

### Negative-Result Ledger Template

```text
Experiment ID / UTC date / commit:
Hypothesis and preregistered numeric prediction:
Observed number and uncertainty:
Failed slice/control/ablation/platform:
Evidence hashes and exact command:
Why the mechanism failed or remains unresolved:
Decision: refuted | inconclusive | retired
Experiment control removed/disabled by / date:
Do not retry unchanged because:
Revisit only when this falsifiable condition changes:
```

## Stop And Revisit Discipline

Stop immediately for a rights gap, checksum mismatch, unexpected network attempt, automatic download,
external AI call, missing model, fallback, placeholder audio, unapproved render, path escape, sensitive
data leak, production-default drift, invalid blinding, split contamination, or corrupted evidence.

Stop the idea at the preregistered resource cap, sample size, failure-rate limit, effect threshold, or
expiry date. Do not extend until significance appears. Revisit a retired idea only when its recorded
condition changes, such as a newly licensed local model, a larger source-disjoint corpus, a corrected
measurement defect, a new discriminating mechanism, or a documented resource-envelope change.

## Learn From The Repository's Actual Origins

| Historical origin | Methodological rule it established |
|---|---|
| Real storage behavior arrived across commits beginning `7e8a214`; earlier state/metadata was not a durable media path. | Measure the real side effect and artifact existence. A successful state transition is not delivery evidence. |
| Commit `1737d8b` moved terminal failure ownership to the DB queue's retry/DLQ policy. | Put the decision at the layer that owns the mechanism; test transient, terminal, duplicate, and boundary attempts. |
| Commit `5d889b0` said “working version but not cinematic.” | Separate renderability from semantic selection and creative quality; require blinded baselines, rubrics, and ablations. |
| Category-aware planning and richer rendering followed in commits such as `5006dfd`, but no controlled causal comparison is recorded. | Treat richer analysis, effects, and assets as candidates until each contribution survives counterfactual testing. |
| Commit `9b56e89` added structural render QA; the current highlight path adds selected duration/black/silence/peak probes while other checks remain structural. | Diagnostics must distinguish measured outputs from plan/command assertions and predict reviewer-relevant failures. |
| Commit `97ba827` added local asset generation with runtime install/model resolution and silence/tone paths. | “Local” does not mean offline, licensed, immutable, or fail-closed; test missing and corrupted capabilities as negative controls. |

Use `video-editing-failure-archaeology` for the full chronicle. History suggests research questions;
it does not prove a current mechanism.

## Completion Checklist

- [ ] Observation, mechanism, alternatives, and numeric predictions were frozen before the run.
- [ ] Baseline, counterfactuals, one-factor ablations, negative controls, boundary cases, slices,
  denominators, exclusions, and uncertainty were specified.
- [ ] Dependencies, models, assets, inputs, splits, licenses, approvals, and checksums are complete.
- [ ] The experiment was isolated behind an approved safe-disabled control with owner and expiry.
- [ ] No download, network, external AI, unlicensed asset, placeholder, unapproved render, fallback, or
  production-default change occurred.
- [ ] The result replicated offline on supported macOS and target Linux/VPS, or is labeled otherwise.
- [ ] An independent reviewer attempted adversarial refutation and explained positive and negative cases.
- [ ] Failed predictions and neutral results are in the negative-result ledger.
- [ ] The idea became a replicated candidate routed through change control, or was documented and retired.

## Provenance and maintenance

Facts and volatile repository observations were rechecked on **2026-07-21** against
`application.yml`, current source/tests, the six cited commits, and the completed sibling skills.

Re-verify the research perimeter and operational-control defaults:

```bash
rg -n 'pip install|get_pretrained|write_silence|write_fallback_tone|fallbackTone' tools src/main
rg -n -C 3 'fallback-to-heuristic:|strict-runtime:|local-director:|highlight-scheduler:|render-enabled:|require-director-approval:' src/main/resources/application.yml
```

Re-verify historical claims and measurement implementations:

```bash
git show -s --format='%h %cs %s' 7e8a214 1737d8b 5d889b0 5006dfd 9b56e89 97ba827
rg -n 'RETRY_SCHEDULED|markTerminalFailure|duration_matches_timeline' src/main src/test
```

Re-verify the absence or presence of a dedicated experiment registry before documenting a flag:

```bash
rg -n -i 'experiment|feature.?flag' pom.xml src/main src/test src/main/resources docs
```

If any command changes its result, update this skill and the authoritative sibling in the same
knowledge-only change. Do not reinterpret drift as approval to alter behavior.