video_editing_poc/.claude/skills/video-editing-validation-and-qa/SKILL.md

---
name: video-editing-validation-and-qa
description: Load this skill when defining acceptance criteria, adding or reviewing tests, validating API/database/local-model/media behavior, interpreting Maven or render QA results, creating fixtures or goldens, or deciding whether a video-editing change is fit for production promotion.
---

# Video Editing Validation and QA

Use evidence to decide whether this service is correct, cinematic, and operable. Treat a green test run as one input, never as a release certificate.

## When not to use this skill

- For symptom-first incident triage, use `video-editing-debugging-playbook`.
- For commands that inspect a running system or media artifact, use `video-editing-diagnostics-and-tooling`.
- For change classification, approval, rollout, or rollback, use `video-editing-change-control`.
- For environment recreation and dependency provisioning, use `video-editing-build-and-env`.
- For runtime profiles and operator workflows, use `video-editing-run-and-operate`.
- For media theory and FFmpeg interpretation, use `cinematic-media-engineering-reference`.
- For the executable highlight-quality program, use `video-editing-cinematic-highlights-campaign`.
- For first-principles experiments and derivations, use `video-editing-proof-and-analysis-toolkit`.

## Non-negotiable release rule

As of 2026-07-21, do **not** describe this repository as production-ready, Fortune 500 reference-quality, or creatively certified. Evidence required for those claims is absent.

Fail a gate when evidence is missing, skipped, stale, environment-dependent, or only asserted. Never convert `unknown` to `pass`.

`video-editing-change-control` is authoritative for the seven no-waiver prohibitions; this table defines their validation evidence. `video-editing-config-and-flags` remains authoritative for defaults.

| Prohibited behavior | Required proof |
|---|---|
| Automatic dependency or model download | Run with network egress denied and pre-provisioned, checksum-pinned dependencies/models. Retain the egress policy and process logs. |
| External AI service access | Deny egress; verify no external endpoint is configured or called. |
| Unapproved network access | Prove model/media inference and acquisition use no network, including loopback; verify in-process or approved non-network IPC. Separately test every approved API/database/storage/telemetry connection for security, timeouts, and failure behavior. Maven/process offline flags alone are not egress denial. |
| Unlicensed music, SFX, fonts, LUTs, footage, or models | Require an asset/model manifest with source, license, version, checksum, and approved usage. Missing provenance blocks use. |
| Silence, tones, or synthetic stand-ins presented as finished assets | Detect and reject placeholders. Do not accept file existence as asset quality. |
| Unapproved rendering | Render only approved fixtures in isolated temporary directories until the change-control gate authorizes broader footage or an environment. |
| Production-facing default changes | Route through `video-editing-change-control`; test both old and proposed behavior and provide rollback evidence. |

The 2026-07-21 working tree removes silence/tone/host-speech fallbacks, restricts AudioCraft to existing local paths with offline flags, and makes strict readiness fail. A zero exit code or output WAV still does not prove semantic fit, license, model identity, loudness, or creative quality; retain final-media and provenance gates.

## Evidence tiers

Classify every claim with the highest tier actually earned.

| Tier | Name | Minimum evidence | Claims it permits |
|---|---|---|---|
| E0 | Assertion | Plan, prompt, comment, checkbox, or unexecuted command | Intent only |
| E1 | Static | Compiles; schema/config/command inspected; deterministic static check passes | Structural property only |
| E2 | Isolated | Deterministic unit, architecture, or adapter test with controlled collaborators | Tested component behavior |
| E3 | Integrated | Real local dependency or executable, representative fixture, machine-read output, no mock at the boundary under test | Integration behavior on the recorded platform |
| E4 | Clean system | Clean checkout, offline/reproducible build, real local models and media tools, source-to-output critical journey | System behavior for the recorded build and environment |
| E5 | Production-like | Security, load, failure injection, observability, container/deploy, and recovery evidence in a production-like environment | Readiness against declared SLOs |
| E6 | Certified | E5 plus independent review, signed evidence, licensed golden inventory, repeatability, and controlled promotion | Reference-quality claim for the tested scope |

Do not promote evidence across scopes. A macOS E3 FFmpeg result does not certify Linux/VPS, container, cloud storage, PostgreSQL, or a different FFmpeg/model build.

## Current evidence inventory

The following inventory was verified on 2026-07-21.

| Area | Current evidence | Honest status |
|---|---|---|
| Maven tests | A 2026-07-21 offline `mvn -q -o verify` run records 245 tests in 62 test classes, 0 failures, 0 errors, 0 skipped in the working tree | E2/E3 mix; not clean-checkout or no-egress certification |
| Coverage | `pom.xml` enforces 100% instruction, line, and branch coverage only for `org.example.videoclips.folder` during `verify` | No repository-wide threshold; no mutation testing |
| Real media tools | `FolderFfmpegIntegrationTest` and `CinematicEditingIntegrationTest` are conditional on local `ffmpeg` and `ffprobe` | Real FFmpeg coverage when available; conditional tests can otherwise skip |
| Clean build | A clean archived checkout has failed `mvn verify` at `LocalAssetGenerationStageTest` because success depends on untracked/runtime-local asset tooling or an audible speech fallback | Release blocker; source-alone reproducibility is unproven |
| API | 18 Spring Boot/MockMvc tests cover selected success, validation, and Problem Details paths | No published/validated OpenAPI, compatibility gate, authentication, or authorization evidence |
| Database | Flyway migrations and PostgreSQL runtime driver exist; normal tests use local/in-memory behavior and H2 can be selected | No real PostgreSQL Testcontainer repository/migration/concurrency evidence |
| Architecture | Package separation exists | No ArchUnit or Spring Modulith boundary test; no automated cycle/core-dependency gate |
| Legacy edit render | Real integration renders generated clips. `FfmpegEditRenderer` runs black-frame, long-silence, and audio-peak probes; unit tests cover their interpretation | Useful technical E3 evidence; the integration does not assert `qaReport.passed()` and creative quality is untested |
| Highlight render | Unit tests cover pieces; `HighlightDirectorFlowServiceTest` uses a mocked renderer and a dummy file | No real source-to-final highlight E2E |
| Highlight QA | `HighlightFfmpegRenderer.buildQaReport` probes output duration, black ranges, long silence, and sample peaks; it also checks assets, plan-level overlay bounds, and the configured mastering command. `HighlightDirectorFlowService` blocks project-final publication on failed `ERROR` checks | A fail-closed selected-defect gate, but no integrated-loudness/true-peak, A/V-sync, freeze, raster-safe-area, semantic-fit, or creative acceptance gate |
| Local models | Adapter/process tests exist | No pinned model inventory, offline no-egress proof, model quality baseline, license proof, or macOS/Linux/container execution matrix |
| Creative quality | Prompts and plans describe cinematic intent | No certified footage set, annotations, baseline, blinded rubric result, or accepted numeric threshold |
| Security | Input/error tests exist | No Spring Security, threat-model gate, auth tests, dependency/static/container/secret scans, or release vulnerability report |
| Performance | Benchmark harnesses and planning baselines exist under `src/test/java/.../perf` and `docs/` | No checked-in load generator or executed production-like load signoff |
| Resilience | Queue/cleanup unit tests and runbooks exist | No fault-injection, shutdown, retry, saturation, or dependency-outage certification |
| Container/deploy/CI | Operational planning documents exist | No CI definition, OCI image definition, deployment manifest, image scan, SBOM/signing gate, or automated promotion evidence |

Do not infer certification from tracked or generated artifacts. `docs/*baseline*.md`, dashboards, manifests, prompts, `qa-report.json`, and files under runtime input/output trees become evidence only when their production method, commit, environment, raw measurements, and reviewer are recorded.

The single-source highlight flow now uses `RenderQaReport.passed()` and failed `ERROR` checks to block project-final publication and mark the project failed. The legacy multi-clip renderer still treats its QA report as diagnostic output. In both paths, inspect warnings and remember that passing the implemented technical checks does not certify creative quality.

## Run the current checks safely

Do not allow Maven, Python, Hugging Face, Torch, FFmpeg inputs, or tests to use the network. Provision Java 21, Maven, FFmpeg/ffprobe, Python dependencies, and all models through an approved offline mechanism first.

Create a disposable copy of the committed tree, then run Maven offline. This command does not test uncommitted changes:

```bash
export MAVEN_REPO="${MAVEN_REPO:?set MAVEN_REPO to an approved pre-provisioned Maven repository}"
qa_root="$(mktemp -d)"
git archive --format=tar HEAD | tar -xf - -C "$qa_root"
(
  cd "$qa_root"
  env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
    mvn -o -Dmaven.repo.local="$MAVEN_REPO" verify
)
```

Run this only inside an execution environment whose egress is denied; environment variables alone are not a network control. Expect the current clean checkout to fail until local-asset tests stop depending on workstation state. Record that failure rather than relaxing the test.

After Maven completes, reject skipped tests and summarize Surefire results:

```bash
! rg -n 'skipped="[1-9][0-9]*"' "$qa_root"/target/surefire-reports/TEST-*.xml
perl -ne 'if (/<testsuite\b([^>]*)>/) {$a=$1; ($t)=$a=~/\btests="(\d+)"/; ($e)=$a=~/\berrors="(\d+)"/; ($f)=$a=~/\bfailures="(\d+)"/; ($s)=$a=~/\bskipped="(\d+)"/; $T+=$t; $E+=$e; $F+=$f; $S+=$s} END {print "tests=$T errors=$E failures=$F skipped=$S\n"}' "$qa_root"/target/surefire-reports/TEST-*.xml
```

Do not make `216` a permanent threshold. Test count can change; require all intended suites, zero failures/errors/skips, and explicit evidence for each risk.

## Apply the test portfolio

Use the smallest test that can falsify the claim, then add boundary evidence proportional to blast radius.

| Layer | Add it when | Required characteristics | Current gap |
|---|---|---|---|
| Domain unit | A rule, score, state transition, interval, or value object changes | No Spring context; fixed clock/seed; boundary and negative cases; invalid states rejected | Coverage exists but is not mapped to every invariant |
| Architecture | Packages/modules or dependency direction change | Fail build on controller/domain, persistence/API, adapter/core, cycle, and cross-module violations | Entire layer absent |
| Web slice | Controller, DTO, validation, status, or Problem Details changes | Mock only application port; cover success, malformed input, validation, error mapping, bounded collections | Existing tests start full Spring context rather than slices |
| Persistence slice | Mapping, query, constraint, migration, transaction, or concurrency changes | Real PostgreSQL Testcontainer; migrate empty DB; test indexes/constraints, locking, pagination, UTC | Entire layer absent |
| Integration | A real adapter/tool contract changes | Exercise actual FFmpeg, ffprobe, local model, filesystem, PostgreSQL, storage, or queue boundary; parse result | FFmpeg only; other real boundaries absent |
| Contract | Public API, model-worker protocol, object-storage, or queue schema changes | Versioned producer/consumer contract; backward-compatibility gate; failure mapping | Entire layer absent |
| E2E | A critical journey or cross-stage contract changes | Start from approved input; use real intended adapters; assert persisted state, output, telemetry, and failure cleanup | Real highlight source-to-final path absent |

Use the user's suggested minimums as target gates until change control approves evidence-based replacements: 90% line coverage for domain/application logic, 80% overall line coverage, and 80% mutation coverage for critical paths. Report branch coverage as well. Never use coverage to waive missing boundary, failure, concurrency, security, or creative-quality tests.

Never use arbitrary sleeps for asynchronous acceptance. The current `VideoAssetControllerTest` contains bounded `Thread.sleep(50L)` polling; treat replacement with a deterministic executor or condition-based bounded wait as test debt, not a pattern to copy.

## Gate each behavior surface

### Build and dependency gate

Require all of the following before E4:

- Build from a clean checkout with one offline command on macOS and the target Linux runtime.
- Pin and centrally manage dependency/plugin versions; prove no snapshot, milestone, RC, deprecated, or unapproved override.
- Lock or otherwise make dependency resolution reproducible. Record JDK, Maven, OS, architecture, and artifact checksums.
- Produce dependency and Software Bill of Materials (SBOM) evidence. Resolve all critical/high vulnerabilities before release.
- Keep dependency/model acquisition out of application startup and tests.

### API gate

- Validate the OpenAPI document during the build and detect breaking changes.
- Cover every endpoint's success, validation, malformed input, authentication, authorization, not-found/conflict, and dependency-failure responses.
- Assert RFC 9457 media type and stable public fields, including a correlation/trace identifier for diagnosable failures.
- Prove persistence/internal exceptions, paths, keys, stack traces, and secrets never enter responses.
- Bound and test growing collections; test idempotency for retryable writes.

### PostgreSQL gate

- Start empty PostgreSQL and apply every Flyway migration in order.
- Test not-null, unique, foreign-key, and check constraints as database behavior.
- Test repository queries against PostgreSQL, including pagination, indexes, UTC timestamps, isolation, optimistic locking, and concurrent updates.
- Test rolling-deployment migration compatibility and the documented recovery path.
- Never certify production persistence from H2.

### Local-model gate

- Inventory each exact model/runtime with version, checksum, source, license, memory/disk needs, supported platform, and owner.
- Load and infer with egress denied. Fail if a model is absent or its checksum differs; do not download or silently select another model.
- Run representative deterministic or tolerance-bounded fixtures on macOS development and Linux/VPS/container targets.
- Measure task quality, inference latency distribution, warm-up, peak memory, CPU/GPU use, concurrency, timeout, crash recovery, and output determinism.
- Prove output is audible/non-placeholder and semantically matches the request. File existence and nonzero PCM samples are insufficient.
- Block final rendering when requested production music, SFX, or voiceover cannot be produced by an approved local model or licensed asset.

### Media technical gate

Parse actual `ffprobe`/FFmpeg output; never infer output properties from the plan or command string.

| Check | Current repository value or behavior | Promotion discipline |
|---|---|---|
| Container/codecs | Intended render profile names use MP4/H.264/AAC | Assert streams and decode the full output without errors |
| Geometry/rate | Defaults are 1920x1080 at 30 fps | Assert actual width, height, pixel format, frame rate, duration, and frame count against the approved profile |
| Audio | Defaults are 48 kHz and 192 kbit/s | Assert stream presence when required, sample rate, channels, duration, and A/V synchronization |
| Loudness | Mix command configures `I=-16`, `TP=-1.5`, `LRA=11` | Measure the completed file with loudness analysis; command presence is not evidence |
| Black frames | Legacy probe flags any `blackdetect=d=0.5:pic_th=0.98` range as an error | Preserve or revise only with baseline evidence and change control; distinguish intentional fades |
| Long silence | Legacy probe flags `silencedetect=noise=-45dB:d=2` as a warning | Require explicit creative disposition; placeholder or accidental silence is a failure |
| Clipping | Legacy probe flags peak at or above -0.1 dBFS as a warning | Require explicit disposition and reconcile with the -1.5 dB true-peak target |
| Timeline | Legacy in-memory tolerance is 0.05 seconds | Measure final output duration and per-cut alignment; do not compare two plan-derived values |

Establish baselines before setting thresholds for A/V sync, frozen frames, dropped/duplicate frames, encoding quality, scene-boundary accuracy, intelligibility, music/voice balance, and platform transcode survival. Label these thresholds `candidate` until approved.

### Highlight identification and cinematic gate

Build a licensed, diverse corpus spanning the supported categories, motion levels, lighting, speech/no-speech, shot lengths, codecs, resolutions, and negative footage with no worthwhile highlight.

Annotate event boundaries, must-include/must-exclude moments, narrative role, technical defects, and reviewer rationale. Pre-register and compute:

- Candidate recall and precision at the configured maximum highlight count.
- Temporal intersection-over-union and boundary error against annotations.
- Weak/repetitive selected duration and missed high-value duration.
- Diversity/redundancy across selected highlights.
- Category accuracy and confidence calibration where category drives the edit.
- Voiceover factual-grounding errors, text-overlay errors, and inappropriate music/SFX events.

Define a cinematic human-review rubric before evaluating a new approach. Blind and randomize baseline/candidate ordering; use multiple qualified reviewers; retain per-reviewer scores and disagreements. Cover story/hook, shot selection, pacing, continuity, visual treatment, sound design, music fit, voiceover quality and grounding, overlay legibility, emotional coherence, and publishability.

The repository provides no accepted media-specific score thresholds. Establish the baseline, predict target numbers before running, approve thresholds through change control, and then freeze a holdout set. Do not approve by watching only successful examples.

### Security gate

- Test authentication and authorization for every public/sensitive operation, including correct 401 versus 403 behavior.
- Run dependency, static, secret, container, and license scans in CI; release with no unresolved critical/high findings unless formally risk-accepted.
- Verify restrictive CORS, security headers, actuator isolation, input/path validation, archive/media parser limits, and least privilege.
- Test that logs, errors, manifests, prompts, and telemetry redact credentials, tokens, personal data, filesystem internals, and protected media content.
- Exercise the threat model's trust boundaries, especially uploaded media, FFmpeg/model subprocesses, filesystem paths, model/asset supply chain, database, storage, and operator-controlled plans.

### Performance and resilience gate

Treat the user's synchronous API baseline as a candidate until the actual workload is approved: at least 200 requests/second per instance, p95 below 200 ms, p99 below 500 ms, error rate below 0.1%, no continuous memory growth for 60 minutes, and graceful behavior at 2x expected peak. Flag regressions over 10% against the approved baseline.

Do not apply those latency numbers to asynchronous model inference or rendering. Define media-specific SLOs from source minutes, resolution, codec, model/hardware class, concurrency, queue delay, render factor, and output count.

- Record p50/p95/p99 latency, throughput, errors, CPU, memory, GC, threads, disk, database pool, queue age/depth, model memory, and FFmpeg process concurrency.
- Inject slow/unavailable PostgreSQL, storage, queue, model process, disk exhaustion, corrupt media, timeout, worker crash, and termination.
- Prove bounded timeouts/retries/concurrency, idempotency, cleanup, no duplicate/corrupt outputs, graceful shutdown, and recovery.
- Retain raw load input/output and telemetry. Planning baseline documents are not executed load evidence.

### Container, deployment, and operations gate

- Build one platform-neutral application artifact and signed Open Container Initiative (OCI)/runtime/model bundles per supported target. Promote each target bundle unchanged between environments; run non-root with least privilege and a read-only filesystem where practical.
- Scan the image and generate an SBOM/provenance record tied to the commit and model/asset checksums.
- Test liveness, readiness, startup/warm-up, graceful shutdown, resource limits, signals, temporary storage, and no hidden local persistent-state dependency.
- Validate deployment definitions and rolling upgrade/rollback with migrations and in-flight work.
- Prove logs, metrics, traces, health, dashboards, alerts, and runbooks correlate one critical journey without leaking protected data.
- Run smoke/acceptance on Linux/VPS and the intended cloud infrastructure. macOS evidence remains development-only.

## Certified and golden inventory

As of 2026-07-21:

- Certified creative goldens: **none**.
- Certified source-to-final highlight fixtures: **none**.
- Certified local-model artifacts: **none**.
- Certified PostgreSQL/container/deployment/load/security evidence: **none**.
- Technical generated fixtures: real FFmpeg tests generate synthetic video/audio under JUnit temporary directories; these are test inputs, not cinematic goldens.
- Runtime input/output trees and existing rendered files: disposable observations unless separately versioned, licensed, hashed, and approved.

Never silently bless an existing artifact as golden. Golden status requires an approved change with provenance, expected measurements, reviewer, scope, and invalidation conditions.

## Add a fixture or golden

Use this checklist:

1. State the requirement, failure mode, layer, platform scope, and exact claim the fixture can falsify.
2. Obtain legal approval; record source, license, consent/privacy classification, allowed distribution/use, and retention.
3. Minimize the fixture without removing the behavior. Keep secrets, customer media, and personal data out of the repository.
4. Record SHA-256, media/model metadata, generation recipe, deterministic seed/clock, and tool versions.
5. Store small approved fixtures under a purpose-specific `src/test/resources` path. Store large models/media only in an approved versioned artifact system; never auto-download them.
6. Store semantic expectations as structured data. Do not compare transcoded media byte-for-byte when decoded measurements are the contract.
7. Add positive, boundary, negative, malformed, missing-dependency, and cleanup assertions. Write only to JUnit temporary directories.
8. Run with egress denied on macOS and target Linux. Add container/cloud evidence when the claim includes them.
9. Route threshold, fixture, license, and behavior changes through `video-editing-change-control`.
10. Document owner, review date, expiry/invalidation trigger, and the command that reproduces the result.

## Record acceptance evidence

For every promoted change, retain:

```text
Requirement and risk:
Commit and artifact digest:
Evidence tier and scope:
Exact offline command:
OS/architecture/JDK/Maven/FFmpeg/ffprobe versions:
Model and asset versions, checksums, and licenses:
Fixture/golden checksums:
Expected thresholds declared before execution:
Observed raw measurements and artifact locations:
Negative and failure-injection results:
Known limitations:
Independent reviewer and date:
Decision, rollout, rollback, and revisit condition:
```

Do not accept screenshots, prose summaries, or a lone `passed=true` as the only evidence when raw machine-readable output exists.

## Fortune 500 promotion gate

Score each category from 0 (missing) to 4 (reference-quality, verified, reusable): architectural clarity, domain modeling, maintainability, security, test quality, API design, data integrity, resilience, observability, performance, cloud-native operation, CI/CD, developer experience, documentation, and operational readiness.

Require all of the following:

- No category below 3.
- Overall average at least 3.5.
- Security, data integrity, testing, and operational readiness each at least 3.
- Every score cites concrete code, tests, documentation, or automated evidence.
- Clean offline build, complete required checks, API contract, empty-database migrations, telemetry, load/resilience SLOs, secure container, deployment/rollback, ADRs, and absence of undocumented critical assumptions are all proven.

Every listed category is required for project qualification. An unscored category is 0; there is no category-exclusion or “not applicable” route. Route the final promotion through `video-editing-change-control`.

## Provenance and maintenance

This skill was verified against the repository on 2026-07-21. Re-run these one-line, read-only checks when the code or build changes:

- Test files: `find src/test/java -type f -name '*Test.java' | sort`
- Maven plugins/dependencies: `sed -n '1,240p' pom.xml`
- Surefire totals: `perl -ne 'if (/<testsuite\b([^>]*)>/) {$a=$1; ($t)=$a=~/\btests="(\d+)"/; ($e)=$a=~/\berrors="(\d+)"/; ($f)=$a=~/\bfailures="(\d+)"/; ($s)=$a=~/\bskipped="(\d+)"/; $T+=$t; $E+=$e; $F+=$f; $S+=$s} END {print "tests=$T errors=$E failures=$F skipped=$S\n"}' target/surefire-reports/TEST-*.xml`
- Conditional integration tests: `rg -n '@EnabledIf|ffmpeg|ffprobe' src/test/java/org/example/videoclips`
- Coverage scope: `rg -n 'jacoco|folder-coverage-check|COVEREDRATIO|minimum' pom.xml`
- Highlight versus legacy QA: `rg -n 'buildQaReport|runQaProbe|new RenderQaCheck' src/main/java/org/example/videoclips/editing/*Renderer.java`
- Placeholder/model-download paths: `rg -n 'fallbackTone|writeSilence|get_pretrained|strictRuntime|strict-runtime' src/main tools src/test`
- Enterprise gate artifacts: `find . -maxdepth 4 -type f \( -path './.github/*' -o -iname '*openapi*' -o -iname 'Dockerfile*' -o -iname 'compose*.yml' -o -iname '*testcontainer*' -o -iname '*archunit*' -o -iname '*helm*' -o -iname '*k8s*' \) -print`
- Current configuration values: `rg -n 'output-width|output-height|output-frame-rate|audio-sample-rate|audio-bitrate|loudness|strict-runtime|auto-start' src/main/resources src/test/resources`