video_editing_poc/.claude/skills/video-editing-validation-and-qa/SKILL.md

name	description
video-editing-validation-and-qa	Load this skill when defining acceptance criteria, adding or reviewing tests, validating API/database/local-model/media behavior, interpreting Maven or render QA results, creating fixtures or goldens, or deciding whether a video-editing change is fit for production promotion.

Video Editing Validation and QA

Use evidence to decide whether this service is correct, cinematic, and operable. Treat a green test run as one input, never as a release certificate.

When not to use this skill

• For symptom-first incident triage, use video-editing-debugging-playbook.
• For commands that inspect a running system or media artifact, use video-editing-diagnostics-and-tooling.
• For change classification, approval, rollout, or rollback, use video-editing-change-control.
• For environment recreation and dependency provisioning, use video-editing-build-and-env.
• For runtime profiles and operator workflows, use video-editing-run-and-operate.
• For media theory and FFmpeg interpretation, use cinematic-media-engineering-reference.
• For the executable highlight-quality program, use video-editing-cinematic-highlights-campaign.
• For first-principles experiments and derivations, use video-editing-proof-and-analysis-toolkit.

Non-negotiable release rule

As of 2026-07-21, do not describe this repository as production-ready, Fortune 500 reference-quality, or creatively certified. Evidence required for those claims is absent.

Fail a gate when evidence is missing, skipped, stale, environment-dependent, or only asserted. Never convert unknown to pass.

video-editing-change-control is authoritative for the seven no-waiver prohibitions; this table defines their validation evidence. video-editing-config-and-flags remains authoritative for defaults.

Prohibited behavior	Required proof
Automatic dependency or model download	Run with network egress denied and pre-provisioned, checksum-pinned dependencies/models. Retain the egress policy and process logs.
External AI service access	Deny egress; verify no external endpoint is configured or called.
Unapproved network access	Prove model/media inference and acquisition use no network, including loopback; verify in-process or approved non-network IPC. Separately test every approved API/database/storage/telemetry connection for security, timeouts, and failure behavior. Maven/process offline flags alone are not egress denial.
Unlicensed music, SFX, fonts, LUTs, footage, or models	Require an asset/model manifest with source, license, version, checksum, and approved usage. Missing provenance blocks use.
Silence, tones, or synthetic stand-ins presented as finished assets	Detect and reject placeholders. Do not accept file existence as asset quality.
Unapproved rendering	Render only approved fixtures in isolated temporary directories until the change-control gate authorizes broader footage or an environment.
Production-facing default changes	Route through video-editing-change-control; test both old and proposed behavior and provide rollback evidence.

The 2026-07-21 working tree removes silence/tone/host-speech fallbacks, restricts AudioCraft to existing local paths with offline flags, and makes strict readiness fail. A zero exit code or output WAV still does not prove semantic fit, license, model identity, loudness, or creative quality; retain final-media and provenance gates.

Evidence tiers

Classify every claim with the highest tier actually earned.

Tier	Name	Minimum evidence	Claims it permits
E0	Assertion	Plan, prompt, comment, checkbox, or unexecuted command	Intent only
E1	Static	Compiles; schema/config/command inspected; deterministic static check passes	Structural property only
E2	Isolated	Deterministic unit, architecture, or adapter test with controlled collaborators	Tested component behavior
E3	Integrated	Real local dependency or executable, representative fixture, machine-read output, no mock at the boundary under test	Integration behavior on the recorded platform
E4	Clean system	Clean checkout, offline/reproducible build, real local models and media tools, source-to-output critical journey	System behavior for the recorded build and environment
E5	Production-like	Security, load, failure injection, observability, container/deploy, and recovery evidence in a production-like environment	Readiness against declared SLOs
E6	Certified	E5 plus independent review, signed evidence, licensed golden inventory, repeatability, and controlled promotion	Reference-quality claim for the tested scope

Do not promote evidence across scopes. A macOS E3 FFmpeg result does not certify Linux/VPS, container, cloud storage, PostgreSQL, or a different FFmpeg/model build.

Current evidence inventory

The following inventory was verified on 2026-07-21.

Area	Current evidence	Honest status
Maven tests	A 2026-07-21 offline mvn -q -o verify run records 245 tests in 62 test classes, 0 failures, 0 errors, 0 skipped in the working tree	E2/E3 mix; not clean-checkout or no-egress certification
Coverage	pom.xml enforces 100% instruction, line, and branch coverage only for org.example.videoclips.folder during verify	No repository-wide threshold; no mutation testing
Real media tools	FolderFfmpegIntegrationTest and CinematicEditingIntegrationTest are conditional on local ffmpeg and ffprobe	Real FFmpeg coverage when available; conditional tests can otherwise skip
Clean build	A clean archived checkout has failed mvn verify at LocalAssetGenerationStageTest because success depends on untracked/runtime-local asset tooling or an audible speech fallback	Release blocker; source-alone reproducibility is unproven
API	18 Spring Boot/MockMvc tests cover selected success, validation, and Problem Details paths	No published/validated OpenAPI, compatibility gate, authentication, or authorization evidence
Database	Flyway migrations and PostgreSQL runtime driver exist; normal tests use local/in-memory behavior and H2 can be selected	No real PostgreSQL Testcontainer repository/migration/concurrency evidence
Architecture	Package separation exists	No ArchUnit or Spring Modulith boundary test; no automated cycle/core-dependency gate
Legacy edit render	Real integration renders generated clips. FfmpegEditRenderer runs black-frame, long-silence, and audio-peak probes; unit tests cover their interpretation	Useful technical E3 evidence; the integration does not assert qaReport.passed() and creative quality is untested
Highlight render	Unit tests cover pieces; HighlightDirectorFlowServiceTest uses a mocked renderer and a dummy file	No real source-to-final highlight E2E
Highlight QA	HighlightFfmpegRenderer.buildQaReport probes output duration, black ranges, long silence, and sample peaks; it also checks assets, plan-level overlay bounds, and the configured mastering command. HighlightDirectorFlowService blocks project-final publication on failed ERROR checks	A fail-closed selected-defect gate, but no integrated-loudness/true-peak, A/V-sync, freeze, raster-safe-area, semantic-fit, or creative acceptance gate
Local models	Adapter/process tests exist	No pinned model inventory, offline no-egress proof, model quality baseline, license proof, or macOS/Linux/container execution matrix
Creative quality	Prompts and plans describe cinematic intent	No certified footage set, annotations, baseline, blinded rubric result, or accepted numeric threshold
Security	Input/error tests exist	No Spring Security, threat-model gate, auth tests, dependency/static/container/secret scans, or release vulnerability report
Performance	Benchmark harnesses and planning baselines exist under src/test/java/.../perf and docs/	No checked-in load generator or executed production-like load signoff
Resilience	Queue/cleanup unit tests and runbooks exist	No fault-injection, shutdown, retry, saturation, or dependency-outage certification
Container/deploy/CI	Operational planning documents exist	No CI definition, OCI image definition, deployment manifest, image scan, SBOM/signing gate, or automated promotion evidence

Do not infer certification from tracked or generated artifacts. docs/*baseline*.md, dashboards, manifests, prompts, qa-report.json, and files under runtime input/output trees become evidence only when their production method, commit, environment, raw measurements, and reviewer are recorded.

The single-source highlight flow now uses RenderQaReport.passed() and failed ERROR checks to block project-final publication and mark the project failed. The legacy multi-clip renderer still treats its QA report as diagnostic output. In both paths, inspect warnings and remember that passing the implemented technical checks does not certify creative quality.

Run the current checks safely

Do not allow Maven, Python, Hugging Face, Torch, FFmpeg inputs, or tests to use the network. Provision Java 21, Maven, FFmpeg/ffprobe, Python dependencies, and all models through an approved offline mechanism first.

Create a disposable copy of the committed tree, then run Maven offline. This command does not test uncommitted changes:

export MAVEN_REPO="${MAVEN_REPO:?set MAVEN_REPO to an approved pre-provisioned Maven repository}"
qa_root="$(mktemp -d)"
git archive --format=tar HEAD | tar -xf - -C "$qa_root"
(
  cd "$qa_root"
  env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
    mvn -o -Dmaven.repo.local="$MAVEN_REPO" verify
)

Run this only inside an execution environment whose egress is denied; environment variables alone are not a network control. Expect the current clean checkout to fail until local-asset tests stop depending on workstation state. Record that failure rather than relaxing the test.

After Maven completes, reject skipped tests and summarize Surefire results:

! rg -n 'skipped="[1-9][0-9]*"' "$qa_root"/target/surefire-reports/TEST-*.xml
perl -ne 'if (/<testsuite\b([^>]*)>/) {$a=$1; ($t)=$a=~/\btests="(\d+)"/; ($e)=$a=~/\berrors="(\d+)"/; ($f)=$a=~/\bfailures="(\d+)"/; ($s)=$a=~/\bskipped="(\d+)"/; $T+=$t; $E+=$e; $F+=$f; $S+=$s} END {print "tests=$T errors=$E failures=$F skipped=$S\n"}' "$qa_root"/target/surefire-reports/TEST-*.xml

Do not make 216 a permanent threshold. Test count can change; require all intended suites, zero failures/errors/skips, and explicit evidence for each risk.

Apply the test portfolio

Use the smallest test that can falsify the claim, then add boundary evidence proportional to blast radius.

Layer	Add it when	Required characteristics	Current gap
Domain unit	A rule, score, state transition, interval, or value object changes	No Spring context; fixed clock/seed; boundary and negative cases; invalid states rejected	Coverage exists but is not mapped to every invariant
Architecture	Packages/modules or dependency direction change	Fail build on controller/domain, persistence/API, adapter/core, cycle, and cross-module violations	Entire layer absent
Web slice	Controller, DTO, validation, status, or Problem Details changes	Mock only application port; cover success, malformed input, validation, error mapping, bounded collections	Existing tests start full Spring context rather than slices
Persistence slice	Mapping, query, constraint, migration, transaction, or concurrency changes	Real PostgreSQL Testcontainer; migrate empty DB; test indexes/constraints, locking, pagination, UTC	Entire layer absent
Integration	A real adapter/tool contract changes	Exercise actual FFmpeg, ffprobe, local model, filesystem, PostgreSQL, storage, or queue boundary; parse result	FFmpeg only; other real boundaries absent
Contract	Public API, model-worker protocol, object-storage, or queue schema changes	Versioned producer/consumer contract; backward-compatibility gate; failure mapping	Entire layer absent
E2E	A critical journey or cross-stage contract changes	Start from approved input; use real intended adapters; assert persisted state, output, telemetry, and failure cleanup	Real highlight source-to-final path absent

Use the user's suggested minimums as target gates until change control approves evidence-based replacements: 90% line coverage for domain/application logic, 80% overall line coverage, and 80% mutation coverage for critical paths. Report branch coverage as well. Never use coverage to waive missing boundary, failure, concurrency, security, or creative-quality tests.

Never use arbitrary sleeps for asynchronous acceptance. The current VideoAssetControllerTest contains bounded Thread.sleep(50L) polling; treat replacement with a deterministic executor or condition-based bounded wait as test debt, not a pattern to copy.

Gate each behavior surface

Build and dependency gate

Require all of the following before E4:

• Build from a clean checkout with one offline command on macOS and the target Linux runtime.
• Pin and centrally manage dependency/plugin versions; prove no snapshot, milestone, RC, deprecated, or unapproved override.
• Lock or otherwise make dependency resolution reproducible. Record JDK, Maven, OS, architecture, and artifact checksums.
• Produce dependency and Software Bill of Materials (SBOM) evidence. Resolve all critical/high vulnerabilities before release.
• Keep dependency/model acquisition out of application startup and tests.

API gate

• Validate the OpenAPI document during the build and detect breaking changes.
• Cover every endpoint's success, validation, malformed input, authentication, authorization, not-found/conflict, and dependency-failure responses.
• Assert RFC 9457 media type and stable public fields, including a correlation/trace identifier for diagnosable failures.
• Prove persistence/internal exceptions, paths, keys, stack traces, and secrets never enter responses.
• Bound and test growing collections; test idempotency for retryable writes.

PostgreSQL gate

• Start empty PostgreSQL and apply every Flyway migration in order.
• Test not-null, unique, foreign-key, and check constraints as database behavior.
• Test repository queries against PostgreSQL, including pagination, indexes, UTC timestamps, isolation, optimistic locking, and concurrent updates.
• Test rolling-deployment migration compatibility and the documented recovery path.
• Never certify production persistence from H2.

Local-model gate

• Inventory each exact model/runtime with version, checksum, source, license, memory/disk needs, supported platform, and owner.
• Load and infer with egress denied. Fail if a model is absent or its checksum differs; do not download or silently select another model.
• Run representative deterministic or tolerance-bounded fixtures on macOS development and Linux/VPS/container targets.
• Measure task quality, inference latency distribution, warm-up, peak memory, CPU/GPU use, concurrency, timeout, crash recovery, and output determinism.
• Prove output is audible/non-placeholder and semantically matches the request. File existence and nonzero PCM samples are insufficient.
• Block final rendering when requested production music, SFX, or voiceover cannot be produced by an approved local model or licensed asset.

Media technical gate

Parse actual ffprobe/FFmpeg output; never infer output properties from the plan or command string.

Check	Current repository value or behavior	Promotion discipline
Container/codecs	Intended render profile names use MP4/H.264/AAC	Assert streams and decode the full output without errors
Geometry/rate	Defaults are 1920x1080 at 30 fps	Assert actual width, height, pixel format, frame rate, duration, and frame count against the approved profile
Audio	Defaults are 48 kHz and 192 kbit/s	Assert stream presence when required, sample rate, channels, duration, and A/V synchronization
Loudness	Mix command configures I=-16, TP=-1.5, LRA=11	Measure the completed file with loudness analysis; command presence is not evidence
Black frames	Legacy probe flags any blackdetect=d=0.5:pic_th=0.98 range as an error	Preserve or revise only with baseline evidence and change control; distinguish intentional fades
Long silence	Legacy probe flags silencedetect=noise=-45dB:d=2 as a warning	Require explicit creative disposition; placeholder or accidental silence is a failure
Clipping	Legacy probe flags peak at or above -0.1 dBFS as a warning	Require explicit disposition and reconcile with the -1.5 dB true-peak target
Timeline	Legacy in-memory tolerance is 0.05 seconds	Measure final output duration and per-cut alignment; do not compare two plan-derived values

Establish baselines before setting thresholds for A/V sync, frozen frames, dropped/duplicate frames, encoding quality, scene-boundary accuracy, intelligibility, music/voice balance, and platform transcode survival. Label these thresholds candidate until approved.

Highlight identification and cinematic gate

Build a licensed, diverse corpus spanning the supported categories, motion levels, lighting, speech/no-speech, shot lengths, codecs, resolutions, and negative footage with no worthwhile highlight.

Annotate event boundaries, must-include/must-exclude moments, narrative role, technical defects, and reviewer rationale. Pre-register and compute:

• Candidate recall and precision at the configured maximum highlight count.
• Temporal intersection-over-union and boundary error against annotations.
• Weak/repetitive selected duration and missed high-value duration.
• Diversity/redundancy across selected highlights.
• Category accuracy and confidence calibration where category drives the edit.
• Voiceover factual-grounding errors, text-overlay errors, and inappropriate music/SFX events.

Define a cinematic human-review rubric before evaluating a new approach. Blind and randomize baseline/candidate ordering; use multiple qualified reviewers; retain per-reviewer scores and disagreements. Cover story/hook, shot selection, pacing, continuity, visual treatment, sound design, music fit, voiceover quality and grounding, overlay legibility, emotional coherence, and publishability.

The repository provides no accepted media-specific score thresholds. Establish the baseline, predict target numbers before running, approve thresholds through change control, and then freeze a holdout set. Do not approve by watching only successful examples.

Security gate

• Test authentication and authorization for every public/sensitive operation, including correct 401 versus 403 behavior.
• Run dependency, static, secret, container, and license scans in CI; release with no unresolved critical/high findings unless formally risk-accepted.
• Verify restrictive CORS, security headers, actuator isolation, input/path validation, archive/media parser limits, and least privilege.
• Test that logs, errors, manifests, prompts, and telemetry redact credentials, tokens, personal data, filesystem internals, and protected media content.
• Exercise the threat model's trust boundaries, especially uploaded media, FFmpeg/model subprocesses, filesystem paths, model/asset supply chain, database, storage, and operator-controlled plans.

Performance and resilience gate

Treat the user's synchronous API baseline as a candidate until the actual workload is approved: at least 200 requests/second per instance, p95 below 200 ms, p99 below 500 ms, error rate below 0.1%, no continuous memory growth for 60 minutes, and graceful behavior at 2x expected peak. Flag regressions over 10% against the approved baseline.

Do not apply those latency numbers to asynchronous model inference or rendering. Define media-specific SLOs from source minutes, resolution, codec, model/hardware class, concurrency, queue delay, render factor, and output count.

• Record p50/p95/p99 latency, throughput, errors, CPU, memory, GC, threads, disk, database pool, queue age/depth, model memory, and FFmpeg process concurrency.
• Inject slow/unavailable PostgreSQL, storage, queue, model process, disk exhaustion, corrupt media, timeout, worker crash, and termination.
• Prove bounded timeouts/retries/concurrency, idempotency, cleanup, no duplicate/corrupt outputs, graceful shutdown, and recovery.
• Retain raw load input/output and telemetry. Planning baseline documents are not executed load evidence.

Container, deployment, and operations gate

• Build one platform-neutral application artifact and signed Open Container Initiative (OCI)/runtime/model bundles per supported target. Promote each target bundle unchanged between environments; run non-root with least privilege and a read-only filesystem where practical.
• Scan the image and generate an SBOM/provenance record tied to the commit and model/asset checksums.
• Test liveness, readiness, startup/warm-up, graceful shutdown, resource limits, signals, temporary storage, and no hidden local persistent-state dependency.
• Validate deployment definitions and rolling upgrade/rollback with migrations and in-flight work.
• Prove logs, metrics, traces, health, dashboards, alerts, and runbooks correlate one critical journey without leaking protected data.
• Run smoke/acceptance on Linux/VPS and the intended cloud infrastructure. macOS evidence remains development-only.

Certified and golden inventory

As of 2026-07-21:

• Certified creative goldens: none.
• Certified source-to-final highlight fixtures: none.
• Certified local-model artifacts: none.
• Certified PostgreSQL/container/deployment/load/security evidence: none.
• Technical generated fixtures: real FFmpeg tests generate synthetic video/audio under JUnit temporary directories; these are test inputs, not cinematic goldens.
• Runtime input/output trees and existing rendered files: disposable observations unless separately versioned, licensed, hashed, and approved.

Never silently bless an existing artifact as golden. Golden status requires an approved change with provenance, expected measurements, reviewer, scope, and invalidation conditions.

Add a fixture or golden

Use this checklist:

1. State the requirement, failure mode, layer, platform scope, and exact claim the fixture can falsify.
2. Obtain legal approval; record source, license, consent/privacy classification, allowed distribution/use, and retention.
3. Minimize the fixture without removing the behavior. Keep secrets, customer media, and personal data out of the repository.
4. Record SHA-256, media/model metadata, generation recipe, deterministic seed/clock, and tool versions.
5. Store small approved fixtures under a purpose-specific src/test/resources path. Store large models/media only in an approved versioned artifact system; never auto-download them.
6. Store semantic expectations as structured data. Do not compare transcoded media byte-for-byte when decoded measurements are the contract.
7. Add positive, boundary, negative, malformed, missing-dependency, and cleanup assertions. Write only to JUnit temporary directories.
8. Run with egress denied on macOS and target Linux. Add container/cloud evidence when the claim includes them.
9. Route threshold, fixture, license, and behavior changes through video-editing-change-control.
10. Document owner, review date, expiry/invalidation trigger, and the command that reproduces the result.

Record acceptance evidence

For every promoted change, retain:

Requirement and risk:
Commit and artifact digest:
Evidence tier and scope:
Exact offline command:
OS/architecture/JDK/Maven/FFmpeg/ffprobe versions:
Model and asset versions, checksums, and licenses:
Fixture/golden checksums:
Expected thresholds declared before execution:
Observed raw measurements and artifact locations:
Negative and failure-injection results:
Known limitations:
Independent reviewer and date:
Decision, rollout, rollback, and revisit condition:

Do not accept screenshots, prose summaries, or a lone passed=true as the only evidence when raw machine-readable output exists.

Fortune 500 promotion gate

Score each category from 0 (missing) to 4 (reference-quality, verified, reusable): architectural clarity, domain modeling, maintainability, security, test quality, API design, data integrity, resilience, observability, performance, cloud-native operation, CI/CD, developer experience, documentation, and operational readiness.

Require all of the following:

• No category below 3.
• Overall average at least 3.5.
• Security, data integrity, testing, and operational readiness each at least 3.
• Every score cites concrete code, tests, documentation, or automated evidence.
• Clean offline build, complete required checks, API contract, empty-database migrations, telemetry, load/resilience SLOs, secure container, deployment/rollback, ADRs, and absence of undocumented critical assumptions are all proven.

Every listed category is required for project qualification. An unscored category is 0; there is no category-exclusion or “not applicable” route. Route the final promotion through video-editing-change-control.

Provenance and maintenance

This skill was verified against the repository on 2026-07-21. Re-run these one-line, read-only checks when the code or build changes:

• Test files: find src/test/java -type f -name '*Test.java' | sort
• Maven plugins/dependencies: sed -n '1,240p' pom.xml
• Surefire totals: perl -ne 'if (/<testsuite\b([^>]*)>/) {$a=$1; ($t)=$a=~/\btests="(\d+)"/; ($e)=$a=~/\berrors="(\d+)"/; ($f)=$a=~/\bfailures="(\d+)"/; ($s)=$a=~/\bskipped="(\d+)"/; $T+=$t; $E+=$e; $F+=$f; $S+=$s} END {print "tests=$T errors=$E failures=$F skipped=$S\n"}' target/surefire-reports/TEST-*.xml
• Conditional integration tests: rg -n '@EnabledIf|ffmpeg|ffprobe' src/test/java/org/example/videoclips
• Coverage scope: rg -n 'jacoco|folder-coverage-check|COVEREDRATIO|minimum' pom.xml
• Highlight versus legacy QA: rg -n 'buildQaReport|runQaProbe|new RenderQaCheck' src/main/java/org/example/videoclips/editing/*Renderer.java
• Placeholder/model-download paths: rg -n 'fallbackTone|writeSilence|get_pretrained|strictRuntime|strict-runtime' src/main tools src/test
• Enterprise gate artifacts: find . -maxdepth 4 -type f \( -path './.github/*' -o -iname '*openapi*' -o -iname 'Dockerfile*' -o -iname 'compose*.yml' -o -iname '*testcontainer*' -o -iname '*archunit*' -o -iname '*helm*' -o -iname '*k8s*' \) -print
• Current configuration values: rg -n 'output-width|output-height|output-frame-rate|audio-sample-rate|audio-bitrate|loudness|strict-runtime|auto-start' src/main/resources src/test/resources