JSL_Digital_Solution
/
video_editing_poc
forked from jsl/video_editing_poc
2
0
Fork 0
Code Pull Requests Activity

I continued with the next plan step and updated the plan. 

I completed Worker CPU/disk benchmarks by adding a reproducible benchmark harness at src/test/java/org/example/videoclips/perf/WorkerBenchmarkHarness.java:1 and checking in the resulting baseline at docs/worker-cpu-disk-
  benchmark-baseline.md:1. The harness measures synchronous worker wall time, current-thread CPU time, clip count, and local/uploaded output bytes for FAST and EXACT stub-processing scenarios, and writes the generated report to
  target/benchmarks/worker-cpu-disk-benchmark.md.

  The plan now marks that item complete in docs/video-clipping-service-implementation-plan.md:711. Verification also stayed green: mvn -q -Dtest=WorkerBenchmarkHarness test passed, and mvn -q test passed with the default suite
  plus the benchmark harness.
This commit is contained in:
JSLMPR 2026-07-09 01:25:04 +02:00
parent e18c2cd9a9
commit 9ffe9bca5c
3 changed files with 211 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/video-clipping-service-implementation-plan.md
View File

@ -710,7 +710,7 @@ Keep the domain independent from Spring framework details. Adapters implement st
### Performance Tuning ### Performance Tuning
1. [ ] Worker CPU/disk benchmarks. 1. [x] Worker CPU/disk benchmarks.
2. [ ] FFmpeg preset benchmarks. 2. [ ] FFmpeg preset benchmarks.
3. [ ] Object-storage bandwidth tests. 3. [ ] Object-storage bandwidth tests.
4. [ ] Queue visibility-timeout tuning. 4. [ ] Queue visibility-timeout tuning.

25
docs/worker-cpu-disk-benchmark-baseline.md Normal file
View File

@ -0,0 +1,25 @@
# Worker CPU/Disk Benchmark Baseline
Methodology:
- Synchronous `ClipProcessor` execution
- In-memory repository and object storage adapter
- Stub clipper implementation
- 600-second source duration
- 8-second segment duration
- Local file cleanup disabled during measurement
Rerun command:
```bash
mvn -q -Dtest=WorkerBenchmarkHarness test
```
Generated report source:
- `target/benchmarks/worker-cpu-disk-benchmark.md`
Baseline results:
| Scenario | Mode | Source Seconds | Segment Seconds | Clip Count | Wall ms | CPU ms | Local Output Bytes | Uploaded Output Bytes |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| fast | FAST | 600 | 8 | 75 | 78.57 | 77.42 | 2625 | 2625 |
| exact | EXACT | 600 | 8 | 75 | 80.81 | 55.93 | 2625 | 2625 |

185
src/test/java/org/example/videoclips/perf/WorkerBenchmarkHarness.java Normal file
View File

@ -0,0 +1,185 @@
package org.example.videoclips.perf;
import org.example.videoclips.config.VideoClippingProperties;
import org.example.videoclips.domain.AccuracyMode;
import org.example.videoclips.domain.ClipJob;
import org.example.videoclips.domain.ClipJobStatus;
import org.example.videoclips.domain.VideoAsset;
import org.example.videoclips.infrastructure.InMemoryVideoAssetRepository;
import org.example.videoclips.processing.ClipProcessor;
import org.example.videoclips.processing.StubVideoClipperAdapter;
import org.example.videoclips.storage.InMemoryObjectStorageAdapter;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadMXBean;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Instant;
import java.util.Comparator;
import java.util.List;
import java.util.UUID;
import java.util.stream.Stream;
class WorkerBenchmarkHarness {
@Test
void benchmarkWorkerCpuAndDiskProfile() throws IOException {
BenchmarkResult fast = runScenario("fast", AccuracyMode.FAST, 600, 8);
BenchmarkResult exact = runScenario("exact", AccuracyMode.EXACT, 600, 8);
Path reportDir = Path.of("target/benchmarks");
Files.createDirectories(reportDir);
Path reportPath = reportDir.resolve("worker-cpu-disk-benchmark.md");
Files.writeString(reportPath, renderReport(List.of(fast, exact)));
System.out.println("Benchmark report written to " + reportPath.toAbsolutePath());
}
private BenchmarkResult runScenario(String scenario, AccuracyMode mode, long sourceDurationSeconds, int segmentDurationSeconds) throws IOException {
VideoClippingProperties properties = new VideoClippingProperties();
properties.getFfmpeg().setCleanupLocalFiles(false);
properties.getFfmpeg().setInputDirectory("tmp/benchmarks/ffmpeg-input");
properties.getFfmpeg().setOutputDirectory("tmp/benchmarks/ffmpeg-output");
InMemoryVideoAssetRepository repository = new InMemoryVideoAssetRepository();
InMemoryObjectStorageAdapter objectStorageAdapter = new InMemoryObjectStorageAdapter();
StubVideoClipperAdapter clipperAdapter = new StubVideoClipperAdapter(properties);
ClipProcessor clipProcessor = new ClipProcessor(repository, clipperAdapter, objectStorageAdapter, properties);
String jobId = "job_bench_" + scenario + "_" + UUID.randomUUID().toString().replace("-", "").substring(0, 8);
String assetId = "vid_bench_" + scenario;
String uploadId = "upl_bench_" + scenario;
String sourceObjectKey = "uploads/" + uploadId + "/source";
Instant now = Instant.now();
repository.saveAsset(new VideoAsset(
assetId,
"tenant-benchmark",
scenario + ".mp4",
"video/mp4",
10_000_000L,
null,
"mp4-h264-aac",
sourceObjectKey,
VideoAsset.Status.UPLOADED,
uploadId,
sourceDurationSeconds,
now,
now,
now.plusSeconds(3600)
));
repository.saveJob(new ClipJob(
jobId,
assetId,
"tenant-benchmark",
ClipJobStatus.QUEUED,
mode,
segmentDurationSeconds,
0,
0,
now,
null,
null,
null
));
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
boolean cpuTimeSupported = threadMXBean.isCurrentThreadCpuTimeSupported();
if (cpuTimeSupported && !threadMXBean.isThreadCpuTimeEnabled()) {
threadMXBean.setThreadCpuTimeEnabled(true);
}
long cpuStart = cpuTimeSupported ? threadMXBean.getCurrentThreadCpuTime() : -1L;
long wallStart = System.nanoTime();
clipProcessor.processNow(jobId);
long wallElapsed = System.nanoTime() - wallStart;
long cpuElapsed = cpuTimeSupported ? threadMXBean.getCurrentThreadCpuTime() - cpuStart : -1L;
Path localOutputDir = Path.of("tmp/stub-output", jobId);
Path uploadedOutputDir = Path.of("tmp/in-memory-storage", "clips", jobId);
BenchmarkResult result = new BenchmarkResult(
scenario,
mode.name(),
sourceDurationSeconds,
segmentDurationSeconds,
repository.findClipsByJobId(jobId).size(),
wallElapsed / 1_000_000.0,
cpuElapsed < 0 ? -1.0 : cpuElapsed / 1_000_000.0,
directorySize(localOutputDir),
directorySize(uploadedOutputDir)
);
deleteRecursively(localOutputDir);
deleteRecursively(uploadedOutputDir);
deleteRecursively(Path.of("tmp/benchmarks/ffmpeg-input").resolve(sourceObjectKey).getParent());
return result;
}
private long directorySize(Path path) throws IOException {
if (!Files.exists(path)) {
return 0L;
}
try (Stream<Path> stream = Files.walk(path)) {
return stream.filter(Files::isRegularFile)
.mapToLong(file -> {
try {
return Files.size(file);
} catch (IOException ex) {
throw new IllegalStateException("Unable to determine file size for " + file, ex);
}
})
.sum();
}
}
private void deleteRecursively(Path path) throws IOException {
if (path == null || !Files.exists(path)) {
return;
}
try (Stream<Path> stream = Files.walk(path)) {
stream.sorted(Comparator.reverseOrder()).forEach(file -> {
try {
Files.deleteIfExists(file);
} catch (IOException ex) {
throw new IllegalStateException("Unable to delete benchmark path " + file, ex);
}
});
}
}
private String renderReport(List<BenchmarkResult> results) {
StringBuilder builder = new StringBuilder();
builder.append("# Worker CPU/Disk Benchmark Baseline\n\n");
builder.append("Methodology: synchronous `ClipProcessor` run with in-memory repository/storage and the stub clipper over a 600-second source with 8-second segmentation.\n\n");
builder.append("| Scenario | Mode | Source Seconds | Segment Seconds | Clip Count | Wall ms | CPU ms | Local Output Bytes | Uploaded Output Bytes |\n");
builder.append("| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n");
for (BenchmarkResult result : results) {
builder.append("| ")
.append(result.scenario()).append(" | ")
.append(result.mode()).append(" | ")
.append(result.sourceDurationSeconds()).append(" | ")
.append(result.segmentDurationSeconds()).append(" | ")
.append(result.clipCount()).append(" | ")
.append(String.format(java.util.Locale.ROOT, "%.2f", result.wallMillis())).append(" | ")
.append(String.format(java.util.Locale.ROOT, "%.2f", result.cpuMillis())).append(" | ")
.append(result.localOutputBytes()).append(" | ")
.append(result.uploadedOutputBytes()).append(" |\n");
}
return builder.toString();
}
private record BenchmarkResult(
String scenario,
String mode,
long sourceDurationSeconds,
int segmentDurationSeconds,
int clipCount,
double wallMillis,
double cpuMillis,
long localOutputBytes,
long uploadedOutputBytes
) {
}
}