forked from jsl/video_editing_poc
• The next plan item is done and the plan is updated. Object-storage bandwidth tests is now marked complete in docs/video-clipping-service-implementation-plan.md:713.
I added a reproducible benchmark harness in src/test/java/org/example/videoclips/perf/ObjectStorageBandwidthBenchmarkHarness.java:17 that exercises ObjectStoragePort materialization and upload paths across a local-disk baseline plus throttled 100/50/25 MiB/s profiles. I also checked in the measured baseline and rerun command in docs/object-storage-bandwidth-benchmark-baseline.md:1. On this workspace, the measured throughput came out at roughly 1282/825 MiB/s for local disk, 93/89 MiB/s for the 100 MiB/s cap, 48/47 MiB/s for 50 MiB/s, and 24/24 MiB/s for 25 MiB/s on materialize/upload respectively.
This commit is contained in:
JSLMPR
parent
9ffe9bca5c
commit
f46055c972
|
|
@ -0,0 +1,28 @@
|
|||
# FFmpeg Preset Benchmark Baseline
|
||||
|
||||
Methodology:
|
||||
- Real `ffmpeg` execution through `FfmpegVideoClipperAdapter`
|
||||
- Synthetic 16-second 1280x720 input generated with `testsrc`
|
||||
- `EXACT` mode with 8-second segmentation
|
||||
- Presets compared: `ultrafast`, `veryfast`, `medium`
|
||||
|
||||
Rerun command:
|
||||
|
||||
```bash
|
||||
mvn -q -Dtest=FfmpegPresetBenchmarkHarness test
|
||||
```
|
||||
|
||||
Generated report source:
|
||||
- `target/benchmarks/ffmpeg-preset-benchmark.md`
|
||||
|
||||
Baseline results:
|
||||
|
||||
| Preset | Clip Count | Wall ms | Output Bytes |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| ultrafast | 2 | 1610.05 | 1098215 |
|
||||
| veryfast | 1 | 1892.61 | 507360 |
|
||||
| medium | 1 | 2507.86 | 533768 |
|
||||
|
||||
Notes:
|
||||
- This benchmark is intended as a reproducible local baseline, not a production throughput claim.
|
||||
- The current adapter manifest logic is synthetic; this baseline measures actual output files written by ffmpeg.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
# Object Storage Bandwidth Benchmark Baseline
|
||||
|
||||
This document captures the reproducible local baseline for performance plan item `Object-storage bandwidth tests`.
|
||||
|
||||
Methodology:
|
||||
|
||||
- Benchmark exercises the `ObjectStoragePort` transfer paths directly.
|
||||
- Source-object materialization downloads one `64 MiB` object into worker input storage.
|
||||
- Generated clip upload writes four clips totaling `32 MiB` back into object storage.
|
||||
- The baseline uses a local file-copy adapter for raw disk throughput and a throttling layer to simulate object-storage caps without requiring a live S3 environment.
|
||||
|
||||
How to rerun:
|
||||
|
||||
```bash
|
||||
mvn -q -Dtest=ObjectStorageBandwidthBenchmarkHarness test
|
||||
```
|
||||
|
||||
Generated artifact:
|
||||
|
||||
- `target/benchmarks/object-storage-bandwidth-benchmark.md`
|
||||
|
||||
Interpretation notes:
|
||||
|
||||
- The `local-disk` row shows the upper bound of the benchmark on the current machine.
|
||||
- The throttled rows approximate how worker staging and clip upload latency degrade as available storage bandwidth drops.
|
||||
- This benchmark is intended as a reproducible local planning baseline, not a production throughput claim.
|
||||
|
||||
Measured baseline on this workspace:
|
||||
|
||||
| Profile | Source MiB | Uploaded MiB | Materialize ms | Upload ms | Materialize MiB/s | Upload MiB/s |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| `local-disk` | 64 | 32 | 49.91 | 38.78 | 1282.23 | 825.19 |
|
||||
| `100 MiB/s` | 64 | 32 | 681.70 | 359.21 | 93.88 | 89.08 |
|
||||
| `50 MiB/s` | 64 | 32 | 1323.63 | 678.67 | 48.35 | 47.15 |
|
||||
| `25 MiB/s` | 64 | 32 | 2610.05 | 1322.83 | 24.52 | 24.19 |
|
||||
|
|
@ -711,8 +711,8 @@ Keep the domain independent from Spring framework details. Adapters implement st
|
|||
### Performance Tuning
|
||||
|
||||
1. [x] Worker CPU/disk benchmarks.
|
||||
2. [ ] FFmpeg preset benchmarks.
|
||||
3. [ ] Object-storage bandwidth tests.
|
||||
2. [x] FFmpeg preset benchmarks.
|
||||
3. [x] Object-storage bandwidth tests.
|
||||
4. [ ] Queue visibility-timeout tuning.
|
||||
5. [ ] Autoscaling policies.
|
||||
6. [ ] Cost model by video minute.
|
||||
|
|
|
|||
|
|
@ -213,6 +213,7 @@ public class VideoClippingProperties {
|
|||
private String ffmpegBinary = "ffmpeg";
|
||||
private String inputDirectory = "./tmp/ffmpeg-input";
|
||||
private String outputDirectory = "./tmp/ffmpeg-output";
|
||||
private String exactPreset = "veryfast";
|
||||
private boolean cleanupLocalFiles = true;
|
||||
|
||||
public String getFfmpegBinary() {
|
||||
|
|
@ -239,6 +240,14 @@ public class VideoClippingProperties {
|
|||
this.outputDirectory = outputDirectory;
|
||||
}
|
||||
|
||||
public String getExactPreset() {
|
||||
return exactPreset;
|
||||
}
|
||||
|
||||
public void setExactPreset(String exactPreset) {
|
||||
this.exactPreset = exactPreset;
|
||||
}
|
||||
|
||||
public boolean isCleanupLocalFiles() {
|
||||
return cleanupLocalFiles;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -87,7 +87,7 @@ public class FfmpegVideoClipperAdapter implements VideoClipperPort {
|
|||
command.add("-c:v");
|
||||
command.add("libx264");
|
||||
command.add("-preset");
|
||||
command.add("veryfast");
|
||||
command.add(properties.getFfmpeg().getExactPreset());
|
||||
command.add("-crf");
|
||||
command.add("20");
|
||||
command.add("-force_key_frames");
|
||||
|
|
|
|||
|
|
@ -0,0 +1,148 @@
|
|||
package org.example.videoclips.perf;
|
||||
|
||||
import org.example.videoclips.config.VideoClippingProperties;
|
||||
import org.example.videoclips.domain.AccuracyMode;
|
||||
import org.example.videoclips.processing.FfmpegVideoClipperAdapter;
|
||||
import org.example.videoclips.processing.GeneratedClip;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Comparator;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
class FfmpegPresetBenchmarkHarness {
|
||||
|
||||
@Test
|
||||
void benchmarkExactModePresets() throws IOException {
|
||||
List<String> presets = List.of("ultrafast", "veryfast", "medium");
|
||||
Path reportDir = Path.of("target/benchmarks");
|
||||
Files.createDirectories(reportDir);
|
||||
|
||||
Path sharedRoot = Path.of("tmp/benchmarks/ffmpeg-presets");
|
||||
Path inputDir = sharedRoot.resolve("input");
|
||||
Path outputDir = sharedRoot.resolve("output");
|
||||
Files.createDirectories(inputDir);
|
||||
Files.createDirectories(outputDir);
|
||||
|
||||
String sourceObjectKey = "uploads/bench-preset/source.mp4";
|
||||
Path inputFile = inputDir.resolve(sourceObjectKey);
|
||||
Files.createDirectories(inputFile.getParent());
|
||||
createSyntheticInput(inputFile);
|
||||
|
||||
StringBuilder builder = new StringBuilder();
|
||||
builder.append("# FFmpeg Preset Benchmark Baseline\n\n");
|
||||
builder.append("Methodology:\n");
|
||||
builder.append("- Real `ffmpeg` execution through `FfmpegVideoClipperAdapter`\n");
|
||||
builder.append("- Synthetic 16-second 1280x720 input generated with `testsrc`\n");
|
||||
builder.append("- `EXACT` mode with 8-second segmentation\n");
|
||||
builder.append("- Presets compared: `ultrafast`, `veryfast`, `medium`\n\n");
|
||||
builder.append("| Preset | Clip Count | Wall ms | Output Bytes |\n");
|
||||
builder.append("| --- | ---: | ---: | ---: |\n");
|
||||
|
||||
for (String preset : presets) {
|
||||
BenchmarkResult result = runPresetBenchmark(preset, inputDir, outputDir, sourceObjectKey);
|
||||
builder.append("| ")
|
||||
.append(result.preset()).append(" | ")
|
||||
.append(result.clipCount()).append(" | ")
|
||||
.append(String.format(Locale.ROOT, "%.2f", result.wallMillis())).append(" | ")
|
||||
.append(result.outputBytes()).append(" |\n");
|
||||
}
|
||||
|
||||
Path generatedReport = reportDir.resolve("ffmpeg-preset-benchmark.md");
|
||||
Files.writeString(generatedReport, builder.toString());
|
||||
System.out.println("FFmpeg preset benchmark report written to " + generatedReport.toAbsolutePath());
|
||||
|
||||
deleteRecursively(sharedRoot);
|
||||
}
|
||||
|
||||
private BenchmarkResult runPresetBenchmark(String preset, Path inputDir, Path outputDir, String sourceObjectKey) throws IOException {
|
||||
VideoClippingProperties properties = new VideoClippingProperties();
|
||||
properties.getFfmpeg().setFfmpegBinary("ffmpeg");
|
||||
properties.getFfmpeg().setInputDirectory(inputDir.toString());
|
||||
properties.getFfmpeg().setOutputDirectory(outputDir.toString());
|
||||
properties.getFfmpeg().setExactPreset(preset);
|
||||
|
||||
FfmpegVideoClipperAdapter adapter = new FfmpegVideoClipperAdapter(properties);
|
||||
String jobId = "job_preset_" + preset;
|
||||
Path scenarioOutputDir = outputDir.resolve(jobId);
|
||||
|
||||
deleteRecursively(scenarioOutputDir);
|
||||
|
||||
long wallStart = System.nanoTime();
|
||||
adapter.generateClips(jobId, "asset-bench", sourceObjectKey, 16, 8, AccuracyMode.EXACT);
|
||||
long wallElapsed = System.nanoTime() - wallStart;
|
||||
|
||||
long clipCount;
|
||||
long outputBytes;
|
||||
try (Stream<Path> stream = Files.list(scenarioOutputDir)) {
|
||||
List<Path> outputs = stream.filter(Files::isRegularFile).sorted().toList();
|
||||
clipCount = outputs.size();
|
||||
outputBytes = outputs.stream()
|
||||
.mapToLong(path -> {
|
||||
try {
|
||||
return Files.size(path);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to read output clip size", ex);
|
||||
}
|
||||
})
|
||||
.sum();
|
||||
}
|
||||
|
||||
deleteRecursively(scenarioOutputDir);
|
||||
return new BenchmarkResult(preset, (int) clipCount, wallElapsed / 1_000_000.0, outputBytes);
|
||||
}
|
||||
|
||||
private void createSyntheticInput(Path inputFile) throws IOException {
|
||||
deleteRecursively(inputFile.getParent());
|
||||
Files.createDirectories(inputFile.getParent());
|
||||
ProcessBuilder processBuilder = new ProcessBuilder(
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-y",
|
||||
"-f", "lavfi",
|
||||
"-i", "testsrc=size=1280x720:rate=30",
|
||||
"-f", "lavfi",
|
||||
"-i", "sine=frequency=1000:sample_rate=48000",
|
||||
"-t", "16",
|
||||
"-c:v", "libx264",
|
||||
"-preset", "ultrafast",
|
||||
"-pix_fmt", "yuv420p",
|
||||
"-c:a", "aac",
|
||||
inputFile.toString()
|
||||
);
|
||||
processBuilder.redirectErrorStream(true);
|
||||
try {
|
||||
Process process = processBuilder.start();
|
||||
String output = new String(process.getInputStream().readAllBytes());
|
||||
int exitCode = process.waitFor();
|
||||
if (exitCode != 0) {
|
||||
throw new IllegalStateException("Unable to generate synthetic ffmpeg input: " + output);
|
||||
}
|
||||
} catch (InterruptedException ex) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw new IllegalStateException("Synthetic ffmpeg input generation interrupted", ex);
|
||||
}
|
||||
}
|
||||
|
||||
private void deleteRecursively(Path path) throws IOException {
|
||||
if (path == null || !Files.exists(path)) {
|
||||
return;
|
||||
}
|
||||
try (Stream<Path> stream = Files.walk(path)) {
|
||||
stream.sorted(Comparator.reverseOrder()).forEach(file -> {
|
||||
try {
|
||||
Files.deleteIfExists(file);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to delete benchmark path " + file, ex);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private record BenchmarkResult(String preset, int clipCount, double wallMillis, long outputBytes) {
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,288 @@
|
|||
package org.example.videoclips.perf;
|
||||
|
||||
import org.example.videoclips.storage.ObjectStoragePort;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.StandardCopyOption;
|
||||
import java.time.Instant;
|
||||
import java.util.Comparator;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.concurrent.locks.LockSupport;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
class ObjectStorageBandwidthBenchmarkHarness {
|
||||
|
||||
private static final int MEBIBYTE = 1024 * 1024;
|
||||
|
||||
@Test
|
||||
void benchmarkObjectStorageBandwidthProfiles() throws IOException {
|
||||
Path reportDir = Path.of("target/benchmarks");
|
||||
Files.createDirectories(reportDir);
|
||||
|
||||
Path sharedRoot = Path.of("tmp/benchmarks/object-storage-bandwidth");
|
||||
deleteRecursively(sharedRoot);
|
||||
Files.createDirectories(sharedRoot);
|
||||
|
||||
Path storageRoot = sharedRoot.resolve("storage");
|
||||
Path workerInputRoot = sharedRoot.resolve("worker-input");
|
||||
Path workerClipRoot = sharedRoot.resolve("worker-clips");
|
||||
Files.createDirectories(storageRoot);
|
||||
Files.createDirectories(workerInputRoot);
|
||||
Files.createDirectories(workerClipRoot);
|
||||
|
||||
String sourceObjectKey = "uploads/bandwidth/source.bin";
|
||||
Path sourceObject = storageRoot.resolve(sourceObjectKey);
|
||||
createDeterministicFile(sourceObject, 64L * MEBIBYTE, (byte) 0x11);
|
||||
|
||||
List<Path> generatedClips = List.of(
|
||||
workerClipRoot.resolve("clip-01.mp4"),
|
||||
workerClipRoot.resolve("clip-02.mp4"),
|
||||
workerClipRoot.resolve("clip-03.mp4"),
|
||||
workerClipRoot.resolve("clip-04.mp4")
|
||||
);
|
||||
for (int i = 0; i < generatedClips.size(); i++) {
|
||||
createDeterministicFile(generatedClips.get(i), 8L * MEBIBYTE, (byte) (0x20 + i));
|
||||
}
|
||||
|
||||
List<Profile> profiles = List.of(
|
||||
new Profile("local-disk", 0),
|
||||
new Profile("100 MiB/s", 100),
|
||||
new Profile("50 MiB/s", 50),
|
||||
new Profile("25 MiB/s", 25)
|
||||
);
|
||||
|
||||
List<BenchmarkResult> results = profiles.stream()
|
||||
.map(profile -> runBenchmark(profile, storageRoot, workerInputRoot, sourceObjectKey, generatedClips))
|
||||
.toList();
|
||||
|
||||
Path reportPath = reportDir.resolve("object-storage-bandwidth-benchmark.md");
|
||||
Files.writeString(reportPath, renderReport(results, sourceObject, generatedClips));
|
||||
System.out.println("Object-storage bandwidth benchmark report written to " + reportPath.toAbsolutePath());
|
||||
|
||||
deleteRecursively(sharedRoot);
|
||||
}
|
||||
|
||||
private BenchmarkResult runBenchmark(
|
||||
Profile profile,
|
||||
Path storageRoot,
|
||||
Path workerInputRoot,
|
||||
String sourceObjectKey,
|
||||
List<Path> generatedClips
|
||||
) {
|
||||
ObjectStoragePort storage = profile.bandwidthMiBPerSecond() > 0
|
||||
? new ThrottledObjectStorageAdapter(storageRoot, profile.bandwidthMiBPerSecond())
|
||||
: new FileCopyObjectStorageAdapter(storageRoot);
|
||||
|
||||
Path localInputPath = workerInputRoot.resolve(profile.slug()).resolve(sourceObjectKey);
|
||||
try {
|
||||
deleteRecursively(localInputPath.getParent());
|
||||
Files.createDirectories(localInputPath.getParent());
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to prepare worker input directory", ex);
|
||||
}
|
||||
|
||||
long sourceBytes = sizeOf(storageRoot.resolve(sourceObjectKey));
|
||||
long uploadedBytes = generatedClips.stream().mapToLong(this::sizeOf).sum();
|
||||
|
||||
long materializeStart = System.nanoTime();
|
||||
storage.materializeSourceObject(sourceObjectKey, localInputPath);
|
||||
long materializeElapsed = System.nanoTime() - materializeStart;
|
||||
|
||||
long uploadStart = System.nanoTime();
|
||||
for (int i = 0; i < generatedClips.size(); i++) {
|
||||
storage.uploadGeneratedClip(generatedClips.get(i), "clips/" + profile.slug() + "/clip-" + (i + 1) + ".mp4");
|
||||
}
|
||||
long uploadElapsed = System.nanoTime() - uploadStart;
|
||||
|
||||
return new BenchmarkResult(
|
||||
profile.label(),
|
||||
sourceBytes,
|
||||
uploadedBytes,
|
||||
materializeElapsed / 1_000_000.0,
|
||||
uploadElapsed / 1_000_000.0,
|
||||
throughputMiBPerSecond(sourceBytes, materializeElapsed),
|
||||
throughputMiBPerSecond(uploadedBytes, uploadElapsed)
|
||||
);
|
||||
}
|
||||
|
||||
private double throughputMiBPerSecond(long bytes, long elapsedNanos) {
|
||||
if (elapsedNanos <= 0L) {
|
||||
return 0.0;
|
||||
}
|
||||
double seconds = elapsedNanos / 1_000_000_000.0;
|
||||
return (bytes / (double) MEBIBYTE) / seconds;
|
||||
}
|
||||
|
||||
private long sizeOf(Path path) {
|
||||
try {
|
||||
return Files.size(path);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to determine file size for " + path, ex);
|
||||
}
|
||||
}
|
||||
|
||||
private void createDeterministicFile(Path path, long sizeBytes, byte seed) throws IOException {
|
||||
Files.createDirectories(path.getParent());
|
||||
byte[] block = new byte[MEBIBYTE];
|
||||
for (int i = 0; i < block.length; i++) {
|
||||
block[i] = (byte) (seed + i);
|
||||
}
|
||||
try (var outputStream = Files.newOutputStream(path)) {
|
||||
long remaining = sizeBytes;
|
||||
while (remaining > 0) {
|
||||
int bytesToWrite = (int) Math.min(block.length, remaining);
|
||||
outputStream.write(block, 0, bytesToWrite);
|
||||
remaining -= bytesToWrite;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void deleteRecursively(Path path) throws IOException {
|
||||
if (path == null || !Files.exists(path)) {
|
||||
return;
|
||||
}
|
||||
try (Stream<Path> stream = Files.walk(path)) {
|
||||
stream.sorted(Comparator.reverseOrder()).forEach(file -> {
|
||||
try {
|
||||
Files.deleteIfExists(file);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to delete benchmark path " + file, ex);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private String renderReport(List<BenchmarkResult> results, Path sourceObject, List<Path> generatedClips) {
|
||||
long sourceMiB = sizeOf(sourceObject) / MEBIBYTE;
|
||||
long totalClipMiB = generatedClips.stream().mapToLong(this::sizeOf).sum() / MEBIBYTE;
|
||||
|
||||
StringBuilder builder = new StringBuilder();
|
||||
builder.append("# Object Storage Bandwidth Benchmark Baseline\n\n");
|
||||
builder.append("Methodology:\n");
|
||||
builder.append("- Local file-copy adapter used as the storage baseline for the `ObjectStoragePort`\n");
|
||||
builder.append("- Optional throttling layer simulates capped object-storage throughput without a live S3 dependency\n");
|
||||
builder.append("- Source materialization downloads one ").append(sourceMiB).append(" MiB object into worker input storage\n");
|
||||
builder.append("- Generated clip upload writes four clips totaling ").append(totalClipMiB).append(" MiB back to object storage\n\n");
|
||||
builder.append("| Profile | Source MiB | Uploaded MiB | Materialize ms | Upload ms | Materialize MiB/s | Upload MiB/s |\n");
|
||||
builder.append("| --- | ---: | ---: | ---: | ---: | ---: | ---: |\n");
|
||||
for (BenchmarkResult result : results) {
|
||||
builder.append("| ")
|
||||
.append(result.profile()).append(" | ")
|
||||
.append(result.sourceBytes() / MEBIBYTE).append(" | ")
|
||||
.append(result.uploadedBytes() / MEBIBYTE).append(" | ")
|
||||
.append(String.format(Locale.ROOT, "%.2f", result.materializeMillis())).append(" | ")
|
||||
.append(String.format(Locale.ROOT, "%.2f", result.uploadMillis())).append(" | ")
|
||||
.append(String.format(Locale.ROOT, "%.2f", result.materializeMiBPerSecond())).append(" | ")
|
||||
.append(String.format(Locale.ROOT, "%.2f", result.uploadMiBPerSecond())).append(" |\n");
|
||||
}
|
||||
return builder.toString();
|
||||
}
|
||||
|
||||
private record Profile(String label, int bandwidthMiBPerSecond) {
|
||||
private String slug() {
|
||||
return label.toLowerCase(Locale.ROOT).replace(" ", "-").replace("/", "-");
|
||||
}
|
||||
}
|
||||
|
||||
private record BenchmarkResult(
|
||||
String profile,
|
||||
long sourceBytes,
|
||||
long uploadedBytes,
|
||||
double materializeMillis,
|
||||
double uploadMillis,
|
||||
double materializeMiBPerSecond,
|
||||
double uploadMiBPerSecond
|
||||
) {
|
||||
}
|
||||
|
||||
private static class FileCopyObjectStorageAdapter implements ObjectStoragePort {
|
||||
|
||||
protected final Path storageRoot;
|
||||
|
||||
private FileCopyObjectStorageAdapter(Path storageRoot) {
|
||||
this.storageRoot = storageRoot;
|
||||
}
|
||||
|
||||
@Override
|
||||
public UploadSessionDescriptor createMultipartUpload(String uploadId, int partCount, long partSizeBytes, Instant expiresAt) {
|
||||
throw new UnsupportedOperationException("Not needed for benchmark");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void completeMultipartUpload(String providerUploadId, String uploadId, List<CompletedUploadPart> parts) {
|
||||
throw new UnsupportedOperationException("Not needed for benchmark");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void materializeSourceObject(String sourceObjectKey, Path targetPath) {
|
||||
copy(storageRoot.resolve(sourceObjectKey), targetPath);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void uploadGeneratedClip(Path sourcePath, String objectKey) {
|
||||
copy(sourcePath, storageRoot.resolve(objectKey));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void deleteObject(String objectKey) {
|
||||
throw new UnsupportedOperationException("Not needed for benchmark");
|
||||
}
|
||||
|
||||
@Override
|
||||
public String createClipDownloadUrl(String objectKey, Instant expiresAt) {
|
||||
throw new UnsupportedOperationException("Not needed for benchmark");
|
||||
}
|
||||
|
||||
private void copy(Path source, Path target) {
|
||||
try {
|
||||
Files.createDirectories(target.getParent());
|
||||
Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to copy benchmark object", ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static final class ThrottledObjectStorageAdapter extends FileCopyObjectStorageAdapter {
|
||||
|
||||
private final int bandwidthMiBPerSecond;
|
||||
|
||||
private ThrottledObjectStorageAdapter(Path storageRoot, int bandwidthMiBPerSecond) {
|
||||
super(storageRoot);
|
||||
this.bandwidthMiBPerSecond = bandwidthMiBPerSecond;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void materializeSourceObject(String sourceObjectKey, Path targetPath) {
|
||||
long bytes = sizeOfUnchecked(super.storageRoot.resolve(sourceObjectKey));
|
||||
super.materializeSourceObject(sourceObjectKey, targetPath);
|
||||
throttle(bytes);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void uploadGeneratedClip(Path sourcePath, String objectKey) {
|
||||
long bytes = sizeOfUnchecked(sourcePath);
|
||||
super.uploadGeneratedClip(sourcePath, objectKey);
|
||||
throttle(bytes);
|
||||
}
|
||||
|
||||
private void throttle(long bytes) {
|
||||
long nanos = (long) ((bytes / (double) MEBIBYTE) / bandwidthMiBPerSecond * 1_000_000_000L);
|
||||
if (nanos > 0L) {
|
||||
LockSupport.parkNanos(nanos);
|
||||
}
|
||||
}
|
||||
|
||||
private long sizeOfUnchecked(Path path) {
|
||||
try {
|
||||
return Files.size(path);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Unable to determine benchmark object size", ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue